tutorials:file_management
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| tutorials:file_management [2023/08/21 00:47] – [File Management] chkuo | tutorials:file_management [2026/09/17 11:20] (current) – chkuo | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== File Management ====== | ====== File Management ====== | ||
| - | * Guidelines | + | * Guidelines that Chih-Horng Kuo (chk@gate.sinica.edu.tw) developed for our group members. Suggestions are welcome. |
| * Related information: | * Related information: | ||
| * [[tutorials: | * [[tutorials: | ||
| * [[tutorials: | * [[tutorials: | ||
| - | |||
| ===== Backup ===== | ===== Backup ===== | ||
| * <color # | * <color # | ||
| - | * You will experience file loss. Not __IF__, but __WHEN__. | + | * File loss happens. Not __IF__, but __WHEN__. |
| * Establish an automatic backup plan; manual backup plans are not practical | * Establish an automatic backup plan; manual backup plans are not practical | ||
| - | | + | |
| - | * Linux: rsync scripts | + | * Linux: |
| + | * Online tools (e.g., Google Drive, Dropbox) | ||
| + | * Easy to use; cost and privacy concern; may or may not be a true backup | ||
| + | * Solution for teams (beyond individuals): | ||
| ===== Working with collaborators ===== | ===== Working with collaborators ===== | ||
| * Discuss and setup a workflow | * Discuss and setup a workflow | ||
| * Avoid emailing files as attachments; | * Avoid emailing files as attachments; | ||
| - | * Pros and cons of setting | + | * Setting |
| + | * File server: | ||
| + | * Pros: Full control, large storage, no vendor lock-in | ||
| + | * Cons: Requires hardware, set up, and maintenance | ||
| + | * File sharing service | ||
| + | * Pros: Easy, low cost | ||
| + | * Cons: Limited space, less control over structure, potential data leakage. | ||
| ===== File directory structure ===== | ===== File directory structure ===== | ||
| * Make it easy to identify the parts that need to be included in backup | * Make it easy to identify the parts that need to be included in backup | ||
| * Example for personal desktops/ | * Example for personal desktops/ | ||
| + | * A few top level folders with clear naming for easy backup/ | ||
| * Example for shared servers | * Example for shared servers | ||
| + | * Depends on the research group, for example: data/, lab_doc/, project/, conference/ | ||
| + | * Clear rules, communication, | ||
| + | * A proper directory hierarchy helps to keep files well-organized, | ||
| ===== Naming and version control ===== | ===== Naming and version control ===== | ||
| * File names | * File names | ||
| - | * Do: short and informative. Include information such as project id, keyword (e.g. manuscript, report, keyword of figure/ | + | * Do: short and informative. Include information such as project id, keyword (e.g. manuscript, report, keyword of figure/ |
| - | * Avoid: | + | * Proper length is affected by directory structure; files in multi-level directories tend to have shorter names |
| - | * Avoid: short names that are not informative (e.g., manuscript.docx, | + | * If the files (but not the directory structure) are to be shared, proper length may change. For example, " |
| - | * Avoid: space or special characters | + | * Avoid: |
| - | * Avoid: | + | * Long names that are difficult to read (e.g., full title of the manuscript) |
| + | * Short names that are not informative (e.g., manuscript.docx, | ||
| + | * Space or special characters | ||
| + | * Names such as " | ||
| * Versioning | * Versioning | ||
| - | * Version number should be the last part of the file name; two digits should be sufficient | + | * Version number should be the last part of the file name |
| + | * Usually, | ||
| + | * Alternatively, | ||
| * Optional: Add initials if multiple people are involved in the project | * Optional: Add initials if multiple people are involved in the project | ||
| + | * Optional: Add additional info; for example, " | ||
| * When in doubt, save as a new version | * When in doubt, save as a new version | ||
| * Recommended: | * Recommended: | ||
| Line 58: | Line 76: | ||
| * After finalizing the order, put all early versions into a separate folder (e.g., " | * After finalizing the order, put all early versions into a separate folder (e.g., " | ||
| * If the order changed, put the previous versions into another separate folder (e.g., " | * If the order changed, put the previous versions into another separate folder (e.g., " | ||
| + | * Very important to save the files in editable formats (e.g., .ai, not just .jpg) | ||
| ===== Tracking changes ===== | ===== Tracking changes ===== | ||
| - | * For Word files: better to use the build-in function "Track Changes" | + | * Microsoft |
| - | * For Google Docs: all changes are automatically tracked. Manually | + | * Use the build-in function "Track Changes" |
| + | * Possible | ||
| + | * Google Docs | ||
| + | * All changes are automatically | ||
| + | * More streamlined operation, may be difficult to track when there are many changes by multiple users. | ||
| + | * Suggestion: | ||
| + | ===== Metadata ===== | ||
| + | * Metadata = data that explains your files: what they are, where they came from, how to use them. | ||
| + | * Examples | ||
| + | * For the raw data file of a gel image (20250509_1428.jpg), | ||
| + | * For a set of raw data files (e.g., Sanger sequencing results), or a file folder, provide a " | ||
tutorials/file_management.1692550038.txt.gz · Last modified: by chkuo