Data Management Skillbuilding Hub

Best Practice: Create, manage, and document your data storage system

BEST PRACTICE

Best Practices by Data Life Cycle




Create, manage, and document your data storage system

Data Life Cycle stage(s): Plan

Data files should be managed to avoid disorder. To facilitate access to files, all storage devices, locations and access accounts should be documented and accessible to team members. Use appropriate tools, such as version control tools, to keep track of the history of the data files. This will help with maintaining files in different locations, such as at multiple off-site backup locations or servers.

Data sets that result in many files structured in a file directory can be difficult to decipher. Organize files logically to represent the structure of the research/data. Include human readable “readme” files at critical levels of the directory tree. A “readme” file might include such things as explanations of naming conventions and how the structure of the directory relates to the structure of the data.

Description Rationale

Keeping a managed file storage system will help prevent inconsistencies, e.g., duplicated, lost, or misplaced files.

Examples

A time series of image files from several remote cameras might be organized so that images from each camera are in different folders. These are, in turn, collected in a folder named “images”. Each folder would be named with the identifier for the camera. The file names for images might reflect the time the image was taken.

A “readme” file would document the structure of this system, and document the name scheme to facilitate future curation and automated gathering of metadata.

Tags

 
 
 
 

Cite this best practice:

Robert Olendorf, Robert Stevenson, DataONE  (May 11, 2011) "Best Practice: Create, manage, and document your data storage system". Accessed through the Data Management Skillbuilding Hub at https://dataoneorg.github.io/Education/bestpractices/create-manage-and on Aug 22, 2019


Home

Hosted by DataONE

In collaboration with the community, DataONE has developed high quality resources for helping educators and librarians with training in data management, including teaching materials, webinars and a database of best-practices to improve methods for data sharing and management.

Question If you have a question or concern, please open an Issue in this repository on GitHub.