Establishing a well-organized project structure is vital for ensuring reproducibility in your numerical modeling projects. A clear and consistent structure not only improves readability but also simplifies navigation, facilitates collaboration, and enhances the overall reproducibility of your work. In this chapter, we will delve into the following aspects of project structure:
- Consistent folder structure
- Readable scripts
- Documentation
- File formats and data organization
Standardized folder structures with Cookiecutter
To make data easily findable in a project directory, it is useful to have a consistent directory structure within the organization with required and optional directories. Furthermore, certain files are required in all projects, e.g., a README, and a LICENSE.
Using the same project template in an organization has the following advantages:
- No time wasted thinking about file organization
- Consistency between work of colleagues and projects
- Scripts more easily transferrable between projects
- Less time wasted searching for files
Cookiecutter is such a project template tool, which helps kickstarting a project with one command. A Deltares template is readily available for use by running the following command:
cookiecutter gl:deltares/imod/cookiecutter-reproducible-project
This will create the following folder and file structure:
├── AUTHORS.md
├── LICENSE
├── README.md
├── bin <- Your compiled model code can be stored here (not tracked by git)
├── config <- Configuration files, e.g., for doxygen or for your model if needed
├── data
│ ├── 1-external <- Data external to the project.
│ ├── 2-interim <- Intermediate data that has been altered.
│ ├── 3-input <- The processed data sets, ready for modeling.
│ ├── 4-output <- Data dump from the model.
│ └── 5-visualization <- Post-processed data, ready for visualisation.
├── docs <- Documentation, e.g., doxygen or scientific papers (not tracked by git)
├── notebooks <- Jupyter notebooks
├── reports <- For a manuscript source, e.g., LaTeX, Markdown, etc., or any project reports
│ └── figures <- Figures for the manuscript or reports
└── src <- Source code for this project
├── 0-setup <- Install necessary software, dependencies, pull other git projects, etc.
├── 1-prepare <- Scripts and programs to process data, from 1-external to 2-interim.
├── 2-build <- Scripts to create model specific inputm from 2-interim to 3-input.
├── 3-model <- Scripts to run model and convert or compress model results, from 3-input to 4-output.
├── 4-analyze <- Scripts to post-process model results, from 4-output to 5-visualization.
└── 5-visualize <- Scripts for visualisation of your results, from 5-visualization to ./report/figures.
This will generate a folder and file structure with predefined directories, such as bin, config, data, docs, notebooks, reports, and src. Each directory serves a specific purpose in organizing different types of project data and code, making it easier to navigate and locate specific resources.
Readable scripts
In addition to having a well-organized project structure, ensuring that your scripts are readable is crucial for reproducibility. Consider the following practices for enhancing script readability:
- Follow a consistent coding style to improve code comprehension.
- Utilize script templates that outline essential sections and provide a standardized format.
- Incorporate comments throughout the code to explain its purpose and functionality.
- Write self-explanatory variable and function names to enhance code understanding.
Remember, readability should take precedence over ease of writing when it comes to scripting or programming.
Documentation and README Files
Documentation plays a vital role in reproducible projects as it provides crucial information about the project's purpose, usage, and dependencies. Including comprehensive documentation makes it easier for others to understand and reproduce your work. A README file serves as the entry point to your project and should contain essential details such as:
- Project overview: Briefly describe the purpose and goals of the project.
- Installation instructions: Provide step-by-step instructions on how to set up the project environment and any necessary dependencies.
- Usage examples: Include examples that demonstrate how to use the project's scripts or modules.
- Contact information: Provide contact details for questions or collaboration opportunities.
