Since scripting becomes increasingly important in our projects, the rules of software development also start to apply to us. This has the downside that it requires extra effort from us, scientists, to learn new tools, but the upside is that there already is a wealth of properly tested tools and documentation available from the software development world. Having reproducible code is one of these important things, for which version control systems have been developed. It is very useful to use these systems in our projects, since they provide the following advantages:
- You keep track of the history of a project. In this way you keep a journal of decisions made in a project.
- If you mess up something, you can always revert to a previous state.
- It allows for collaborative development, where individuals can create their own branch to safely work on new developments and later merge their changes.
SVN and Git are two common version control systems, both used within Deltares. These VCSs differ in implementation: SVN is a centralized version control system, whereas Git is a distributed version control system. This means that the project history in SVN only exists on a central server, whereas each Git user has a clone of the project history on their machine. Git has multiple advantages over SVN:
- Mistakes introduced in the version control are not immediately sent to the server, so are less likely to affect others
- Larger userbase, so more documentation and more auxiliary applications available. The editor VSCode, for example, has direct Git integration.
- Smoother development between multiple organizations, as no central server is required at one organization. Repositories can be easily shared via platforms such as GitHub or Gitlab.
However, Git by itself has one major downside: It is not suited for large and/or binary files. Furthermore, platforms GitHub and Gitlab usually only allow storing of files up to 1 GB. To solve this, DVC (Data Version Control) was developed, which is built on top of Git and uses similar commands. This program allows storing and distributing data in the cloud (e.g., Google Drive, Google Cloud, Amazon S3 and Microsoft Azure) and at network drives (e.g., our N: - or P: -drive).
A thorough tutorial of how to use Git and DVC can be found here: https://deltares.github.io/iMOD-Documentation/practical_git_dvc.html
