Some points to pay attention during RTC-Tools intergration into FEWS.
Forecast date
For rtc-tools, the start date is the earliest timestamp present in a timeseries. This is possibly historical data. forecast date is the point from which the optimization/simulation starts. In most cases the start data and the forecast date overlap. However, we have to leave the flexibility for the existence of historical information. (Say, for e.g., that the decision that you take now depends on what happened in the last 4 hours). In FEWS the users have a different concept for forecast date. In order to be able to trigger historical runs, without confusing the user the following be done is that in the python script of the model, the forecast date is overwritten such that is becomes the start date (therefore, bypassing the forecast date information from FEWS). That can be done as following:
def pre(self):
super().pre()
self.io.reference_datetime = self.timeseries_import.start_datetime
...
Pickle
RTC-Tools is creating a pickle at the first run to store the data about the model. (You can find it in the model folder). During the following runs the pickle is first compared to the model, and if the model has not changed the pickle is used. It makes the runs faster. However, if the pickle is created from another machine, it is not compatible with the current version. For this reason it is better not to zip it in the model folder. In order to keep it it can be created an exception when purging.
Enesmbles
RTC-Tools does not run if the ensemble id is in the timeseries_import.xml file. Therefore "<ensembleMemberFormat>hide</ensembleMemberFormat>" should be used in the general adapter.
Simulation mode side notes
- Simulation mode checks at every step if there is a timeserie for a specific variable. If there is a valid value, it assignes that value. This occurs even for timeseries that we would like to modify during the simulation internally, e.g. Q turbine. Therefore if you want to set a value for a variable in RTC-Tools, make sure that the corresponding timeserie at that timestep is empty (NaN) or does not exist.
- RTC-Tools simulation sets values for timeseries always to the following time step.
- There is no built-in check for NaNs! Therefore if there is a NaN value in the input that is needed by the model, the model will go through without saying anything (and uses the value from the previous timestep). This is often unexpected behaviour. If you want to make sure that your model fails in case of absence of necessary data, you have to make this explicit in the Python script.
- Simulation model is by definition not sending error message even if it fails. Therefore it is possible that a workflow is executed without errors, but the result is an empy timeseries. These issues should be caught by the modeller and add some meaningful error message in the Python code of the RTC-Tools simulation model.
