Original question

As this is my first time submitting to the forum, please let me explain my needs of Delft-FEWS. I'm new to FEWS (v2019.02) to which I'm hoping to adopt as an interface for generating numerical model forcing inputs. I am NOT using FEWS for forecasting.I've been working on a set of workflows, to (i) import (20 years worth) hourly data (from 40 stations) --> (ii) aggregate the data to 6-hourly time steps --> (iii) interpolate to a set (~2800) of 10km² subwatersheds  --> (iv) export to NetCDF.I have been successful with the first 2 workflows, and have successfully tested (iii) for 10 days of data.  Subsequent to the successful test of (iii), I've gone ahead to interpolate the remaining 20 years.  Once complete, I see it reported in the Database Lister, can view the data in the Data Display, etc. From my perspective the data are in the database.
The problem I'm having is that upon shutting down FEWS and restarting, I am left with the original 10 days of data I used for testing. No sign remains of the rest of the 20 years.  So far, (that I can recall) I've looked into:

 - the status of the rolling barrel
 - extended my global expiry time to 10,000 days
 - experimented with readWriteMode to both "read only" and "add originals"
 - increased ram allocation
 - run the processed using the Manual Forcast display chunk by chunk with debug output
 - ensured workflow autoApprove="true"
 - banged my head on the table, etc.
 
Any help would be much appreciated. Thanks

Solution

When you mention that data from a run is only present in the localDataStore until a restart, I am reminded of the Delft-FEWS feature where you can run workflows as an "IFD run" (local) vs. "Server run" server side. When you are running a stand alone (SA) Delft-FEWS (you do not specify this), this seems not relevant, as there is no "server side", but the concept still is in place. This means that also in a SA setup, if you run a workflow as an IFD, or local run, it will not remain in the database after a restart. If you run it as a "server run" it will remain in the database.

Note that the information below plays a role when activating a workflow from the Topology (or "Interactive Forecast Display"). 
Below information is copied from the Topology section of the Delft-FEWS wiki:

IFD run

By default all the workflows started from a leaf node are started in IFD mode. The results of an IFD-run are only temporary available in FEWS (until the next restart) and are only available and visible at the operator client which started the run.IFD runs are typically used to evaluate the effect of modifiers on the results of a forecast. 

Server run

A workflow which is started from a group node will by default be started as a server run. A server run will run at a FSS when it is started from an operator client. When it is started from a stand alone it will run locally but the results of the run will be available after a restart of the system.

Now, critically, it is possible for workflows defined in the Topology to actually specify if you want a "local run" or a "server run", by setting the localrun element in Tpology.xml. What I would suggest is setting up a Topology.xml and use a <localRun> element set to false:

<localRun>false</localRun>


It is possible to change these default settings by using the element localRun. In the example below the runs from leaf nodes A (localRun=false) and B (defaults to false) will run in server mode and the runs started from the group node C will run in IFD mode (localRun=true).

<?xml version="1.0" encoding="UTF-8"?>
<topology xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/topology.xsd">
    <nodes id="C" name="C">
        <workflowId>workflowC</workflowId>
        <localRun>true</localRun>
        <node id="A" name="A">
            <workflowId>workflowA</workflowId>
            <localRun>false</localRun>
        </node>
        <node id="B" name="B"/>
    </nodes>
</topology>


  • No labels

Disclaimer