Scenario editing

Scenario generation

A scenario for the water coach can be created in several ways.
1. Make a copy of the localDataStore directly after an interesting event. If the events spans a longer period of time, in which case relevant data might be removed by the rolling barrel, several copies should be made while the event is still in progress.
2. If the data from FEWS is archived in some way, retrieve the data for an interesting historic event from the archive. This could be done using the archiving functionality from FEWS. To facilitate this, the bulletinBoardPlus has been developped. hiero
3. Recreate the data for an event, by running all relevant workFlows in a stand alone copy of the FEWS system.

Manier realiseren om gemakkelijk op basis van bestaand materiaal een scenario generen. Hiervoor zijn verschillende aanpakken mogelijk, zoals:
1. Methodiek / tool om automatisch of met een kleine inspanning data (uit FEWS) van een opgetreden echt situatie op te slaan om als scenario te gebruiken. Het huidige project voorziet alleen in een methodiek voor het verzamelen van alle benodigde data voor een scenario. Er wordt momenteel niet gewerkt aan een zogenaamde automatische scenario generator.
2. Bewerken van bestaand scenario, bijvoorbeeld het debiet op de Maas met een factor verhogen of verlagen. Deze aanpassingen zouden ook moeten doorwerken in het weerbeeld en meetlocaties. Dit is een complexe taak, hiervoor wordt eerst een aanpak uitgewerkt en voorgelegd. Het huidige project voorziet alleen in een methodiek voor het bewerken van een bestaand scenario op basis van eerdere ervaringen binnen FEWS Rivieren. Aad D. is van mening dat een kloppende meteorologie van ondergeschikt belang is.

Bewerken scenario

The GUI should offer an intuitive method to adjust the data of the timeline, i.e. adapt the scenario:
Increase occurred water levels (measured and/or forecast)
Simulated failure of measurements by deleting certain signals
Manipulate availability of data, to simulate (temporarily) failing data and systems:
Measurements not available for the entire period of interest
Forecasts that come in (too) late
etc
etc
A scenario editor is not part of the current project plan. It is suggested to look into this in a later stage.

In een interne brainstormsessie op 2012-06-01 kwam het idee naar voren om in een scenario ipv de meetwaardes (i.e. de waarheid) de uitkomst van 1 van de ensemble members te gebruiken. KNMI maakt 50 weersverwachtingen elk met een kleine perturbatie. Met elke afzonderlijke weersverwachting kan het waterstandsmodel (en golvenmodel) ook worden doorgerekend. Een tijdserie hieruit ter plaatse van de meetstations kan worden gebruikt als een alternatief meetsignaal. Voordeel hiervan is dat het scenario consistent blijft (i.e. weer en water). Door het meetsignaal zo aan te passen, weet de speler niet wat de "goede" uitkomst is en verklein je de kans dat men alle scenario's "uit het hoofd" kent.

Meerdere scenario's maken obv ensemble's?
>>>>>> Metingen vervangen door 1 van de ensemble runs (betekent dat je ook het water 50 keer moet door rekenen)
Model data die fews toont verandert niet!!

Archiveren events

This functionality is built in gradually into FEWS. Every step will be discussed with the client (or a representative) after implementation. During these discussion, ideas to fine-tune the next implementation step will arise and can still be incorporated.

Making a copy of the relevant data may not under any circumstances disturb the workflow of the operators. Therefore, the actual copying of the dataset will only be performed after the crisis has passed. The relevant data is stored within the database MATROOS and the local datastore of FEWS North Sea. Both MATROOS and the local datastore keep all data for at least 10 days, which is enough history for a storm surge event as is the case for the SVSD. (Please note that this condition needs to be check for each individual FEWS configuration where this method will be applied.)

This will result in a prototype, that keeps growing in functionality. If the remaining budget and time permits it, the final prototype will be tested within the FC2015 Serious Gaming environment.

Make a copy of all data in FEWS
The archive functionality should be configured and implemented in the operational FEWS system. This includes:
Archive of the Forecasts, Thresholds, TimeSeries of selected modules and the FEWS Configuration into Zip files.
Data storage availability
The archive_run will be scheduled on a daily / weekly basis and thus performed automatically. The archive is saved on a pre-defined location. The archive can be stored for a period of for example 1 year (via a rolling barrel method).
Note: It is not correct to use the local data store to make a copy of the data in FEWS, since the content of the local data store is not complete, but only a snap-shot of the current state of FEWS for a fixed period of time (e.g. the last 10 days). The content changes every second via the rolling barrel method. Therefore, this can not be used to recreate an event.
Retrieve copy based on time markers
Within the period that the archive is saved, the data for a specific time window can be retrieved from the archive based on time markers (start and stop time) and imported to a local data store. For this, the archive server will be used. The following functionality is required (partially to be developed):
Define time window for which to restore data
Import data (i.e. Zip files) into a local data store
Filter log file from Bulletin Board on event-messages to generate the time line (see below)
Furthermore, you want to log the occurrence of interesting events during the scenario, i.e. the historical time line. This time line can be entered to by hand or automatically, see below.
Retrieve copy based on time markers for selected parameters
It is too complex to archive only selected parameters. Also, it is not necessary from a performance point of view. The size of the local data store does not effect the performance of Fews, as long as it is configured correctly (e.g. use of the amalgamme module). Therefore, we will not look into this.
2a: Manual generation of time markers via "bulletin board

Create an event-message using Bulletin Board to mark
Beginning and end of the scenario
Interesting events
The messages created by all users are stored in the same overall log file.
The template of the event should be the following (requires discipline of the user)
yyyy-mm-dd hh:mm <description of event>
These even-messages are stored amongst other messages and have to be filtered out afterwards, based on
Events entered by certain users
Events starting with the date and time tag as described above.
2b: Automated generation of time markers via thresholds

Create an event message into the Bulletin Board to mark
Beginning (and end) of the scenario
Interesting events, ie.
Approve forecast
Occurrence of predefined (water) levels
Which data is analyzed by the operator
The template of the event should be similar to the manual messages.
The messages are filtered out similar to the manual messages.
Please note that in the operational system there will be multiple users, simultaneously logged on to the FEWS North Sea system. The implementation for the above should be able to deal with this in a logical way.

FEWS-5806 Archiving and extended filtering of log messages

Archiving log messages

Create ArchiveRun module with export task LogEntriesArchive and include it in an archiving workflow. Log messages are archived by (periodically) running this archiving workflow .
Use constraints to select log messages that should be archived. A log message will be archived when its event code is configured as logEventContstraint option and/or the log message text contains the specified messageText configured as logEventConstraint option.

Configuration example:

<exportArchiveRun>
<archivePeriod start="-2" end="0" unit="day"/>
<archiveType>LogEntriesArchive</archiveType>

/exportArchiveRun>

In this example the following messages will be archived:

all manual messages,
all Validation.HardLimit messages
Validation.SoftLimit messages only for Lobith
Workflow.ActivityStarted/Finished messages only for workflow Rijntakken_Forecast

Retrieving archived log messages from archive

Log messages can be retrieved using ArchivedDialog. The user selects and downloads the log messages archive in the same way as for example thresholds.

1. ArchiveDialog configuration
Configure in this dialog an archive type LogEntriesArchive.
An example:
<archiveTask name="Log Entries" workflowId="Import_Archive_LogEntries" iconName="icons/LookUpTable.gif" queryServiceUrl="$ARCHIVE_SERVER_URL$">

<archiveRun>
<importDirectory>$ARCHIVE_IMPORT_PATH$</importDirectory>
<importArchiveRun>
<archiveType>LogEntriesArchive</archiveType>
</importArchiveRun>
</archiveRun>
</archiveTask>

2. ArchiveRun configuration
Create ArchiveRun module with import task LogEntriesArchive and include this module in the workflow specified in ArchiveDialog.
An example:
<importArchiveRun>
<archiveType>LogEntriesArchive</archiveType>
</importArchiveRun>

Displaying/filtering (archived) log messages in SystemMonitor

By default, all non-manual messages are displayed in LogBrowser tab, and all manual messages are displayed in BulletinBoardPlus.
For Water Coach purposes we need to display both manual and selected non-manual messages in one tab. For this purpose a new tab has been added – BulletinBoardPlus In this new tab we also have some more choices to create manual messages quickly.

To show this tab, configure in SystemMonitorDisplay:

<bulletinBoardPlus>
<tabName>Interesting events</tabName>
</bulletinBoardPlus>

We can also add several optional 'messageTemplates' to create quickly messages with a specific mark. An example:

<bulletinBoardPlus>
<tabName>Interesting events</tabName>
<messageTemplate>Analysed by operator</messageTemplate>
<messageTemplate>Phone line went dead</messageTemplate>
<messageTemplate>Emergency advice requested</messageTemplate>
</bulletinBoardPlus>

Similarly we can add several optional 'users' to quickly create messages from a specific user. An example:

<bulletinBoardPlus>
<tabName>Interesting events</tabName>
<usertemplate>Name 1</usertemplate>
<usertemplate>Name 2</usertemplate>
</bulletinBoardPlus>

Button Add message
Using this button we add a manual message 'plus'.

Manual message 'plus' is actually a standard manual message. The information entered in the additional fields, is stored in the messages text (message body) in LogEntries table field 'logMessage'. Compared to standard manual message , the additional fields are Event Time (default value is Current System Time) and messageTemplate.
The message text (body) entered in the example above is stored as follows:
Simone De Kleermaeker@05-12-2011 13:47:00@Phone line went dead@

If we enter also some additional text, the messages body looks like:
Simone De Kleermaeker@05-12-2011 13:47:00@Phone line went dead@my comment

View period
is applicable to the Event Time, not to the log message creation time. For example, if we choose one month, the log messages with event time inside this period are displayed. Event time of the messages generated automatically is the same as message creation time so column "Event time" shows the same time as column "event creation time"

User
The drop down box 'User' is filled with the current user name, the user names that are found in the table FewsSessions and the userTemplates as defined in the configuration. It is also possible to enter any name in this editable drop down box.
The users, configured with user templates, can be entered while creating manual messages. Configure the user templates if you want to enter usernames that are different from the system user.
<bulletinBoardPlus>
<tabName>Interesting events</tabName>
<messageTemplate>Analysed by operator</messageTemplate>
<messageTemplate>Phone line went dead</messageTemplate>
<messageTemplate>Emergency advice requested</messageTemplate>
<userTemplate>Simone De Kleermaeker</userTemplate>
<userTemplate>gast gebruiker</userTemplate>
</bulletinBoardPlus>
Template
The drop down box 'Template' is filled with templates configured in SystemMonitorDisplay <bulletinBoardPlus> tab. It is also possible to enter any text in this editable drop down box.

Other fields in the tab BulletinBoardPlus
Buttons Refresh, Save and fields log Level, Search in Message, Max number of events have the same meaning and behaviour as in standard BulletinBoard.

Zie ook notulen
Water Coach werksessie 2012-06-01

Aanwezig:
Frank Keppel
Rens van den Bergh
Michelline Hounjet
Simone De Kleermaeker
Martin Verlaan

Scenario bibliotheek

Verzameling opbouwen, waarin alle ooit gemaakte scenario's (= datastores) met per scenario alle bijbehorende scripts worden gearchiveerd. In dit assortiment van scenario's moeten ook scenario's zitten die over de overlappende gebieden van de verschillende FEWS systemen gaan (Kampen, Dordrecht)
Aanvulling:
Tijdens het gesprek van 2012-05-21 is nogmaals bevestigd dat het niet de bedoeling is om binnen dit project extra scenario's te maken. Scenario's worden gemaakt op het moment dat ze nodig zijn en verzameld op een centrale plek, i.e. de scenario bibliotheek. Aad Dollee acht het wenselijk om op termijn 10 of meer scenario's per groep (LCW, LCO, SVSD, HWG, WDIJ) ter beschikking te hebben. De applicatiebeheerder (Peter Weiland) zal dit moeten inrichten.
Planning (product 2): Afgerond. Het begin van de scenario bibliotheek is opgeleverd met de scenario's zoals gebruikt bij de WMCN oefening. De inrichting hiervan bij RWS ligt in handen van Peter Weiland. De al bestaande handleiding van de Water Coach (wiki) kan hierbij ter ondersteuning dienen.

Ter ondersteuning kan de Waterdienst/DID het mogelijk maken om in de operationele omgeving naast het scenario ook zaken ten behoeve van het script vast te leggen. Denk bijvoorbeeld aan:

emails archiveren
telefoon gesprekken opnemen en achteraf analyseren
Deltares neemt hiervoor geen actie, maar kan hierbij indien nodig ondersteunen. Wat wel al is uitgevoerd is de uitbreiding van de mogelijkheden om handmatig berichten en opmerkingen toe te voegen en te archiveren vanuit de operationele FEWS applicatie. Het hoe en wat staat gedocumenteerd in de FEWS documentatie (wiki). Om deze functionaliteit te kunnen gebruiken moet de config worden aangepast (kleine inspanning) en moet aan de kant van RWS de opslag van de archief files worden geregeld.

Page tree