Summary
Importing data in
...
DELFT-FEWS
...
is
...
different
...
from
...
running
...
a
...
wizard
...
in
...
e.g.
...
a
...
much
...
used
...
office
...
software
...
package.
...
To
...
be
...
able
...
to
...
import
...
data
...
into
...
the
...
system
...
the
...
following
...
requirements
...
should
...
be
...
met:
...
- The
...
- file-type
...
- you
...
- want
...
- to
...
- import
...
- should
...
- be
...
- readable
...
- by
...
...
...
...
...
...
...
- Delft-Fews
- You must configure an instance of this import module
- You must add the moduleInstance to an workflow (usually tne ImportExternal Workflow)
- In case the locationId's and parameterId's in your Delft-Fews configuration are not identical to those in the files you need to import a IdMap file should be set up and referred to in the import module
- You will probably need to adjust one or more locationsets, the filters.xml file and the DisplayGroups.xml file if you need to display or process the newly imported data types.
Info Please note that Delft-Fews DELETES all files after importing them. As such, you should ALWAYS copy files from another directory to the location from which Delft-Fews imports files. If you do not do this you might loose you files.
The XML configuration files shown in this example are also attached to this page:
| Attachments | ||||
|---|---|---|---|---|
|
Setting up the import module
Steps:
- make the Configuration file for the import module and store in the ModuleConfigFiles directory. In this example it should be named Import 1.00 default.xml
- register this new module in the ModuleInstanceDescriptors 1.00 default.xml file in the RegionConfigFiles directory
- Add the new module at a workflow in order to run it.
Step 1: Configure the import module (create the Import 1.00 default.xml configuration File)
The first step is to configure an instance of the import module for the type of data you need to import. In this example we will import data in PI XML format for 4 locations. The file we want to import is shown below:
| No Format |
|---|
|Which file types can Delft-Fews Import?] # You must configure an instance of this import module # You must add the moduleInstance to an workflow (usually tne ImportExternal Workflow) # In case the locationId's and parameterId's in your Delft-Fews configuration are not identical to those in the files you need to import a IdMap file should be set up and referred to in the import module # You will probably need to adjust one or more locationsets, the filters.xml file and the DisplayGroups.xml file if you need to display or process the newly imported data types. {info} Please note that Delft-Fews _DELETES_ all files after importing them. As such, you should _ALWAYS_ copy files from another directory to the location from which Delft-Fews imports files. If you do not do this you might loose you files. {info} {iframe}Some content{iframe} The XML configuration files shown in this example are also attached to this page: {attachments:patterns=.*xml|old=true} h1. Setting up the import module Steps: # make the Configuration file for the import module and store in the ModuleConfigFiles directory. In this example it should be named *Import 1.00 default.xml* # register this new module in the *ModuleInstanceDescriptors 1.00 default.xml* file in the RegionConfigFiles directory # Add the new module at a workflow in order to run it. h2. Step 1: Configure the import module (create the Import 1.00 default.xml configuration File) The first step is to configure an instance of the import module for the type of data you need to import. In this example we will import data in [PI|PI - Delft-Fews Published Interface Format] XML format for 4 locations. The file we want to import is shown below: {noformat} <?xml version="1.0" encoding="UTF-8"?> <TimeSeries xmlns="http://www.wldelft.nl/fews/PI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews/PI http://fews.wldelft.nl/schemas/version1.0/pi-schemas/pi_timeseries.xsd" version="1.2"> <series> <header> <type>instantaneous</type> <locationId>locA</locationId> <parameterId>WaterLevel</parameterId> <timeStep unit="second" multiplier="3600"/> <startDate date="2006-08-23" time="17:00:00"/> <endDate date="2006-08-24" time="00:00:00"/> <missVal>-8888.0</missVal> <longName>locA</longName> <units>m</units> </header> <event date="2006-08-23" time="17:00:00" value="8.66"/> <event date="2006-08-23" time="18:00:00" value="9.66"/> <event date="2006-08-23" time="19:00:00" value="8.66"/> <event date="2006-08-23" time="20:00:00" value="-8888.0"/> <event date="2006-08-23" time="21:00:00" value="3"/> <event date="2006-08-23" time="22:00:00" value="3"/> <event date="2006-08-23" time="23:00:00" value="3"/> <event date="2006-08-24" time="00:00:00" value="-8888.0"/> </series> <series> <header> <type>instantaneous</type> <locationId>locB</locationId> <parameterId>WaterLevel</parameterId> <timeStep unit="second" multiplier="3600"/> <startDate date="2006-08-23" time="13:00:00"/> <endDate date="2006-08-24" time="00:00:00"/> <missVal>-999.0</missVal> <longName>locB</longName> <units>m</units> </header> <event date="2006-08-23" time="13:00:00" value="0.63"/> <event date="2006-08-23" time="14:00:00" value="0.61"/> <event date="2006-08-23" time="15:00:00" value="0.59"/> <event date="2006-08-23" time="16:00:00" value="0.59"/> <event date="2006-08-23" time="17:00:00" value="0.59"/> <event date="2006-08-23" time="18:00:00" value="0.59"/> <event date="2006-08-23" time="19:00:00" value="0.60"/> <event date="2006-08-23" time="20:00:00" value="0.57"/> <event date="2006-08-23" time="21:00:00" value="0.59"/> <event date="2006-08-23" time="22:00:00" value="0.59"/> <event date="2006-08-23" time="23:00:00" value="-999.0"/> <event date="2006-08-24" time="00:00:00" value="-999.0"/> </series> <series> <header> <type>instantaneous</type> <locationId>LocC</locationId> <parameterId>WaterLevel</parameterId> <timeStep unit="second" multiplier="3600"/> <startDate date="2006-08-23" time="13:00:00"/> <endDate date="2006-08-24" time="00:00:00"/> <missVal>-999.0</missVal> <longName>LocC</longName> <units>m</units> </header> <event date="2006-08-23" time="13:00:00" value="-1.42"/> <event date="2006-08-23" time="14:00:00" value="-1.42"/> <event date="2006-08-23" time="15:00:00" value="-1.44"/> <event date="2006-08-23" time="16:00:00" value="-1.46"/> <event date="2006-08-23" time="17:00:00" value="-1.46"/> <event date="2006-08-23" time="18:00:00" value="-1.46"/> <event date="2006-08-23" time="19:00:00" value="-1.48"/> <event date="2006-08-23" time="20:00:00" value="-1.48"/> <event date="2006-08-23" time="21:00:00" value="-1.48"/> <event date="2006-08-23" time="22:00:00" value="-1.49"/> <event date="2006-08-23" time="23:00:00" value="-999.0"/> <event date="2006-08-24" time="00:00:00" value="-999.0"/> </series> <series> <header> <type>instantaneous</type> <locationId>LocD</locationId> <parameterId>WaterLevel</parameterId> <timeStep unit="second" multiplier="3600"/> <startDate date="2006-08-23" time="13:00:00"/> <endDate date="2006-08-24" time="00:00:00"/> <missVal>-999.0</missVal> <longName>LocD</longName> <units>m</units> </header> <event date="2006-08-23" time="13:00:00" value="-1.42"/> <event date="2006-08-23" time="14:00:00" value="-1.42"/> <event date="2006-08-23" time="15:00:00" value="-1.44"/> <event date="2006-08-23" time="16:00:00" value="-1.46"/> <event date="2006-08-23" time="17:00:00" value="-1.46"/> <event date="2006-08-23" time="18:00:00" value="-1.46"/> <event date="2006-08-23" time="19:00:00" value="-1.48"/> <event date="2006-08-23" time="20:00:00" value="-1.48"/> <event date="2006-08-23" time="21:00:00" value="-1.48"/> <event date="2006-08-23" time="22:00:00" value="-1.49"/> <event date="2006-08-23" time="23:00:00" value="-999.0"/> <event date="2006-08-24" time="00:00:00" value="-999.0"/> </series> </TimeSeries> {noformat} |
When
...
configuring
...
the
...
import
...
module
...
we
...
will
...
need
...
to
...
extract
...
the
...
following
...
information
...
from
...
the
...
file:
...
- the
...
- locationId's
...
- in
...
- the
...
- XML
...
- file
...
- the
...
- parameterID's
...
- in
...
- the
...
- XML
...
- file
...
- the
...
- timestep
...
- in
...
- the
...
- XML
...
- file
...
This
...
information
...
is
...
needed
...
to
...
configure
...
the
...
timeseriesSet
...
in
...
the
...
import
...
module.
...
A
...
quick
...
look
...
at
...
the
...
file
...
shows
...
that
...
there
...
are
...
four
...
locations:
...
LocA,
...
LocB,
...
LocC
...
and
...
LocD.
...
Only
...
one
...
parameter
...
is
...
present
...
(WaterLevel)
...
and
...
the
...
data
...
comes
...
in
...
hourly
...
timesteps.
...
Another
...
type
...
of
...
information
...
that
...
we
...
will
...
need
...
is
...
the
...
location
...
at
...
which
...
Delft-Fews
...
can
...
find
...
to
...
files
...
to
...
be
...
imported.
...
For
...
this
...
example
...
we
...
assume
...
the
...
files
...
will
...
be
...
located
...
in
...
c:/Import/PI
...
.
...
With
...
this
...
we
...
can
...
make
...
the
...
file
...
below:
| Info |
|---|
| }
Please see the Delft-Fews configuration guide for detailed information on how to configure the Import modules. In addition, the .xsd schema file will provide documentation. |
| No Format |
|---|
< {info} {noformat} <?xml version="1.0" encoding="UTF-8"?> <timeSeriesImportRun xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/timeSeriesImportRun.xsd"> <import> <general> <importType>PI</importType> <!-- Indicated we will read PI Files --> <folder>c:/Import/PI</folder> <!-- read the files from here --> <!-- Files WILL BE DELETED after import --> <failedFolder>c:/Backup/PI</failedFolder> <!-- If specified the imported files will be copied to this location --> <idMapId>IdImport</idMapId> <!-- The IdMap for this import --> <importTimeZone> <timeZoneOffset>+00:00</timeZoneOffset> </importTimeZone> </general> <timeSeriesSet> <moduleInstanceId>Import</moduleInstanceId> <!-- The name of this moduleinstance --> <valueType>scalar</valueType> <parameterId>H.obs</parameterId> <!-- The parameter as it will be stored in Delft-Fews, different from the parameter in the XML. Therefore, IdMapping has to be set up--> <locationSetId>LevelGauges</locationSetId> <timeSeriesType>external historical</timeSeriesType> <timeStep unit="hour" multiplier="1"/> <readWriteMode>add originals</readWriteMode> <synchLevel>1</synchLevel> </timeSeriesSet> </import> </timeSeriesImportRun> {noformat} |
In
...
the
...
general
...
section
...
of
...
the
...
configuration
...
the
...
location
...
at
...
which
...
the
...
file(s)
...
are
...
to
...
be
...
found
...
is
...
give
...
in
...
addition
...
to
...
an
...
optional
...
backup
...
location.
...
Imported
...
data
...
will
...
be
...
copied
...
to
...
the
...
backup
...
location
...
if
...
it
...
is
...
defined
...
and
...
accessible.
...
All
...
imported
...
files
...
in
...
the
...
folder
...
location
...
will
...
be
...
deleted.
...
The
...
contents
...
of
...
the
...
idMapId
...
element
...
refers
...
to
...
the
...
IdMapping
...
table
...
that
...
converts
...
the
...
external
...
locations/parameters
...
to
...
those
...
used
...
in
...
your
...
Delft-Fews
...
application.
...
If
...
the
...
mapping
...
is
...
one-to-one
...
the
...
idmap
...
is
...
not
...
needed.
...
The
...
timeSeriesSet
...
element
...
defines
...
how
...
the
...
data
...
will
...
be
...
stored
...
in
...
delft-Fews.
...
If
...
you
...
later
...
want
...
to
...
view,
...
use
...
or
...
export
...
the
...
data
...
you
...
will
...
need
...
the
...
same
...
timeseriesset
...
to
...
retrieve
...
or
...
view
...
the
...
data.
...
In
...
the
...
timeseriesset
...
we
...
have
...
defined
...
the
...
following:
- moduleInstanceId – Name of this configuration instance, reflects the XML file name
- valueType – scalar, this is scalar data
- parameterId – H.obs – This is the parameterId in Delft-Fews. The parameter Id in the XML file is WaterLevel. The Id mapping (shown below) will map Waterlevel to H.obs
- locationSetId – In order to simplify configuration we have defined a list of locations (a locatioset) in the file LocationSets.xml (in the RegionConfigFiles directory). This set holds the following four locations: one, two, three, fours. As these are differenmt from the locations in the XML file we are importing the id mapping will need to map them.
- timeStep – the timestep defined here should map the timestep in the XML files to be imported.
Step 2: Register the new module – Add to the ModuleInstanceDescriptors 1.00 default.xml file
All moduleInstances must be registered in Delft-Fews. This is done by adding en entry in the moduleInstancseDescriptors file in the RegionConfigDiles section of the configuration. To register this module we will need to a the following to the moduleInstancedescriptors:
| Info |
|---|
In this example only one moduleInstance is present. Usually you will need to ad your module to an existing list. |
| No Format |
|---|
- moduleInstanceId -- Name of this configuration instance, reflects the XML file name - valueType -- scalar, this is scalar data - parameterId -- H.obs -- This is the parameterId in Delft-Fews. The parameter Id in the XML file is WaterLevel. The Id mapping (shown below) will map Waterlevel to H.obs - locationSetId -- In order to simplify configuration we have defined a list of locations (a locatioset) in the file LocationSets.xml (in the RegionConfigFiles directory). This set holds the following four locations: one, two, three, fours. As these are differenmt from the locations in the XML file we are importing the id mapping will need to map them. - timeStep -- the timestep defined here should map the timestep in the XML files to be imported. h2. Step 2: Register the new module -- Add to the ModuleInstanceDescriptors 1.00 default.xml file _All_ moduleInstances must be registered in Delft-Fews. This is done by adding en entry in the moduleInstancseDescriptors file in the RegionConfigDiles section of the configuration. To register this module we will need to a the following to the moduleInstancedescriptors: {info} In this example only one modulinstance is present. Usually you will need to ad your module to an existing list. {info} {noformat} <?xml version="1.0" encoding="UTF-8"?> <moduleInstanceDescriptors xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/moduleInstanceDescriptors.xsd" version="1.0"> <!--Import Module Instances--> <moduleInstanceDescriptor id="Import"> <description>Import Our new data</description> <moduleId>TimeSeriesImportRun</moduleId> </moduleInstanceDescriptor> </moduleInstanceDescriptors> {noformat} h2. Step 3: Add the module to a workflow (ImportExternal |
Step 3: Add the module to a workflow (ImportExternal 1.00D
...
efault.xml)
...
In
...
order
...
to
...
use
...
the
...
module
...
(to
...
be
...
able
...
to
...
run
...
it
...
from
...
the
...
manual
...
forecast
...
dialog)
...
you
...
will
...
need
...
to
...
list
...
the
...
module
...
as
...
an
...
activity
...
in
...
a
...
workflow.
...
Assuming
...
this
...
workflow
...
is
...
already
...
registered
...
(see
...
the
...
Delft-Fews
...
configuration
...
manual
...
on
...
how
...
to
...
register
...
workflow)
...
the
...
only
...
thing
...
to
...
do
...
is
...
to
...
add
...
in
...
to
...
the
...
workflow.
...
In
...
the
...
following
...
example
...
the
...
new
...
module
...
is
...
the
...
only
...
activity
...
in
...
the
...
workflow:
| Code Block |
|---|
} <?xml version="1.0" encoding="UTF-8"?> <workflow xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/workflow.xsd" version="1.1"> <activity> <runIndependent>true</runIndependent> <moduleInstanceId>Import</moduleInstanceId> </activity> </workflow> {code} h1. Set up Id Mapping h2. Step 4: Make the idmap file h2. Step 5: register the idmap file h1. Setting up a (temporary) display to view the imported data h1. Test the import h1. Add to the appropiate locationset h1. Add to the appropriate |