Compact index and cache files.
Because time series are combined in one record FEWS uses a custom index to know which records are involved when reading time series. Expired records are automatically removed from the database but not from the index files that reside on forecasting shells. To do this the compact cache files activity should be scheduled on every forecasting shell daily. On operator clients and stand-alone systems this happens automatically. When the obsolete ModuleDescriptors.xml still exist the module “nl.wldelft.fews.system.plugin.rollingbarrel.RollingBarrelModule” is redirected to the compact cache file activity. It is not running the rolling barrel any more. This is done by the MC or every time the forecasting shell starts when a local data store is still used instead of DDA
Compact_FSSCache.xml in WorkflowFiles
<?xml version="1.0" encoding="UTF-8"?> <workflow version="1.1" xmlns="http://www.wldelft.nl/fews" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.wldelft.nl/fews http://fews.wldelft.nl/schemas/version1.0/workflow.xsd"> <activity> <predefinedActivity>compact cache files</predefinedActivity> </activity> </workflow>
WorkflowDescriptors.xml in RegionConfigFiles
<workflowDescriptor id="Compact_FSSCache" name="Compact FSS Cache Files" forecast="false" visible="true"> <runExpiryTime unit="week" multiplier="52"/> </workflowDescriptor>
In the workflow mapping, the workflow should be set to; staggered-all. This will make sure the workflow is run on all FSS's, but not at the same time. Staggering time should be minimal 15 minutes.
Example of a log message when the compact cache is finished
20-02-2016 04:53:46 INFO - TimeSeriesIndex.CompactFinished: 6 seconds, removed rows 142844/403158, removed time series 923/50598, removed groups 39168/100775, removed ids 8/13565, removed qualifier sets 0/0, removed id sequences 27256/78750, released memory 10 MB/45 MB
Every unique combination of time series that shares one record is called a group. Every unique sorted combination of ids for parameters, locations and qualifiers that exist in any group is called an id sequence. A unique sorted combination of qualifiers attached to one or more time series is called a qualifier set. Multiple qualifiers can be attached to a single time series.
Running the compact cache files regularly not only reduces disk space but also make the indexes more memory efficient and faster.
When an external state zip directory is configured in the clientConfig.xml this module is also deleting state zip files that no longer exist in the database.
