...
The original FEWS import from WIWB data is done through a direct soap connection. Because this method is not intended to be used for bulky data downloads, another second so-called asynchronous download option has been developed. This async option is to be used for non-operational and larger datasets. The normal importType is WIWB, while the asynchronous option is WIWB_async. Note that the async option is only used in case of gridded data. Scalar data is always imported through the normanormal, direct way.
The WIWB_async import type works as follows:
...
- Idle : this means it is on the waiting list for being processed, process will always be 0%
- Processing: the download is being prepared this involves getting the data, processing the data and writing to netcdf files and zipping them. This will always have a progress larger than 0% and lower than 100%.
- Error: something went wrong on the API server side, currently . Currently there will be no explanation provided on why it went wrong, therefor FEWS has now ability to provide an explanation for it since it is out of FEWS control.
- Finished: The download is ready.
- When the download is ready a separate request is done for this specific download. First the whole zip is downloaded, then unzipped in a temporary directory and finally those files are imported one by one.
| Info | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
The amount of data in one WIWB request (download option) also has its maximum. This maximum data request is limited to the currently installed WIWB-server (internal) memory capacity, which is 12 GB. This means that an individual data request can not exceed this amountsize. This maximum of 12 GB is valid per data request, since all data requests are processed sequentially. If the data request is too large, the server runs out of memory and this data requests fails. Delft-FEWS reports that an error has occured at the WIWB-server side. To download the requested data, the user needs to split up the data request. A common approach is would be to reduce the number of timesteps. Below, 3 examples of benchmark data requests are given. These examples stay within the current WIWB-server limits and will be successful. Please use these as inspiration for your own data requests.
|
...