| Children Display |
|---|
Catalogue read-only mode
| Warning |
|---|
If the catalogue is in read-only mode then the catalogue can no longer be updated. Elastic will switch to read-only mode when the available disk-space is less then 5%. Note that elastic will stay in read-only mode even when you make more disk space available. |
If you run the harvester when the catalogue is in read-only mode then you will see the following errors in the log-files of the archive.
[0]: index [metadatafiles], type [search], id [ovl0nH8BF4WRu26gnjcI], message [ElasticsearchException[Elasticsearch exception [type=cluster_block_exception, reason=blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]]]
[1]: index [simulatedmetadata_seamless], type [seamless_integration_search], id [o_l0nH8BF4WRu26gnjcI], message [ElasticsearchException[Elasticsearch exception [type=cluster_block_exception, reason=blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]]]
The latest version of the archive admin console indicates if your catalogue is in read-only mode.
Control when read-only mode is activated
Elastic can be configured when to to read-only mode. This can be done by manually configuring the thresholds in the elasticsearch.yml. This file can be found in the config folder of elastic.
| Code Block | ||
|---|---|---|
| ||
cluster.routing.allocation.disk.watermark.low: "20gb"
cluster.routing.allocation.disk.watermark.high: "15gb"
cluster.routing.allocation.disk.watermark.flood_stage: "10gb" |
NB. cluster.routing.allocation.disk.watermark.low must be higher than cluster.routing.allocation.disk.watermark.high and cluster.routing.allocation.disk.watermark.high should be higher than cluster.routing.allocation.disk.watermark.flood_stage.
The config above will ensure that elastic only switches to read-only mode when there is less then 10gb available. To force elastic to leave the read-only mode when there is more disk space available or when the settings in the elasticsearch.yml are adjusted you can execute the following commands at the elastic server.
linux
| Code Block | ||
|---|---|---|
| ||
curl -XPUT -H "Content-Type:application/json" http://localhost:9200/_all/_settings -d '{ "index.blocks.read_only_allow_delete": null }' |
Windows
| Code Block | ||
|---|---|---|
| ||
curl -XPUT -H "Content-Type:application/json" http://localhost:9200/_all/_settings -d "{ \"index.blocks.read_only_allow_delete\": null }" |
Rebuild catalogue from scratch
Another option is to delete the current catalogue and rebuild it from scratch.
- stop the tomcat instance which hosts the archive.
- remove the data folder in the elasticsearch installation directory.
- start the archive again.
- start the harvester from Archive tasks tab to rebuild the catalogue.
Speed up harvester
The harvesting process can be quite time consuming. For simulated data and external forecasts both the available netCDF-files and the available time series are stored in the catalogue. The indexing of the time series of simulated data and external forecasts consumes a lot of time. These indexes are only needed when you want to retrieve simulated data or external forecasts from the archive by using the Delft-FEWS web services. This feature is only used by limited amount of clients. If you do not use this feature then you can disable this indexing by the following configuration option the archiveServerConfig.xml. Below an example.
...
| language | xml |
|---|
...