Catalogue read-only mode

If the catalogue is in read-only mode then the catalogue can no longer be updated. By default, Elastic will switch to read-only mode when the available disk-space is less then 5%. Note that elastic will stay in read-only mode even when you make more disk space available.

If you run the harvester when the catalogue is in read-only mode then you will see the following errors in the log-files of the archive.

    [0]: index [metadatafiles], type [search], id [ovl0nH8BF4WRu26gnjcI], message [ElasticsearchException[Elasticsearch exception [type=cluster_block_exception, reason=blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]]]
    [1]: index [simulatedmetadata_seamless], type [seamless_integration_search], id [o_l0nH8BF4WRu26gnjcI], message [ElasticsearchException[Elasticsearch exception [type=cluster_block_exception, reason=blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]]]

The latest version of the archive admin console indicates if your catalogue is in read-only mode.

Control when read-only mode is activated

Elastic can be configured when read-only mode is triggered. This can be done by manually configuring the thresholds in the elasticsearch.yml. This file can be found in the config folder of elastic.

cluster.routing.allocation.disk.watermark.low: "20gb"
cluster.routing.allocation.disk.watermark.high: "15gb"
cluster.routing.allocation.disk.watermark.flood_stage: "10gb"

NB. cluster.routing.allocation.disk.watermark.low must be higher than cluster.routing.allocation.disk.watermark.high and cluster.routing.allocation.disk.watermark.high should be higher than cluster.routing.allocation.disk.watermark.flood_stage. 

The config above will ensure that elastic only switches to read-only mode when there is less then 10 GB available.  To force elastic to leave the read-only mode when there is more disk space available or when the settings in the elasticsearch.yml are adjusted you can execute the following commands at the elastic server.

linux

curl -XPUT -H "Content-Type:application/json" http://localhost:9200/_all/_settings -d '{ "index.blocks.read_only_allow_delete": null }'

Windows

curl -XPUT -H "Content-Type:application/json" http://localhost:9200/_all/_settings -d "{ \"index.blocks.read_only_allow_delete\": null }"

Disclaimer