View Source

Import type NETCDF-CF_GRID_S3 imports grids from nc (and sometimes grib2) files stored in S3 buckets.

To find the URLs to the required S3 buckets and to list the nc files stored in the buckets, the Amazon Web Services (AWS) are used.
The listed nc files are read using the existing import type NETCDF-CF_GRID. It means that all features of NETCDF-CF_GRID are also available for reading from S3 buckets.

Importing data for certain period

To import data from multiple S3 Url’s for certain period, configure:

keyword DATE_TIME together with date/time pattern in the serverUrl, for example %DATE_TIME(yyyy)%
relativeViewPeriod to indicate for which period the data should be imported
optional fileNameObservationDateTimePattern if only selected files should be imported, for example only last 10 days before T0

Example for CMEMS data

<serverUrl>https://s3.waw3-1.cloudferro.com/mdl-native-14/native/GLOBAL_ANALYSISFORECAST_PHY_001_024/cmems_mod_glo_phy_anfc_0.083deg_PT1H-m_202211/%DATE_TIME(yyyy)%/%DATE_TIME(MM)%</serverUrl>	
<fileNameObservationDateTimePattern>'glo12_rg_1h-m_'yyyyMMdd'?'</fileNameObservationDateTimePattern> 		
<relativeViewPeriod unit="day" start="-10" end="0"/>

If the T0 is 01-May-2024 23:00:00 , then this example imports data for 10 days, from 21-04-2024 to 01-05-2024, from these S3 folders ./2024/04 and ./2024/05

Importing data without period specification

Configuration example 1: single file glo12_rg_1h.nc will be imported

<importType>NETCDF-CF_GRID_S3</importType>
<serverUrl>https://s3.waw3-1.cloudferro.com/mdl-native-14/native/GLOBAL_ANALYSISFORECAST_PHY_001_024/cmems_mod_glo_phy_anfc_0.083deg_PT1H-m_202211/2024/03/glo12_rg_1h.nc</serverUrl>

Configuration example 2: all nc files under the folder /03 will be listed and imported. In this example all files in month March

<importType>NETCDF-CF_GRID_S3</importType>
<serverUrl>https://s3.waw3-1.cloudferro.com/mdl-native-14/native/GLOBAL_ANALYSISFORECAST_PHY_001_024/cmems_mod_glo_phy_anfc_0.083deg_PT1H-m_202211/2024/03</serverUrl>

Configuration example 3: all nc files under the folder /2024 will be listed and imported. In this example all files available for year 2024

<importType>NETCDF-CF_GRID_S3</importType>
<serverUrl>https://s3.waw3-1.cloudferro.com/mdl-native-14/native/GLOBAL_ANALYSISFORECAST_PHY_001_024/cmems_mod_glo_phy_anfc_0.083deg_PT1H-m_202211/2024</serverUrl>

Configuration example 4: all nc files under the folder /cmems_mod_glo_phy_anfc_0.083deg_PT1H-m_202211 will be listed and imported. In this example all files available for all years under the mentioned folder

<importType>NETCDF-CF_GRID_S3</importType>
<serverUrl>https://s3.waw3-1.cloudferro.com/mdl-native-14/native/GLOBAL_ANALYSISFORECAST_PHY_001_024/cmems_mod_glo_phy_anfc_0.083deg_PT1H-m_202211</serverUrl>

Example for ECMWF open data

This How-to page provides a configuration example for importing ECMWF open data using the NETCDF-CF_GRID_S3 importType.