NOAA 3-D Surge and Tide Operational Forecast System for the Atlantic Basin (STOFS-3D-Atlantic) on AWS

Overview 🔗

The 3-D component of the STOFS, namely, STOFS-3D-Atlantic provides users with the 24-hour nowcasts (analyses of near present conditions) and up to 48-hour forecast guidance of the water level conditions, as well as 2- and 3-dimensional fields of water temperature, salinity, and currents. STOFS-3D-Atlantic is run one time (on 12 UTC) per day. The system domain encompasses the U.S. East Coast, Gulf of Mexico, and Puerto Rico. The water levels produced by the STOFS-3D-Atlantic represent the combined tidal and subtidal water surface elevations and are unanimously referenced to NAVD88. STOFS-3D-Atlantic is providing a new variable called “water level disturbance”, which is defined as water level height above NAVD88 for topobathy below NAVD88 (e.g. ocean), and as water level height above ground for topobathy above NAVD88 (e.g. land). Maximum height in the STOFS-3D-Atlantic grid is 10 m above NAVD88.

STOFS-3D-Atlantic employs the Semi-implicit Cross-scale Hydroscience Integrated System Model (SCHISM) as the hydrodynamic model core. Its unstructured model grid consists of 2,654,153 nodes and 5,137,229 triangular or quadrilateral elements. The total number of 3D grid points is 24,088,817. Grid resolution is 1.5-2 km near the shoreline, 200-550 m for the floodplain, and up to 10 m for both levees and some small rivers. Along the U.S. coastline, the land boundary of the domain aligns with the 10-m contour above NAVD88, encompassing the coastal transitional zone most vulnerable to coastal and inland flooding. The number of vertical layers ranges from 49 in the deepest part of the Atlantic Ocean to just 1 layer in floodplain areas. The model run uses a computational time step of 150 s. In total, STOFS-3D-Atlantic currently simulates 1 day of simulation in 0.41 wall clock hours on 3,480 computational cores.

The files provided here include GRIB2 format for regional grids and SHEF format for station time series; both GRIB2 and SHEF are coincidentally provided to National Weather Service forecasters through AWIPS. The files are also provided here in native netCDF format, which include the full native resolution model grids and station time series; and the geopackage (gpkg) files for the field time series of water level disturbance over the entire domain.

This service is made possible through a public-private partnership enabled by the NOAA Open Data Dissemination (NODD).

File Format

The files are in NETCDF, GRIB2, SHEF, and GPKG format. Both NOAA and NASA provide tools for accessing NETCDF files:

·         Panoply is an application from NASA that allows users to view and graph data that are contained in NETCDF files.

·         ncdump is a software library that can be used for extracting data from a NETCDF file. It is aimed at advanced users.

·         Other tools are also available here.

NOAA provides tools for accessing GRIB2 files:

·         NCEP GRIB2 tools include wgrib2 and g2lib.

·         wgrib2 is a utility to read and write grib2 files. This link provides more details on the utility.

More details on SHEF files are available here.

Python Package Index (PyPI) provides utilities to read and write GPKG files:

·         geopackage-python is the python-based software tool to read and write GeoPackage (gpkg) files.

 

Accessing the Data on AWS

Access to the datasets is being provided on Amazon Web Services (AWS) via Simple Storage Service (S3). Files can be downloaded via the native AWS S3 API using tools such as the AWS CLI or AWS SDK libraries such as the Python boto3 package.

AWS S3 also provides direct HTTP access to the files, so datasets can be downloaded using any HTTP client such as web browser. To make discovering and accessing the data easier, a simple bucket explorer web application is also provided which allows you to navigate through the object key structure and download files using a web browser.

The Amazon Resource Name (ARN) for the S3 Bucket is: arn:aws:s3::: noaa-nos-stofs3d-pds/STOFS-3D-Atl.

Here is an example of using the AWS CLI to list a particular day in the S3 bucket:

·         aws s3 ls s3://noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.YYYYMMDD

·         Where YYYY = year, MM = month, DD = day

Additionally, an AWS Simple Notification Service (SNS) Topic has also been created to provide automated notifications when new files are added to the S3 Bucket. Users who operate their own infrastructure on AWS can integrate directly with this SNS Topic to trigger automated workflows using AWS Lambda or other services.

The Amazon Resource Name (ARN) for the SNS Topic is: arn:aws:sns:us-east-1:709902155096:NewICOGS3DObject.

Data Organization 🔗

NOAA's operational STOFS-3D-Atlantic output is available here: s3://noaa-nos-stofs3d-pds/STOFS-3D-Atl/

NOAA's collaborator, Virginia Institute of Marine Science (VIMS), runs a shadow model for STOFS-3D-Atlantic. Its output is available here: s3://noaa-nos-stofs3d-pds/STOFS-3D-Atl-shadow-VIMS/

All of the text below describes data organization only for NOAA's operational STOFS-3D-Atlantic.

Files are organized by day, i.e., all model outputs on a particular day are included in that day's directory, e.g.:

·         stofs_3d_atl.YYYYMMDD

·         Where YYYY = year, MM = month, DD = day

Within each directory, the below files are present. Use this key to interpret file names:

·         CC is the model run cycle, 12, for the nowcast and forecast runs

·         HHH is the nowcast hour (i.e., 023 - 000) or the forecast hour (i.e., 000 – 048).

·         cwl stands for combined (subtidal plus tidal) water level (unit: m) referenced to NAVD88.

·         maxele stands for maximum water level elevation across entire forecast cycle

File Name

Content

Resolution

File Type

stofs_3d_atl.tCCz.conus.east.fHHH.grib2

Combined water level (m, NAVD88) field on Regional Eastern CONUS gridded files

2.5 km

GRIB2

stofs_3d_atl.tCCz.conus.east.cwl.grib2

Hourly combined water level (m, NAVD88) field on Regional Eastern CONUS gridded files for all forecast hours

2.5 km

GRIB2

stofs_3d_atl.tCCz.puertori.fHHH.grib2

Combined water level (m, NAVD88) field on Regional Puerto Rico gridded files at forecast HHH

1.25 km

GRIB2

stofs_3d_atl.tCCz.puertori.cwl.grib2

Hourly combined water level (m, NAVD88) field on Regional Puerto Rico gridded files for all forecast hours

1.25 km

GRIB2

stofs_3d_atl.tCCz.hotstart.stofs3d.nc

Model run hotstart file (this serves as the restart file, to provide initial 3-D ocean state variables for the next cycle)

Native model

NetCDF

schout_adcirc_YYYYMMDD.nc

Hourly fields of combined (subtidal plus tidal) water level (m, NAVD88), maximum water level disturbance, time of occurrence of maximum water level/maximum water level disturbance, and 10 m wind field on ADCIRC-like triangular grid; totally three files for the nowcast (n023 to n000), and forecast guidance days 1 (f001 to f024) & 2 (f025 to f048)

Triangular grid

NetCDF

stofs_3d_atl.tCCz.f001_024.field2d.nc

Hourly forecast guidance (f001 to f024) fields of combined water level (m, NAVD88), water temperature, salinity, and horizontal water velocity at both surface and bottom, and horizontal water velocity at 4.5 m below the water surface

Native model

NetCDF

stofs_3d_atl.tCCz.f025_048.field2d.nc

Hourly forecast guidance (f025 to f048) fields of combined water level (m, NAVD88), and water temperature, salinity, and horizontal water velocity at both surface and bottom, and horizontal water velocity at 4.5 m below the water surface 

Native model

NetCDF

stofs_3d_atl.tCCz.n001_024.field2d.nc

Hourly nowcast guidance (n023 to n000) fields of combined water level (m, NAVD88), and water temperature, salinity, horizontal water velocity at both surface and bottom, and horizontal water velocity at 4.5 m below the water surface 

Native model

NetCDF

stofs_3d_atl.tCCz.fields.out2d_forecast_day1.nc

Hourly forecast (f001 to f024) fields of combined water level (m, NAVD88) and water-depth averaged horizontal water velocity

Native model

NetCDF

stofs_3d_atl.tCCz.fields.out2d_forecast_day2.nc

Hourly forecast (f025 to f048) fields of combined water level (m, NAVD88) and water-depth averaged horizontal water velocity

Native model

NetCDF

stofs_3d_atl.tCCz.fields.out2d_nowcast.nc

Hourly nowcast (n023 to n000) fields of combined water level (m, NAVD88) and water-depth averaged horizontal water velocity

Native model

NetCDF

stofs_3d_atl.tCCz.fields.cwl.maxele.nc

Maximum of nowcast and forecast (hours n023 to n000, f001 to f048) combined water level (m, NAVD88) field

Native model

NetCDF

stofs_3d_atl.tCCz.fcast.station.profile.nc

Hourly forecast (f001 to f048) combined water level (m, NAVD88) and horizontal 10 m wind velocity, and vertical profiles of water temperature, salinity, horizontal water velocity at verification sites

--

NetCDF

stofs_3d_atl.tCCz.ncast.station.profile.nc

Hourly nowcast (n023 to n000) combined water level (m, NAVD88) and horizontal 10 m wind velocity, and vertical profiles of water temperature, salinity, horizontal water velocity at verification sites

--

NetCDF

stofs_3d_atl.tCCz.points.cwl.nc

Six-minute nowcast and forecast combined water level (m, NAVD88) at verification sites

--

NetCDF

stofs_3d_atl.tCCz.points.cwl.temp.salt.vel.nc

Six-minute nowcast and forecast combined water level (m, NAVD88), surface water temperature, surface salinity, and surface horizontal water velocity at verification sites

--

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelX_forecast_day1.nc

Hourly forecast (between hours f001 and f024) 3-D horizontal water velocity fields in east-west direction

Native model

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelX_forecast_day2.nc

Hourly forecast (between hours f025 and f048) 3-D horizontal water velocity fields in east-west direction

Native model

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelX_nowcast.nc

Hourly nowcast (between hours n023 and n000) 3-D horizontal water velocity fields in east-west direction

Native model

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelY_forecast_day1.nc

Hourly forecast (between hours f001 and f024) 3-D horizontal water velocity fields in north-south direction

Native model

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelY_forecast_day2.nc

Hourly forecast (between hours f025 and f048) 3-D horizontal water velocity fields in north-south direction

Native model

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelY_nowcast.nc

Hourly nowcast (between hours n023 and n000) 3-D horizontal water velocity fields in north-south direction

Native model

NetCDF

stofs_3d_atl.tCCz.fields.salinity_forecast_day1.nc

Hourly forecast (between hours f001 and f024) 3-D salinity fields

Native model

NetCDF

stofs_3d_atl.tCCz.fields.salinity_forecast_day2.nc

Hourly forecast (between hours f025 and f048) 3-D salinity fields

Native model

NetCDF

stofs_3d_atl.tCCz.fields.salinity_nowcast.nc

Hourly nowcast (between hours n023 and n000) 3-D salinity fields

Native model

NetCDF

stofs_3d_atl.tCCz.fields.temperature_forecast_day1.nc

Hourly forecast (between hours f001 and f024) 3-D temperature fields

Native model

NetCDF

stofs_3d_atl.tCCz.fields. temperature _forecast_day2.nc

Hourly forecast (between hours f025 and f048) 3-D temperature fields

Native model

NetCDF

stofs_3d_atl.tCCz.fields. temperature _nowcast.nc

Hourly nowcast (between hours n023 and n000) 3-D temperature fields

Native model

NetCDF

stofs_3d_atl.tCCz.points.cwl.shef

30-minute combined water level (ft, MLLW) at verification sites, for the entire forecast period 

--

SHEF

stofs_3d_atl.tCCz.disturbance.fHHH.gpkg

Hourly water level disturbance in the forecast period

Native model

GPKG

stofs_3d_atl.tCCz.disturbance.nHHH.gpkg

Hourly water level disturbance in the nowcast period

Native model

GPKG

rerun/

This is the folder that holds the initial condition and forcing files for the system run on YYYYMMDD. The files are stored for the NOAA internal reference. Hence no further details are given here.

--

--

../post_processing/

This is the folder that holds post processing for STOFS-3D-Atlantic, including skill assessment. The files are mainly stored for NOAA internal reference. Hence no further details are given here.

--

--

../para/

This is the folder that holds the parallel run (i.e. before the model or model upgrade is operational) for STOFS-3D-Atlantic. The files are mainly stored for NOAA internal reference. Hence no further details are given here.

--

--

For example:

·         noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.20221220/stofs_3d_atl.t12z.conus.east.cwl.grib2

·         noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.20221220/stofs_3d_atl.t12z.f001_024.field2d.nc

·         noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.20221220/stofs_3d_atl.t12z.points.cwl.shef

·         noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.20221220/stofs_3d_atl.t12z.disturbance.f005.gpkg

·         etc.

Contact Information 🔗

For questions regarding data content or quality, visit the STOFS site. For any questions regarding data delivery not associated with this platform or any general questions regarding the NOAA Open Data Dissemination (NODD), email nodd@noaa.gov.

AWS Public Datasets