NOAA 3-D Surge and Tide Operational Forecast System for the Atlantic Basin (STOFS-3D-Atlantic) on AWS

Overview 🔗

The 3-D component of the STOFS, namely, STOFS-3D-Atlantic provides users with the 24-hour nowcasts (analyses of near present conditions) and up to 96-hour forecast guidance of the water level conditions, as well as 2- and 3-dimensional fields of water temperature, salinity, and currents. STOFS-3D-Atlantic is run one time (on 12 UTC) per day. The system domain encompasses the U.S. East Coast, Gulf of Mexico, and Puerto Rico, and Gulf of St. Lawrence. The water levels produced by the STOFS-3D-Atlantic represent the combined tidal and subtidal water surface elevations and are unanimously referenced to xGEOID20B. STOFS-3D-Atlantic is providing a new variable called “water level disturbance”, which is defined as water level height above xGEOID20B for topobathy below xGEOID20B (e.g. ocean), and as water level height above ground for topobathy above xGEOID20B (e.g. land). Maximum height in the STOFS-3D-Atlantic grid is 10 m above xGEOID20B.

STOFS-3D-Atlantic employs the Semi-implicit Cross-scale Hydroscience Integrated System Model (SCHISM) as the hydrodynamic model core. Its unstructured model grid consists of 2,926,236 nodes and 5,654,157 triangular or quadrilateral elements. The total number of 3D grid points is 31,307,038. Grid resolution is 1.5-2 km near the shoreline, ~600 m for the floodplain, down to 8 m for watershed rivers (at least 3 nodes across each river cross-section), and around 2-10 m for levees. Along the U.S. coastline, the land boundary of the domain aligns with the 10-m contour above xGEOID20B, encompassing the coastal transitional zone most vulnerable to coastal and inland flooding. The number of vertical layers ranges from 49 in the deepest part of the Atlantic Ocean to just 1 layer in floodplain areas. The model run uses a computational time step of 150 s. In total, STOFS-3D-Atlantic currently simulates 1 day of simulation in 0.40 wall clock hours on 4320 computational cores.

The files provided here include GRIB2 format for regional grids and SHEF format for station time series; both GRIB2 and SHEF are coincidentally provided to National Weather Service forecasters through AWIPS. The files are also provided here in native netCDF format, which include the full native resolution model grids and station time series; and the geopackage (gpkg) files for the field time series of water level disturbance over the entire domain.

This service is made possible through a public-private partnership enabled by the NOAA Open Data Dissemination (NODD).

File Format

The files are in NETCDFGRIB2, and SHEF format. Both NOAA and NASA provide tools for accessing NETCDF files:

NOAA provides tools for accessing GRIB2 files:

More details on SHEF files are available here.

 

Accessing the Data on AWS

Access to the datasets is being provided on Amazon Web Services (AWS) via Simple Storage Service (S3). Files can be downloaded via the native AWS S3 API using tools such as the AWS CLI or AWS SDK libraries such as the Python boto3 package.

AWS S3 also provides direct HTTP access to the files, so datasets can be downloaded using any HTTP client such as web browser. To make discovering and accessing the data easier, a simple bucket explorer web application is also provided which allows you to navigate through the object key structure and download files using a web browser.

The Amazon Resource Name (ARN) for the S3 Bucket is: arn:aws:s3::: noaa-nos-stofs3d-pds/STOFS-3D-Atl.

Here is an example of using the AWS CLI to list a particular day in the S3 bucket:

·        aws s3 ls s3://noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.YYYYMMDD

·        Where YYYY = year, MM = month, DD = day

Additionally, an AWS Simple Notification Service (SNS) Topic has also been created to provide automated notifications when new files are added to the S3 Bucket. Users who operate their own infrastructure on AWS can integrate directly with this SNS Topic to trigger automated workflows using AWS Lambda or other services.

The Amazon Resource Name (ARN) for the SNS Topic is: arn:aws:sns:us-east-1:709902155096:NewICOGS3DObject.

Data Organization 🔗

NOAA's operational STOFS-3D-Atlantic output is available here: s3://noaa-nos-stofs3d-pds/STOFS-3D-Atl/

NOAA's collaborator, Virginia Institute of Marine Science (VIMS), runs a shadow model for STOFS-3D-Atlantic. Its output is available here: s3://noaa-nos-stofs3d-pds/STOFS-3D-Atl-shadow-VIMS/

All of the text below describes data organization only for NOAA's operational STOFS-3D-Atlantic.

Files are organized by day, i.e., all model outputs on a particular day are included in that day's directory, e.g.:

·        stofs_3d_atl.YYYYMMDD

·        Where YYYY = year, MM = month, DD = day

Within each directory, the below files are present. Use this key to interpret file names:

·        CC is the model run cycle, 12, for the nowcast and forecast runs

·        HHH is the nowcast hour (i.e., 023 - 000) or the forecast hour (i.e., 000 – 096).

·        cwl stands for combined (subtidal plus tidal) water level (unit: m) referenced to xGEOID20B.

·        maxele stands for maximum water level elevation across entire forecast cycle

File Name

Content

Resolution

File Type

stofs_3d_atl.tCCz.conus.east.fHHH.grib2

Combined water level (m, xGEOID20B) field on Regional Eastern CONUS gridded files

2.5 km

GRIB2

stofs_3d_atl.tCCz.conus.east.cwl.grib2

Hourly combined water level (m, xGEOID20B) field on Regional Eastern CONUS gridded files for all forecast hours

2.5 km

GRIB2

stofs_3d_atl.tCCz.puertori.fHHH.grib2

Combined water level (m, xGEOID20B) field on Regional Puerto Rico gridded files at forecast HHH

1.25 km

GRIB2

stofs_3d_atl.tCCz.puertori.cwl.grib2

Hourly combined water level (m, xGEOID20B) field on Regional Puerto Rico gridded files for all forecast hours

1.25 km

GRIB2

stofs_3d_atl.tCCz.hotstart.stofs3d.nc

Model run hotstart file (this serves as the restart file, to provide initial 3-D ocean state variables for the next cycle)

Native model

NetCDF

schout_adcirc_YYYYMMDD.nc

Hourly fields of combined (subtidal plus tidal) water level (m, xGEOID20B), maximum water level disturbance, time of occurrence of maximum water level/maximum water level disturbance, and 10 m wind field on ADCIRC-like triangular grid; totally three files for the nowcast (n023 to n000), and forecast guidance days 1 (f001 to f024), 2 (f025 to f048), up to 4 (f073 to f096).

Triangular grid

NetCDF

stofs_3d_atl.tCCz.field2d_fhhh1_hhh2.nc

Hourly forecast guidance (fhhh1 to fhhh2) fields of combined water level (m, xGEOID20B), water temperature, salinity, and horizontal water velocity at both surface and bottom, and horizontal water velocity at 4.5 m below the water surface. hhh1_hhh2 denotes {001_012,013_024,025_036,037_048,049_060,061_072,073_084,085_096}.

Native model

NetCDF

stofs_3d_atl.tCCz.field2d_nhhh1_hhh2.nc

Hourly nowcast guidance (n023 to n000) fields of combined water level (m, xGEOID20B), and water temperature, salinity, horizontal water velocity at both surface and bottom, and horizontal water velocity at 4.5 m below the water surface.  hhh1_hhh2 denotes {001_012,013_024}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.out2d_fhhh1_hhh2.nc

Hourly forecast (f001 to f096) fields of combined water level (m, xGEOID20B) and water-depth averaged horizontal water velocity. hhh1_hhh2 denotes {001_012,013_024,025_036,037_048,049_060,061_072,073_084,085_096}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.out2d_nhhh1_hhh2.nc

Hourly nowcast (n023 to n000) fields of combined water level (m, xGEOID20B) and water-depth averaged horizontal water velocity.  hhh1_hhh2 denotes {001_012,013_024}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.cwl.maxele.nc

Maximum of nowcast and forecast (hours n023 to n000, f001 to f096) combined water level (m, xGEOID20B) field

Native model

NetCDF

stofs_3d_atl.tCCz.fcast.station.profile.nc

Hourly forecast (f001 to f096) combined water level (m, xGEOID20B) and horizontal 10 m wind velocity, and vertical profiles of water temperature, salinity, horizontal water velocity at verification sites

--

NetCDF

stofs_3d_atl.tCCz.ncast.station.profile.nc

Hourly nowcast (n023 to n000) combined water level (m, xGEOID20B) and horizontal 10 m wind velocity, and vertical profiles of water temperature, salinity, horizontal water velocity at verification sites

--

NetCDF

stofs_3d_atl.tCCz.points.cwl.nc

Six-minute nowcast and forecast combined water level (m, NAVD88) at verification sites

--

NetCDF

stofs_3d_atl.tCCz.points.cwl.temp.salt.vel.nc

Six-minute nowcast and forecast combined water level (m, NAVD88), surface water temperature, surface salinity, and surface horizontal water velocity at verification sites

--

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelX_ fhhh1_hhh2.nc

Hourly forecast (between hours f001 and f096) 3-D horizontal water velocity fields in east-west direction. hhh1_hhh2 denotes {001_012,013_024,025_036,037_048,049_060,061_072,073_084,085_096}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelX_nhhh1_hhh2.nc

Hourly nowcast (between hours n023 and n000) 3-D horizontal water velocity fields in east-west direction. hhh1_hhh2 denotes {001_012,013_024}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelY_ fhhh1_hhh2.nc

Hourly forecast (between hours f001 and f096) 3-D horizontal water velocity fields in north-south direction. hhh1_hhh2 denotes {001_012,013_024,025_036,037_048,049_060,061_072,073_084,085_096}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.horizontalVelY_ nhhh1_hhh2.nc

Hourly nowcast (between hours n023 and n000) 3-D horizontal water velocity fields in north-south direction.  hhh1_hhh2 denotes {001_012,013_024}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.salinity_ fhhh1_hhh2.nc

Hourly forecast (between hours f001 and f096) 3-D salinity fields. hhh1_hhh2 denotes {001_012,013_024,025_036,037_048,049_060,061_072,073_084,085_096}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.salinity_nhhh1_hhh2.nc

Hourly nowcast (between hours n023 and n000) 3-D salinity fields.  hhh1_hhh2 denotes {001_012,013_024}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.temperature_ fhhh1_hhh2.nc

Hourly forecast (between hours f001 and f096) 3-D temperature fields. hhh1_hhh2 denotes {001_012,013_024,025_036,037_048,049_060,061_072,073_084,085_096}.

Native model

NetCDF

stofs_3d_atl.tCCz.fields.temperature_nhhh1_hhh2.nc

Hourly nowcast (between hours n023 and n000) 3-D temperature fields.  hhh1_hhh2 denotes {001_012,013_024}.

Native model

NetCDF

stofs_3d_atl.tCCz.points.cwl.shef

30-minute combined water level (ft, MLLW) at verification sites, for the entire forecast period 

--

SHEF

stofs_3d_atl.tCCz.disturbance.fHHH.gpkg

Hourly water level disturbance in the forecast period

Native model

GPKG

stofs_3d_atl.tCCz.disturbance.nHHH.gpkg

Hourly water level disturbance in the nowcast period

Native model

GPKG

wmo/shef_stofs_3d_atl.t12z.points.cwl

30-minute combined water level (ft, MLLW) at verification sites, for the entire forecast period

--

CWL

wmo/grib2_stofs_3d_atl.t12z.puertori.cwl

Hourly combined water level (m, xGEOID20B) field on Regional Puerto Rico gridded files for all forecast hours

1.25 km

CWL

wmo/grib2_stofs_3d_atl.t12z.conus.east.cwl

Hourly combined water level (m, xGEOID20B) field on Regional Eastern CONUS gridded files for all forecast hours

1.25 km

CWL

rerun/

This is the folder that holds the initial condition and forcing files for the system run on YYYYMMDD. The files are stored for the NOAA internal reference. Hence no further details are given here.

--

--

./post_processing/

This is the folder that holds post processing for STOFS-3D-Atlantic, including skill assessment. The files are mainly stored for NOAA internal reference. Hence no further details are given here.

--

--

../para/

This is the folder that holds the parallel run (i.e. before the model or model upgrade is operational) for STOFS-3D-Atlantic. The files are mainly stored for NOAA internal reference. Hence no further details are given here.

--

--

 

Autoval Coastline File                                                      https://noaa-nos-stofs3d-pds.s3.amazonaws.com/index.html#STOFS-3D-Atl/coastline/coastline.dat

 

SCHISM Static Files

stofs-3d-atl v2.1: May 14, 2024 12 UTC cycle – present

       https://noaa-nos-stofs3d-pds.s3.amazonaws.com/index.html#STOFS-3D-Atl/staticfiles/v2.1/

stofs-3d-atl v1.1: January 12, 2023 12 UTC cycle – May 13, 2024 12 UTC cycle

       https://noaa-nos-stofs3d-pds.s3.amazonaws.com/index.html#STOFS-3D-Atl/staticfiles/v1.1/

 

File Name

Content

stofs_3d_atl_albedo.gr3

Specification of the magnitudes of albedo

stofs_3d_atl_diffmax.gr3

Specification of maximum diffusivity/viscosity

stofs_3d_atl_diffmin.gr3

Specification of minimum diffusivity/viscosity

stofs_3d_atl_drag.gr3

Bottom friction coefficient on model grid

stofs_3d_atl_hgrid.2dm

node centered spatial data and mesh connectivity

stofs_3d_atl_hgrid.gr3

node centered spatial data and mesh connectivity

stofs_3d_atl_hgrid.ll

node centered spatial data and mesh connectivity

stofs_3d_atl_param.nml_6globaloutput

Parameter name list file. The corresponding file for each run cycle is in rerun/ stofs_3d_atl.t12z.param.nml

stofs_3d_atl_partition.prop

Specification of model domain partition in support of multi-processor parallel computation

stofs_3d_atl_shapiro.gr3

Coefficient of the Shapiro filter

stofs_3d_atl_station.in

Specification of the locations and physical parameters for the station time series output

stofs_3d_atl_tvd.prop

Specification of horizontal regions for various types of numerical, horizontal transport schemes (e.g., upwind or TVD/WENO)

stofs_3d_atl_vgrid.in

Vertical grid file

stofs_3d_atl_watertype.gr3

Specification of water types

stofs_3d_atl_windrot_geo2proj.gr3

Angle differences between model’s y-axis and true north

For example:

·        noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.20221220/stofs_3d_atl.t12z.conus.east.cwl.grib2

·        noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.20221220/stofs_3d_atl.t12z.field2d_f001_012.nc

·        noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.20221220/stofs_3d_atl.t12z.points.cwl.shef

·        noaa-nos-stofs3d-pds/STOFS-3D-Atl/stofs_3d_atl.20221220/stofs_3d_atl.t12z.disturbance.f005.gpkg

·        etc.

 

Contact Information 🔗

For questions regarding data content or quality, visit the STOFS site. For any questions regarding data delivery not associated with this platform or any general questions regarding the NOAA Open Data Dissemination (NODD), email nodd@noaa.gov.

AWS Public Datasets