A Unified Framework for IOOS Model Data Access
Project Lead/Scientific PI: Sean C. Arms, UCAR/Unidata
Transition PI: Kyle Wilcox, Axiom Data Science
NOAA/NOS Technical Points of Contact: Gregory Seroka, John Kelley, Edward Myers, Aijun Zhang, Jiangtao Xu
Project Overview and Results
In the last decade, IOOS has made remarkable progress toward an interoperable framework for working with ocean forecast model data. Providers deliver data via community standard OPeNDAP services using Climate and Forecast (CF) metadata conventions, and developers then use standard libraries to build applications that will work with any structured grid model (e.g. ROMS, NCOM, HYCOM, Delft3D) without the need to write any model-specific code. An outstanding gap, however, is the ability to fully support unstructured grid (e.g. triangular mesh) models, which are now being used throughout IOOS for both inundation and full 3D operational ocean forecasts.
The need for better tools to work with unstructured grids has been apparent for some time: In 2006, NOAA and NSF co-sponsored a workshop, “Community Standards for Unstructured Grids,” which was attended by 30 representatives from academic, government, and commercial organizations. The consensus statement of the workshop was “The unstructured ocean model community should define a data model (in the language of OPeNDAP) and ensure the mapping of that model onto netCDF3 and HDF5 is functional”. The community subsequently developed the UGRID Conventions, which extend the CF metadata conventions to describe unstructured grid model output. It remains to embed these standards into the services most commonly used by the IOOS community.
While IOOS-supported services can be delivered and consumed by a variety of software components, Unidata’s THREDDS Data Server (TDS) is the most widely used in IOOS. The TDS not only provides OPeNDAP access, but provides the OGC Web Map Service to create maps and NetCDF Subset Service to extract datasets based on longitude/latitude and calendar date extents. The Unidata NetCDF-Java library, which underlies the TDS, also supports many client applications (e.g. Unidata IDV, Matlab NCTOOLBOX, University of Reading NcWMS, NASA Panoply).
IOOS COMT 1.0 supported the development of the UGRID Conventions and the development of pilot UGRID-compliant tools, such as the pyugrid python library and the prototype ugrid branch of the NCTOOLBOX Matlab toolbox. The NCTOOLBOX work showed the promise of supporting UGRID in NetCDF-Java, enabling interoperable access to water level predictions -- a strong focus in COMT 1.0. The code was not integrated into the Unidata code base, however, and significant technical and code development (with documentation, testing, support) are still needed to bring unstructured grid models to the same level of functionality as structured grid models.
We propose to build on the success of this framework and these proof-of-concept demonstrations, extending the NetCDF-Java library to allow unstructured grid models to be “first-class citizens” in the Unidata software stack. While Unidata developers have long been interested in developing these capabilities, Unidata is primarily driven by the needs of atmospheric forecast model users. Unidata’s technical expertise as developers of the THREDDS technologies, coupled with Axiom Data Science’s working relationship with the IOOS Regional Association partners, will help ensure that this gap in functionality is closed and that the new features are communicated to potential users efficiently. We hope COMT will help bring this long-needed functionality to the IOOS community.