.. _Configuration: ============================================================================== Configuration ============================================================================== Config Creation ============================================================================== The final EAGLE YAML config is created by composing a base config together with one or more fragments providing values for specific platforms, use cases, etc. The command ``make config compose=a:b:c`` would compose together ``config/a.yaml``, ``config/b.yaml``, and ``config/c.yaml``. In practice, composition should begin with the ``base`` config (i.e. ``config/base.yaml``), which provides generally applicable settings for EAGLE runs (see the Quickstart Guide for an example.) The composed config can then be manually edited for experiment-specific requirements. For advanced use cases, for example for composing configs in arbitrary locations, the underlying :term:`uwtools` command can be used. In the ``src/`` directory: .. code-block:: bash source conda/etc/profile.d/conda.sh conda activate base uw config compose /path/to/some/a.yaml /path/to/another/b.yaml >eagle.yaml exit Individual sections of the configuration are discussed in detail below. Config Description ============================================================================== The following subsections describe various parts of the EAGLE YAML configuration. Some configuration parameters are common across ``uwtools``-based component drivers and occur in multiple configuration blocks: ``execution`` The `execution `_ block provides information required to correctly execute the component. ``platform`` The `platform `_ block provides information about the system EAGLE is running on. ``rundir`` The ``rundir:`` parameter specifies where driver runtime assets will be created. ``common`` Many configuration blocks include a ``common:`` block, which provides parameters shared by several configurations, to avoid unnecessary repetition. app ------------------------------------------------------------------------------ This block provides various global configuration parameters for the application, especially those thought most likely to require configuration by users. grids_and_meshes ------------------------------------------------------------------------------ Configuration for the ``GridsAndMeshes`` driver. ``filenames`` The ``filenames:`` block provides paths to data files created by this step. inference ------------------------------------------------------------------------------ Configuration for the ``Inference`` driver. ``anemoi`` The ``anemoi:`` block provides the YAML config for the `anemoi-inference `_ :term:`anemoi` component. ``checkpoint_dir`` The ``checkpoint_dir:`` parameter specifies the location of the checkpoints created by the training step. platform ------------------------------------------------------------------------------ In the EAGLE base config, this ``uwtools``-required parameter delegates to ``app.platform``. prewxvx ------------------------------------------------------------------------------ This block provides general configuration parameters for the :term:`prewxvx` component executed by the ``WXVX`` driver. This configuration is used as a source for default and common configuration parameters, which are supplemented by the WXVX driver when it executes ``eagle-tools prewxvx``. ``global`` and ``lam`` The ``global:`` and ``lam:`` blocks provide configurations for global and :term:`LAM` extents, respectively, each borrowing from ``common:``. Their ``prewxvx:`` sub-blocks are ultimately passed to the ``WXVX`` driver as the runtime configuration for the ``eagle-tools``' ``prewxvx`` component. training ------------------------------------------------------------------------------ Configuration for the ``Training`` driver. ``anemoi`` The ``anemoi:`` block provides the YAML config for the `anemoi-training `_ component. ``remove`` The ``remove:`` block specifies values from the default configurations `generated by anemoi `_ that should be removed at execution time, via `command-line config overrides `_. ufs2arco ------------------------------------------------------------------------------ This block provides general configuration parameters for the :term:`ufs2arco` component. This configuration is used as a source for default and common configuration parameters, which are supplemented by the Zarr driver when it executes ``ufs2arco`` for specific use cases. val ------------------------------------------------------------------------------ This block provides both static and derived values that are referenced by other configuration blocks. It is the appropriate place to define values that need to be shared and kept in sync across pipeline steps, but less likely to be manually modified by users like values in the ``app:`` block. visualization ------------------------------------------------------------------------------ Configuration for the ``Visualization`` driver. ``component`` This driver executes the ``eagle-tools postwxvx`` component and plots its output. ``common``, ``grid2grid``, and ``grid2obs`` The ``common:`` block provides parameters shared by the ``grid2grid:`` and ``grid2obs:`` blocks, which provide configurations for running ``eagle-tools postwxvx`` to create netCDF files containing verification statistics that are then plotted by the driver. Sub-blocks ``global:`` and ``lam:`` provide configuration refinements for verifying global and limited-area grids, respectively. vx ------------------------------------------------------------------------------ Configuration for the ``VX`` driver. ``component`` This driver executes the :ref:`Verify Model Performance ` component. ``grid2grid`` The ``grid2grid:`` block provides configuration for running :term:`wxvx` with MET's `grid_stat `_ tool to verify against gridded analyses. Sub-blocks ``global:`` and ``lam:`` provide configuration refinements for verifying global and limited-area grids, respectively. ``grid2obs`` The ``grid2obs:`` block provides configuration for running ``wxvx`` with MET's `point_stat `_ tool to verify against point observations. Sub-blocks ``global:`` and ``lam:`` provide configuration refinements for verifying global and limited-area grids, respectively. zarrs ------------------------------------------------------------------------------ Configuration for the ``Zarr`` driver. ``component`` This driver executes the :term:`ufs2arco` component. ``gfs`` and ``hrrr`` The ``gfs:`` and ``hrrr:`` sub-blocks provide refinements for ingesting GFS and HRRR data, respectively, for EAGLE. .. _ConfigRealization: Config Realization ============================================================================== EAGLE YAML configurations contain a variety of Jinja2 expressions that are normally resolved by the various pipeline steps at run time. Sometimes it can be helpful to resolve these references ("realize" the config in ``uwtools`` terms) in advance, to get a better understanding of the final configuration parameters. To do so, run: .. code-block:: bash make realize config=eagle.yaml The resulting config could be used in place of the unrealized ``eagle.yaml``, as the two should be equivalent -- though the realized config may be significantly longer due to, for example, repetition of common elements previously factored out using Jinja2. Note that the realized config may still contain some Jinja2 expressions that can only be realized at run time by the component using a particular config block. .. _ConfigValidation: Config Validation ============================================================================== To validate the EAGLE YAML configuration: .. code-block:: bash make validate config=eagle.yaml This will perform validation of config blocks that are not owned by drivers; driver config blocks will be validated at run time by the drivers themselves.