METSIM: Meteorology Simulator¶
MetSim is a meteorological simulator and forcing disaggregator for hydrologic modeling and climate applications. Metsim is based on MtClim and the preprocessor from version 4 of the VIC hydrologic model.
MetSim consists of 3 main modules that govern the operation of 3 major aspects of its operation:
1. Management of dataset preprocessing and IO
The MetSim object provides high level support for setting up jobs and infrastructure for running simulation/disaggregation steps. It is the main interface through which the other modules are accessed.
2. Simulation of daily meteorological forcings
The base implementation of the meteorological simulator is based off of the algorithms described in [1]. This component has been designed to be flexible in allowing for alternative implementations which may be specified during the setup of the MetSim object. The default implementation allows for the daily simulation of:
- Mean daily temperature
- Incoming shortwave radiation
- Cloud cover fraction
- Potential evapotranspiration
- Vapor pressure
3. Disaggregation of daily simulation values to sub-daily timesteps
Daily data from given input or simulated via the forcings generation component of MetSim can be disaggregated down to sub-daily values at intervals specified in minutes (provided they divide evenly into 24 hours). The operation of these algorithms is also described in [1]. The variables estimated are:
- Temperature
- Vapor pressure
- Relative and specific humidity
- Air pressure
- Cloud cover fraction
- Longwave radiation
- Shortwave radiation
- Precipitation
- Wind speed
For the “triangle” and “mix” methods of precipitation disaggregation, doumentation can be found here. This will eventually be superceded by a journal article that is currently in review [7].
If you don’t find what you’re looking for here, check out MetSim’s Github page.
Getting Started¶
A tutorial for running MetSim and working with input/output data can be run via binder here: https://github.com/UW-Hydro/MetSim-tutorial
Installation¶
MetSim itself is a pure Python package, but its dependencies are not. You should ensure that you have all of the required dependencies:
- Python 3.5 or greater
- xarray (0.10.9 or later)
- pandas (0.19.0 or later)
- numba (0.31.0 or later)
- netCDF4
- scipy
Then, install MetSim with pip or conda:
$ pip install metsim
or:
$ conda install -c conda-forge metsim
Alternatively, you can install MetSim directly from the source if you desire to:
$ git clone https://github.com/UW-Hydro/MetSim.git
$ cd MetSim
$ python setup.py install
$ py.test --verbose
Basic Usage¶
MetSim provides a simple command line interface which is primarily operated via configuration files. For more information about the options available to be set in the configuration files see the configuration page.
Once installed, MetSim can be used from the command line via:
usage: ms [-h] [-n NUM_WORKERS] [-s SCHEDULER] [-v] [--version] config
positional arguments:
config Input configuration file
optional arguments:
-h, --help show this help message and exit
-n NUM_WORKERS, --num_workers NUM_WORKERS
Parallel mode: number of processes to use
-s SCHEDULER, --scheduler SCHEDULER
Dask scheduler to use
-v, --verbose Increase the verbosity of MetSim
--version Name and version number
Bracketed flags are optional; -v
activates verbose mode to print messages
about the status of a run, and -n
activates parallelism. The number given
after the -n
flag is the number of processes to run. A good rule of thumb is
to use one less process than the number of processsors (or threads) that the
machine you are running on has.
Warning
Users in environments where OpenMP is available may experience over-utilization of CPU resources, leading to lower performance. If you experience this issue try setting the OMP_NUM_THREADS environment variable to 1 before running MetSim.. This can be done in bash and similar shells by running export OMP_NUM_THREADS=1.
References¶
[1] | (1, 2) Bohn, T. J., B. Livneh, J. W. Oyler, S. W. Running, B. Nijssen, and D. P. Lettenmaier, 2013. Global evaluation of MTCLIM and related algorithms for forcing of ecological and hydrological models, Agricultural and Forest Meteorology, 176:38-49, doi:10.1016/j.agrformet.2013.03.003. |
[2] | Bristow, K.L., and G.S. Campbell, 1984. On the relationship between incoming solar radiation and daily maximum and minimum temperature. Agricultural and Forest Meteorology, 31:159-166. |
[3] | Running, S.W., R.R. Nemani, and R.D. Hungerford, 1987. Extrapolation of synoptic meteorological data in mountainous terrain and its use for simulating forest evaporation and photosynthesis. Canadian Journal of Forest Research, 17:472-483. |
[4] | Glassy, J.M., and S.W. Running, 1994. Validating diurnal climatology of the MT-CLIM model across a climatic gradient in Oregon. Ecological Applications, 4(2):248-257. |
[5] | Kimball, J.S., S.W. Running, and R. Nemani, 1997. An improved method for estimating surface humidity from daily minimum temperature. Agricultural and Forest Meteorology, 85:87-98. |
[6] | Thornton, P.E., and S.W. Running, 1999. An improved algorithm for estimating incident daily solar radiation from measurements of temperature, humidity, and precipitation. Agricultural and Forest Meteorology, 93:211-228. |
[7] | Bohn, T. J., K. M. Whitney, G. Mascaro, and E. R. Vivoni, 2019. A deterministic approach for approximating the diurnal cycle of precipitation for large-scale hydrological simulations. Journal of Hydrometeorology, 20(2):297-317. doi: 10.1175/JHM-D-18-0203.1. |
Sitemap¶
Examples¶
Basics¶
Provided in the source are several examples that can help you to
get started using MetSim. They are located in the examples
directory. For demonstration here is an example YAML configuration file:
# This is an example of an input file for MetSim
# Overall configuration, specification of parameters and input/output
# paths goes in the "MetSim" section
MetSim:
# Time step in minutes
time_step: 30
# Forcings begin here (year-month-day)
start: 1950-1-1
# Forcings end at this date (year-month-day)
stop: 1950-1-31
# Input and output directories
forcing: './metsim/data/test.nc'
domain: './metsim/data/tiny_domain.nc'
state: './metsim/data/state_nc.nc'
forcing_fmt: 'netcdf'
in_format: 'netcdf'
out_dir: './results'
out_prefix: 'yaml_output'
prec_type: 'triangle'
utc_offset: True
out_vars:
temp:
out_name: 'airtemp'
units: 'K'
prec:
out_name: 'pptrate'
units: 'mm/s'
shortwave:
out_name: 'SWradAtm'
spec_humid:
out_name: 'spechum'
air_pressure:
out_name: 'airpres'
units: 'kPa'
wind:
out_name: 'windspd'
chunks:
lat: 3
lon: 3
forcing_vars:
# Format is metsim_name: input_name
prec : 'Prec'
t_max : 'Tmax'
t_min : 'Tmin'
state_vars:
# Format is metsim_name: input_name
prec : 'Prec'
t_max : 'Tmax'
t_min : 'Tmin'
domain_vars:
# Format is metsim_name: input_name
lat : 'lat'
lon : 'lon'
mask : 'mask'
elev : 'elev'
t_pk : 't_pk'
dur : 'dur'
constant_vars:
wind : 2.0
This is a minimal configuration file for MetSim. For a complete description of the input format see the configuration page.
To run this example from the command line, once you have installed MetSim, use the following command:
ms path/to/example.yaml --verbose
This will run MetSim and disaggregate to half-hourly data, and write
out the results in a NetCDF file located in the directory specified
under out_dir
in the configuration file (here ./results
).
The addition of the --verbose
flag provides some
information back to you as MetSim runs. In the absence of this
flag MetSim will quietly run in the background until finished, or
some error has occurred.
Generating daily values¶
Daily values can be output by specifying a time_step
of 1440
in the
configuration file, such as the one shown in the previous section. This will
prevent MetSim’s disaggregation routines from being run, and the results written
out will be daily values.
Input Specifications¶
There are 3 required input data sources that must be specified in the
configuration file or dictionary. Note that it is possible for a single file to
be specified for all three sources, provided that it has all of the required
data. For examples of the data see the tests/data
directory within the
MetSim code.
Input forcings¶
Specified as forcing
in the configuration file. This can either be the path
to a NetCDF file, or the path to a directory containing ASCII or binary data (in
the VIC4 format). The input forcing data is used to provide the base forcing
variables. The required variable data is minimum daily temperature, maximum
daily temperature, and daily precipitation.
The variable names can be mapped via the configuration file in the forcing_vars
section. For more information about how to set up your configuration file see
the configuration page.
Domain file¶
Specified as domain
in the configuration file. The domain file provides
information about the domain MetSim is to be run over. It is required to be a
NetCDF file. The domain requires the following variables to be valid:
1. mask
: This provides information about which grid cells are valid to run
MetSim on. Values that specify grid cells which should be processed are
specified via a positive, finite number (one or greater). Cells which MetSim
should ignore can be given as 0 or NaN
.
It is important to ensure that all valid locations in mask
have data in
elev
and any other variables. Failure to ensure this will result in
errors during runtime.
2. elev
: This provides elevation data (in m) used for calculation of solar
geometry. It only needs to be given at sites which are marked to be processed
via the mask
variable.
The next two variables are only needed if prec_type
= triangle
or
mix
in the input file:
3. dur
: This provides the climatological monthly storm event duration (in
minutes) used for disaggregating daily precipitation according to the
“triangle” method. Requires one value for each month (12).
4. t_pk
: This provides the climatological monthly time to storm peak (in
minutes starting from midnight) used for disaggregating daily precipitation to
sub-daily time scales using the “triangle” method. Requires one value for
each month (12).
For more information about the “triangle” method see this description. If you use this feature, please cite Bohn et al. (2019) as listed in the references.
A domain file for the CONUS+Mexico domain, at 0.0625 degree resolution, and
containing dur
and t_pk
values, is available here.
State file¶
The state file provides information about the history of each of the grid cells to be processed. There are four required variables.
The first two are daily minimum and daily maximum temperatures for the 90 days
preceeding the start date specified in the configuration file. They should be
specified as t_min
and t_max
respectively. Similarly precipitation
should be given as prec
. These variables are used to generate seasonal
averages which are used in the calculation of shortwave and longwave radiation.
Output Specifications¶
Attention
The time
coordinate in MetSim’s output is local to the location of each cell unless the utc_offset
is set to
True
! This means that for a single time slice in the NetCDF file all locations along a parallel (same latitude)
will have the same solar geometry at that time.
The output variables that are available are dependent on the time step being used. There are two cases:
Daily Output¶
When time_step
is set to 1440 in the configuration file, daily values are
generated. The following variables are available for output at a daily time
step:
t_min
: Minimum temperature (also a required input value) (C)t_max
: Maximum temperature (also a required input value) (C)prec
: Precipitation (also a required input value) (mm/day)vapor_pressure
: Vapor pressure (Pa)shortwave
: Shortwave radiation (W/m^2)tskc
: Cloud cover fractionpet
: Potential evapotranpiration (mm/day)wind
: Wind speed (only if given as an input) (m/s)
Sub-daily Output¶
If disaggregation to shorter time steps is configured, the data is passed from the meteorology simulation module to the disaggregation module. MetSim implements several variable-specific disaggregation routines. Here we briefly mention the disaggregation procedures for completeness, but no substantial changes were made to the earlier algorithms.
Shortwave is disaggregated by multiplying the total daily shortwave by the fraction of radiation received in a given timestep (provided by the solar geometry module). This calculation is corrected for cloud cover by assuming constant transmissivity throughout the day (which is calculated in the meteorological simulation module). Temperature is disaggregated by estimating the time at which the daily maximum and daily minimum temperatures occur. These are chosen so that the daily minimum temperature occurs at sunrise and the daily maximum temperature occurs at a fixed time during the day (which is configurable by the user as a parameter in the configuration file if desired). Then a Hermite polynomial interpolation is used to obtain the full temperature timeseries at sub-daily time steps. Vapor pressure is disaggregated by linearly interpolating between the saturation vapor pressure values calculated based on the daily minimum temperature and that are assumed to occur at the time of the daily minimum temperature. An additional correction is made to ensure that the vapor pressure at any given time step does not exceed the saturation vapor pressure, which is calculated directly from the disaggregated temperature timeseries. Air pressure is disaggregated by using the disaggregated temperature as well as the elevation data provided by the domain file. Both specific and relative humidity are then disaggregated using the disaggregated temperature and air pressure time series. If provided, wind speed is disaggregated, but is assumed to be constant throughout the day.
As part of the model configuration, the user can select from a number of different algorithms to estimate longwave radiation. Sub-daily values are calculated with the selected method using the disaggregated values for vapor pressure and temperature.
Precipitation can be disaggregated in one of two ways. The first and simplest way is to evenly spread the daily precipitation across the sub-daily time steps. The second method requires two additional parameters to be specified in the domain file to represent the average precipitation duration and the time of peak precipitation for each cell. The method then disaggregates precipitation by constructing a triangular kernel with total area equal to the daily precipitation centered at the time which is specified as the time of peak precipitation.
The following variables (and name conventions) are allowed to be specified for subdaily output:
temp
: Temperature (C)prec
: Precipitation (mm/timestep)shortwave
: Shortwave radiation (W/m^2)vapor_pressure
: Vapor pressure (Pa)air_pressure
: Air pressure (kPa)rel_humid
: Relative humidityspec_humid
: Specific humiditylongwave
: Longwave radiation (W/m^2)tsck
: Cloud cover fractionwind
: Wind speed (only if given as an input) (m/s)
Configuration Specifications¶
This page documents the various options and parameters that can be set in the configuration file. An example configuration file can be found on the examples page.
MetSim Section¶
Required Variables
time_step :: int
: The timestep to disaggregate in minutes. If given as 1440
(number of minutes in a day), no disaggregation will occur. This value must
divide 1440 evenly.
start :: str
: The time to start simulation given in the format
yyyy/mm/dd
stop :: str
: The time to end simulation given in the format
yyyy/mm/dd
.
forcing :: path
: The path to the input forcing file(s). See the section
on __forcing_vars__ for more details.
domain :: path
: The path to the input domain file. See the section on
__domain_vars__ for more details.
state :: path
: The path to the input state file.
out_dir :: path
: The location to write output to. If this path doesn’t
exist, it will be created.
forcing_fmt :: str
: A string representing the type of input files specified in
the forcing
entry. Can be one of the following: ascii
, binary
,
netcdf
, or data
.
Optional Variables
method ::str
: The method to use for estimation of meteorological quantities.
This can be either mtclim
to estimate missing variables or passthrough
if
some of the meteorological variables have already been estimated (for example, by
DayMet, PRISM, or GridMET). Defaults to mtclim
.
out_prefix :: str
: The output file base name. Defaults to forcing
.
out_precision :: str
: Precision to use when writing output. Defaults to
f8
. Can be either f4
or f8
.
time_grouper :: str
: Whether to chunk up the timeseries into pieces for
processing. This option is useful to set for when you are limited on
memory. Each chunk of output is written as {out_prefix}_{date_range}
when
active. Any valid pandas.TimeGrouper
string may be used (e.g. use ‘10AS’
for 10 year chunks).
verbose :: bool
: Whether to print output to stdout
. Should be set using
the -v
flag for command line usage. This can be set for scripting purposes,
if desired. Set to 1
to print output; defaults to 0
.
sw_prec_thresh :: float
: Minimum precipitation threshold to take into
account when simulating incoming shortwave radiation. Defaults to 0
.
rain_scalar :: float
: Scale factor for calculation of cloudy sky
transmittance. Defaults to 0.75
, range should be between 0
and
1
.
utc_offset :: bool
: Whether to use UTC timecode offsets for shifting
timeseries. Without this option all times should be considered local to
the gridcell being processed. Large domain runs probably want to set this
option to True
.
lw_cloud :: str
: Type of cloud correction to longwave radiation to apply.
Can be either DEFAULT
or CLOUD_DEARDORFF
. Defaults to
CLOUD_DEARDORFF
. Capitalization does not matter.
lw_type :: str
: Type of longwave radiation parameterization to apply. Can be
one of the following: DEFAULT
, TVA
, ANDERSON
, BRUTSAERT
,
SATTERLUND
, IDSO
, or PRATA
. Defaults to PRATA
. Capitalization
does not matter.
tdew_tol :: float
: Convergence criteria for the iterative calculation of
dewpoint temperature in MtClim. Defaults to 1e-6
.
tmax_daylength_fraction :: float
: Weight for calculation of time of maximum
daily temperature. Must be between 0
and 1
. Defaults to 0.67
.
tday_coef :: float
: Scale factor for calculation of daily mean temperature.
Defaults to 0.45
, range should be between 0
and 1
.
lapse_rate :: float
: Used to calculate atmospheric pressure. Defaults to
0.0065 K/m
.
out_vars :: list
: List of variables to write to output. Should be a list
containing valid variables. The list of valid variables is dependent on which
simulation method is used, as well as whether disaggregation is used. Defaults
to ['temp', 'prec', 'shortwave', 'longwave', 'vapor_pressure', 'red_humid']
.
prec_type :: str
: Type of precipitation disaggregation method to use. Can be
one of the following: uniform
, triangle
, or mix
. Defaults to
uniform
. Capitalization does not matter. Under uniform
method,
precipitation is disaggregated by dividing uniformly over all sub-daily
timesteps. Under triangle
the “triangle” method is employed whereby daily
precipitation is distributed assuming an isosceles triangle shape with peak and
width determined from two domain variables, t_pk
and dur
. Under
mix
, the “uniform” method is used on days when t_min
< 0 C, and
“triangle” is used on all other days; this hybrid method retains the improved
accuracy of “triangle” in terms of warm season runoff but avoids the biases
in snow accumulation that the “triangle” method sometimes yields due to fixed
event timing within the diurnal cycle of temperature. A domain file for the
CONUS+Mexico domain, containing the dur
and t_pk
parameters is
available at: <https://zenodo.org/record/1402223#.XEI-mM2IZPY>. For more
information about the “triangle” method see PtriangleMethod.pdf.
For more information about input and output variables see the Input Specifications page.
# Comments begin with hashtags
# The first non-comment line must begin with the following:
Metsim:
time_step: int
start: YYYY-MM-DD
stop: YYYY-MM-DD
# Paths to input files
forcing: str
domain: str
state: str
# Output file specification
out_dir: str
out_prefix: str
# Algorithmic controls
utc_offset: bool
prec_type: str
lw_type: str
lw_cloud: str
chunks section¶
The chunks
section describes how parallel computation should be grouped
in space. For example, to parallelize over 10 by 10 chunks of latitude and
longitude (with netcdf dimensions named lat
and lon
, respectively) you would use:
chunks:
lat: 10
lon: 10
Alternatively, for an HRU based run chunked into 50 element jobs you would use:
chunks:
hru: 50
As a general rule of thumb, try to evenly chunk the domain in such a way that the number of jobs to run is some multiple of the number of processors you wish to run on.
forcing_vars and state_vars section¶
The forcing_vars
and state_vars
sections are where you can specify which
variables are in your input data, and the corresponding symbols which MetSim will
recognize. The in_vars
section for acts as a mapping between the variable
names in the input dataset to the variable names expected by MetSim. The format
is given as metsim_varname: netcdf_varname
. The minimum required variables
given have metsim_varname
s corresponding to t_min
, t_max
, and
prec
; these variable names correspond to minimum daily temperature (Celcius),
maximum daily temperature (Celcius), and precipitation (mm/day).
domain_vars section¶
The domain_vars
section is where information about the domain file is given.
Since the domain file is given as a NetCDF file this section has a similar
format to that of the NetCDF input file format described above. That is,
entries should be of the form metsim_varname = netcdfvarname
. The minimum
required variables have metsim_varname
s corresponding to lat
, lon
,
mask
, and elev
; these variable names correspond to latitude, longitude,
a mask of valid cells in the domain, and the elevation given in meters. If
prec_type
= triangle
or mix
, two additonal variables are required
including dur
and t_pk
for disaggregating daily precipitation according
to the “triangle” method.
out_vars section¶
The out_vars
section is where you can specify the output variables that you
want to include. There are two formats for this section. The first is the old format,
which we provide backwards compatibility for. You simply specify in the top level
[MetSim]
section a list of output variables with the names used by MetSim. They
will be written out with the same names used internally. Available options are
dependent on whether daily or subdaily output is being generated. Options for
daily output are:
- pet
- shortwave
- t_max
- t_min
- tskc
Options for subdaily output are:
- prec
- shortwave
- longwave
- temp
- vapor_pressure
- air_pressure
- tskc
- rel_humid
- spec_humid
- wind
The syntax for output specification is as follows:
out_vars:
metsim_varname:
out_name: str
units: str
unit conversions¶
The out_vars
section allows for specification of some simple unit conversions
for MetSim output. The allowed options are as follows (invalid options will revert
to the default after issuing a warning):
- prec -
mm timestep-1
(default) -mm s-1
-mm h-1
- pet (daily output only) -
mm timestep-1
(default) -mm s-1
-mm h-1
- t_max (daily output only) -
C
(default) -K
- t_min (daily output only) -
C
(default) -K
- temp -
C
(default) -K
- vapor_pressure -
Pa
(default) -hPa
-KPa
- air_pressure -
kPa
(default) -hPa
-Pa
- tskc (cloud fraction) -
fraction
(default) -%
- rel_humid -
%
(default) -fraction
constant_vars section¶
The constant_vars
section is optional and allows you to set some of the
forcing inputs to a constant value. The specification simply consists of entries
of the form metsim_varname: value
, where value
is a number that can be
converted to a double. There can only be one entry per line. If the
metsim_varname
corresponds to an entry that is already in the forcing_vars
section, then the constant value will take precedence. In the current
implementation there must be at least one non-constant entry in forcings_vars
(i.e. at least one entry that is not also in constant_vars
).
For example:
constant_vars:
wind: 2.0
will result in a constant wind field in the output file. In this case wind
does not need to be specified in the forcing_vars
section. If it was, it
will still be set to a constant value of 2 m/s.
Similarly:
constant_vars:
t_max = 30.0
t_min = 10.0
will result in output with a diurnal cycle in which the temperature varies at
all locations between 10C and 30C. However, all estimation and disaggregation
routines are still evaluated, with constant t_max
and t_min
as input.
INI Configuration Specifications¶
Warning
This page provides documentation only for backwards compatibility. New users should use the YAML configuration file format, as it is more flexible and is where new features will be added. The INI format will be depreciated in version 3 of MetSim.
This page documents the various options and parameters that can be set in the configuration file.
MetSim Section¶
Required Variables
time_step :: int
: The timestep to disaggregate in minutes. If given as 1440
(number of minutes in a day), no disaggregation will occur. This value must
divide 1440 evenly.
start :: str
: The time to start simulation given in the format
yyyy/mm/dd
stop :: str
: The time to end simulation given in the format
yyyy/mm/dd
.
forcing :: path
: The path to the input forcing file(s). See the section
on __forcing_vars__ for more details.
domain :: path
: The path to the input domain file. See the section on
__domain_vars__ for more details.
state :: path
: The path to the input state file.
out_dir :: path
: The location to write output to. If this path doesn’t
exist, it will be created.
forcing_fmt :: str
: A string representing the type of input files specified in
the forcing
entry. Can be one of the following: ascii
, binary
,
netcdf
, or data
.
Optional Variables
out_prefix :: str
: The output file base name. Defaults to forcing
.
out_precision :: str
: Precision to use when writing output. Defaults to
f8
. Can be either f4
or f8
.
time_grouper :: str
: Whether to chunk up the timeseries into pieces for
processing. This option is useful to set for when you are limited on
memory. Each chunk of output is written as {out_prefix}_{date_range}
when
active. Any valid pandas.TimeGrouper
string may be used (e.g. use ‘10AS’
for 10 year chunks).
verbose :: bool
: Whether to print output to stdout
. Should be set using
the -v
flag for command line usage. This can be set for scripting purposes,
if desired. Set to 1
to print output; defaults to 0
.
sw_prec_thresh :: float
: Minimum precipitation threshold to take into
account when simulating incoming shortwave radiation. Defaults to 0
.
rain_scalar :: float
: Scale factor for calculation of cloudy sky
transmittance. Defaults to 0.75
, range should be between 0
and
1
.
utc_offset :: bool
: Whether to use UTC timecode offsets for shifting
timeseries. Without this option all times should be considered local to
the gridcell being processed. Large domain runs probably want to set this
option to True
.
lw_cloud :: str
: Type of cloud correction to longwave radiation to apply.
Can be either DEFAULT
or CLOUD_DEARDORFF
. Defaults to
CLOUD_DEARDORFF
. Capitalization does not matter.
lw_type :: str
: Type of longwave radiation parameterization to apply. Can be
one of the following: DEFAULT
, TVA
, ANDERSON
, BRUTSAERT
,
SATTERLUND
, IDSO
, or PRATA
. Defaults to PRATA
. Capitalization
does not matter.
tdew_tol :: float
: Convergence criteria for the iterative calculation of
dewpoint temperature in MtClim. Defaults to 1e-6
.
tmax_daylength_fraction :: float
: Weight for calculation of time of maximum
daily temperature. Must be between 0
and 1
. Defaults to 0.67
.
tday_coef :: float
: Scale factor for calculation of daily mean temperature.
Defaults to 0.45
, range should be between 0
and 1
.
lapse_rate :: float
: Used to calculate atmospheric pressure. Defaults to
0.0065 K/m
.
out_vars :: list
: List of variables to write to output. Should be a list
containing valid variables. The list of valid variables is dependent on which
simulation method is used, as well as whether disaggregation is used. Defaults
to ['temp', 'prec', 'shortwave', 'longwave', 'vapor_pressure', 'red_humid']
.
prec_type :: str
: Type of precipitation disaggregation method to use. Can be
one of the following: uniform
, triangle
, or mix
. Defaults to
uniform
. Capitalization does not matter. Under uniform
method,
precipitation is disaggregated by dividing uniformly over all sub-daily
timesteps. Under triangle
the “triangle” method is employed whereby daily
precipitation is distributed assuming an isosceles triangle shape with peak and
width determined from two domain variables, t_pk
and dur
. Under
mix
, the “uniform” method is used on days when t_min
< 0 C, and
“triangle” is used on all other days; this hybrid method retains the improved
accuracy of “triangle” in terms of warm season runoff but avoids the biases
in snow accumulation that the “triangle” method sometimes yields due to fixed
event timing within the diurnal cycle of temperature. A domain file for the
CONUS+Mexico domain, containing the dur
and t_pk
parameters is
available at: <https://zenodo.org/record/1402223#.XEI-mM2IZPY>. For more
information about the “triangle” method see PtriangleMethod.pdf.
For more information about input and output variables see the Input Specifications page.
chunks section¶
The chunks
section describes how parallel computation should be grouped
in space. For example, to parallelize over 10 by 10 chunks of latitude and
longitude (with netcdf dimensions named lat
and lon
, respectively) you would use:
Alternatively, for an HRU based run chunked into 50 element jobs you would use:
As a general rule of thumb, try to evenly chunk the domain in such a way that the number of jobs to run is some multiple of the number of processors you wish to run on.
forcing_vars and state_vars section¶
The forcing_vars
and state_vars
sections are where you can specify which variables are in your
input data, and the corresponding symbols which MetSim will recognize. The
format of this section depends on the value given in the in_fmt
entry in
the MetSim
section of the configuration file. See below for conventions for
each input type.
netcdf and data¶
The in_vars
section for NetCDF and xarray input acts as a mapping between the variable
names in the input dataset to the variable names expected by MetSim. The format
is given as metsim_varname = netcdf_varname
. The minimum required variables
given have metsim_varname
s corresponding to t_min
, t_max
, and
prec
; these variable names correspond to minimum daily temperature (Celcius),
maximum daily temperature (Celcius), and precipitation (mm/day).
ascii¶
The in_vars
section for ASCII input acts similarly to the NetCDF input
format, except for one key point. Variables should be given as a tautology: the
format is given as metsim_varname = metsim_varname
. The order that the
variables are given corresponds to the column numbers that they appear in the
input files. The minimum required variables are t_min
, t_max
, and
prec
; these variable names correspond to minimum daily temperature (Celcius),
maximum daily temperature (Celcius), and precipitation (mm/day).
binary¶
This section has an input style for binary files that mimics the VIC version 4
input style. Each line is specified as varname = scale cdatatype
, where
varname
is the name that MetSim should use for the column, scale
is a
floating point scaling factor that should be applied after conversion from
binary to floating point; the conversion applied by the scale
is applied
after the value in the input is converted from binary to the cdatatype
specified for each variable. Valid cdatatype
s are signed
and
unsigned
. signed
values are interpreted as values which can be positive
or negative, whereas unsigned
values are interpreted as values that can only
be greater than or equal to zero.
domain_vars section¶
The domain_vars
section is where information about the domain file is given.
Since the domain file is given as a NetCDF file this section has a similar
format to that of the NetCDF input file format described above. That is,
entries should be of the form metsim_varname = netcdfvarname
. The minimum
required variables have metsim_varname
s corresponding to lat
, lon
,
mask
, and elev
; these variable names correspond to latitude, longitude,
a mask of valid cells in the domain, and the elevation given in meters. If
prec_type
= triangle
or mix
, two additonal variables are required
including dur
and t_pk
for disaggregating daily precipitation according
to the “triangle” method.
out_vars section¶
The out_vars
section is where you can specify the output variables that you
want to include. There are two formats for this section. The first is the old format,
which we provide backwards compatibility for. You simply specify in the top level
[MetSim]
section a list of output variables with the names used by MetSim. They
will be written out with the same names used internally. Available options are
dependent on whether daily or subdaily output is being generated. Options for
daily output are:
- pet
- shortwave
- t_max
- t_min
- tskc
Options for subdaily output are:
- prec
- shortwave
- longwave
- temp
- vapor_pressure
- air_pressure
- tskc
- rel_humid
- spec_humid
- wind
It may also be preferable to write the output with different variable names.
MetSim allows for this in a similar way as the forcing_vars
section.
The syntax for this type of output specification is as follows:
constant_vars section¶
The constant_vars
section is optional and allows you to set some of the
forcing inputs to a constant value. The specification simply consists of entries
of the form metsim_varname = value
, where value
is a number that can be
converted to a double. There can only be one entry per line. If the
metsim_varname
corresponds to an entry that is already in the forcing_vars
section, then the constant value will take precedence. In the current
implementation there must be at least one non-constant entry in forcings_vars
(i.e. at least one entry that is not also in constant_vars
).
For example:
wind = 2.0
will result in a constant wind field in the output file. In this case wind
does not need to be specified in the forcing_vars
section. If it was, it
will still be set to a constant value of 2 m/s.
Similarly:
t_max = 30
t_min = 10
will result in output with a diurnal cycle in which the temperature varies at
all locations between 10C and 30C. However, all estimation and disaggregation
routines are still evaluated, with constant t_max
and t_min
as input.
API reference¶
This page provides an auto-generated summary of metsim’s API. For more details and examples, refer to the relevant chapters in the main part of the documentation.
MetSim¶
Physics¶
MtClim¶
Disagg¶
What’s New¶
v2.4.1¶
Bug fixes¶
- Move from collections import Iterable to from collections.abc import Iterable for compatibility with newer python versions
v2.4.0¶
Enchancements¶
- Allow for passing already estimated met variables
(such as shortwave and/or longwave radiation)
through to the disaggregation routines. This
functionality can be accessed by setting the
method
topassthrough
in the configuration
v2.3.2¶
Bug fixes¶
- Fix a bug in ascii input reading due to pandas change
Enhancements¶
- Drastically speed up PITRI precipitation disaggregation
v2.3.1¶
Bug fixes¶
- Fixed an error in unit conversions for vapor pressure
- Fixed documentation on vapor pressure units
v2.3.0¶
Enhancements¶
- Allow for variable renaming in INI configuration files.
- Added capability for new YAML configuration file format
- Added capability simple unit conversions on output when using the YAML configuration file format
Bug fixes¶
- Fixed a bug where utc_offset causes radiation to be incorrectly scaled
v2.2.2¶
Bug fixes¶
- Fixed bug where utc_offset doesn’t get converted to the correct boolean when reading the configuration.
v2.2.l¶
Bug fixes¶
- Fixed bug where timestamps got duplicated for sub-hourly disaggregation time periods
- Fixed bug where polar latitudes caused error in setting the rise and set times for temperature disaggregation
v2.2.0¶
Enhancements¶
- Can now specify
period_ending
in the configuration to move timestamps to end of period instead of beginning of period - Addition of tutorial in README.md and main documentation
- Addition of paper to be submitted to JOSS
v.2.1.2 (under development)¶
Bug fixes¶
- Can now handle dimensions without coordinate variables, which previously caused a bug in the chunk selection of the worker processes. The fix is to simply add a sequential coordinate any time this occurs.
v.2.0.1¶
Enhancements¶
- Allow for specification of constant fields, through addition
of an optional
constant_vars
section.
v.2.0.0¶
Enhancements¶
- Implemented UTC offsets, which puts all gridcell times in reference to UTC.
- Moved parallelism to dask, which allows for greater scalability and significantly less memory overhead.
Bug fixes¶
- Disallow timesteps > 6 hours, which raised errors.
- Raise error when t_min > t_max at beginning of runtime.
v.1.1.1¶
Enhancements¶
- Added option to disaggregate precipitation via a triangular hyetograph. (GH42). By Kristen Whitney and Theodore Bohn.
Bug fixes¶
- Fixed a bug where if iter_dims is not [lat, lon] the selected lat value that goes into solar_geom ends up as a list. The fix is also added for elevation and longitude, for redundancy. Fixes GH132. By Andrew Bennett.
v1.1.0¶
Enhancements¶
- Added option to use forcing start/stop dates to define run length (GH93). By Joe Hamman.
- Added option a flexible time grouper when chunking MetSim runs (GH93). By Joe Hamman.
- Improved configuration validation by checking for correctness of output variables (GH96) By Andrew Bennett.
- Added option to skip reading
swe
variable from state file if it is not going to be used by MtClim. (GH103). By Joe Hamman. - Added support for supplying a glob-like file path or multiple input forcing files (netCDF) (GH126). By Joe Hamman.
- Refactored
mtclim
anddisaggregate
functions to reduce interdependency and increase modularity. By Andrew Bennett. - Removed
swe
calculations. By Andrew Bennett.
Bug fixes¶
- Fixed bug where output files were not written with the appropriate calendar encoding attribute (GH97). By Joe Hamman.
- Fixed a bug where invalid timesteps were used in subdaily disaggregation. Added a clear error message explaining that subdaily timesteps must be evenly divisible into 24 hours and less than 6 hours in length. (GH110). By Joe Hamman.
- Fixed a bug during disaggregation when
t_min > t_max
. This now raises an exception. By Andrew Bennett.