Project Configuration¶
There are three layers of configuration in order to use the simulation modelling integration framework to conduct system-of-system modelling.
A project is the highest level container which holds all the elements required to run models, configure simulation models and define system-of-system models.
The basic folder structure looks like this:
project.yml
/config
/dimensions
annual.yml
country.yml
oxfordshire.yml
/model_runs
energy_central.yml
energy_water_cp_cr.yml
/narratives
technology.yml
/scenarios
climate.yml
population.yml
water_sector_energy_demand.yml
/sector_models
energy_demand.yml
water_supply.yml
/sos_models
energy_water.yml
energy.yml
/data
/coefficients
...
/dimensions
hourly.csv
annual.csv
lad.shp
/initial_conditions
energy_demand_existing.csv
energy_supply_existing.csv
/interventions
energy_demand.csv
energy_supply.csv
/narratives
energy_demand_high_tech.csv
central_planning.csv
/scenarios
population_high.csv
population_low.csv
/strategies
pipeline_2020.csv
/models
energy_demand.py
water_supply.py
/results
/energy_central
/energy_demand
...
/water_supply
...
The Project File¶
This file holds a small amount of project-level configuration.
The project name is a unique identifier for this project.
Unit definitions references a file containing custom units, not included in the Pint library default unit register (e.g. non-SI units).
Model Run¶
A model run brings together a system-of-systems model definition with timesteps over which planning takes place, and a choice of scenarios and narratives to population the placeholder scenario sets in the system-of-systems model.
name: energy_central
description: Energy demand under central scenario, only 2010
stamp: "2017-09-18T12:53:23+00:00"
timesteps:
- 2010
- 2015
- 2020
sos_model: energy
scenarios:
population: population_med
water_sector_energy_demand: central
narratives: {}
strategies:
- type: pre-specified-planning
description: build_nuclear
filename: build_nuke.csv
model_name: energy_supply
- type: rule-based
description: reduce emissions
path: planning/energyagent.py
classname: EnergyAgent
Hints
[A] Create a new model run
[B] Click on the row to edit an existing model run
[C] Click on the bin icon to delete a configuration
# | Attribute | Notes |
---|---|---|
1 | Name | A unique name that identifies the Model Run configuration. Note: this field is non-editable. |
2 | Description | A description that shortly describes the Model Run for future reference. |
3 | Created | A timestamp that identifies at which time this Model Run was created. Note: this field is non-editable. |
4 | System-of-System model | The System-of-Systems Model that this Model Run configuration is using. |
5 | Scenarios | The selected Scenario for this Model Run within each of the available Scenario Sets. Note: Only the Scenario Sets that were configured in the selected System-of-System Model will be available here. |
6 | Narrative | The selected Narratives for this Model Run within each of the available Narrative Sets. Note: Only the Narrative Sets that were configured in the selected System-of-System Model will be available here. |
7 | Resolution | The number of years between each of the Timesteps. |
8 | Base year | The Timestep where this Model Run must start the simulation. |
9 | End year | The last Timestep that this Model Run must simulate. |
Hints
[A] “Save” will save changes to this configuration. Click “Cancel” to leave the configuration without saving.
Timesteps¶
A list of timesteps define the years in which planning takes place, and the simulation models are executed.
timesteps:
- 2010
- 2015
- 2020
Scenarios¶
For each scenario available in the contained system-of-systems model, one variant should be chosen.
scenarios:
population: population_med
water_sector_energy_demand: central
Narratives¶
Narratives override the default values of parameters in simulation models.
For each narrative available in the project, any number of narrative variants can be chosen (or none at all).
narratives: {}
System-of-Systems Models¶
A system-of-systems model collects together scenario sets and simulation models. Users define dependencies between scenario and simulation models.
name: energy_water
description: 'The future supply and demand of energy and water for the UK'
scenarios: # Select 0 or more of the scenario sets
- population
- climate
- reservoir_level # needed for initial inputs to water_supply
- water_sector_energy_demand
narratives:
- name: technology
description: Describes the evolution of technology
sos_model: energy_water
provides:
energy_demand:
- smart_meter_savings
water_supply:
- clever_water_meter_savings
- per_capita_water_demand
variants:
- name: high_tech_dsm
description: High penetration of SMART technology on the demand side
data:
smart_meter_savings: high_tech_dsm.csv
clever_water_meter_savings: high_tech_dsm.csv
per_capita_water_demand: high_tech_dsm.csv
sector_models: # Select 1 or more of the sector models
- water_supply
- energy_demand
scenario_dependencies:
- source: climate
source_output: precipitation
sink: water_supply
sink_input: precipitation
- source: population
source_output: population
sink: water_supply
sink_input: population
- source: population
source_output: population
sink: energy_demand
sink_input: population
- source: reservoir_level
source_output: reservoir_level
sink: water_supply
sink_input: reservoir_level
- source: water_sector_energy_demand
source_output: energy_demand
sink: energy_demand
sink_input: energy_demand
model_dependencies:
- source: energy_demand
source_output: water_demand
sink: water_supply
sink_input: water_demand
- source: water_supply
source_output: reservoir_level
sink: water_supply
sink_input: reservoir_level
timestep: PREVIOUS # dependency on own outputs from previous timestep
# | Attribute | Notes |
---|---|---|
1 | Name | A unique name that identifies the System-of-Systems model configuration. Note: this field is non-editable. See also A System-of-Systems Model File |
2 | Description | A description that shortly describes the System-of-Systems model for future reference. See also A System-of-Systems Model File |
3 | Sector Models | The selection of Simulation Models that are used in this System-of-Systems Model. See also A System-of-Systems Model File |
4 | Scenario Sets | The selection of Scenario Sets that are used in this System-of-Systems Model. See also A System-of-Systems Model File |
5 | Narrative Sets | The selection of Narrative Sets that are used in this System-of-Systems Model. See also A System-of-Systems Model File |
6 | Dependencies | The list of Dependencies that are defined between sources and links. See also Dependencies |
Hints
[A] “Add Dependency” opens a form to add a new dependency
[B] “Save” will save changes to this configuration. Click “Cancel” to leave the configuration without saving.
Scenarios and Narratives¶
Scenarios are the categories in which scenario data are organised. Choosing a scenario set at this point allows different scenario data to be chosen in model runs which share the same system-of-systems model configuration defintion.
scenarios: # Select 0 or more of the scenario sets
- population
- climate
- reservoir_level # needed for initial inputs to water_supply
- water_sector_energy_demand
Similarly, narratives can be made available if desired.
narratives:
- name: technology
description: Describes the evolution of technology
sos_model: energy_water
provides:
energy_demand:
- smart_meter_savings
water_supply:
- clever_water_meter_savings
- per_capita_water_demand
variants:
- name: high_tech_dsm
description: High penetration of SMART technology on the demand side
data:
smart_meter_savings: high_tech_dsm.csv
clever_water_meter_savings: high_tech_dsm.csv
per_capita_water_demand: high_tech_dsm.csv
Simulation Models¶
This section contains a list of pre-configured simulation models which exist in the current project.
sector_models: # Select 1 or more of the sector models
- water_supply
- energy_demand
Dependencies¶
In this section, dependencies are defined between sources and sinks. For convenience, they are split into scenario dependencies (where a model will receive data from a scenario) and model dependencies (where a model will receive data from another model).
scenario_dependencies:
- source: climate
source_output: precipitation
sink: water_supply
sink_input: precipitation
- source: population
source_output: population
sink: water_supply
sink_input: population
- source: population
source_output: population
sink: energy_demand
sink_input: population
- source: reservoir_level
source_output: reservoir_level
sink: water_supply
sink_input: reservoir_level
- source: water_sector_energy_demand
source_output: energy_demand
sink: energy_demand
sink_input: energy_demand
model_dependencies:
- source: energy_demand
source_output: water_demand
sink: water_supply
sink_input: water_demand
- source: water_supply
source_output: reservoir_level
sink: water_supply
sink_input: reservoir_level
timestep: PREVIOUS # dependency on own outputs from previous timestep
Attribute | Type | Notes |
---|---|---|
source | string | The source of the data |
source_output | string | The output in the source |
sink | string | The model which depends on the source |
sink_input | string | The input which should receive the data |
Simulation Models¶
A model file contains all the configuration data necessary for smif to run the model, and link the model to data sources and sinks. This file also contains a list of parameters, the ‘knobs and dials’ the user wishes to expose to smif which can be adjusted in narratives. Intervention files and initial condition files contain the collections of data that are needed to expose the model to smif’s decision making functionality.
# | Attribute | Notes |
---|---|---|
1 | Name | A unique name that identifies the simulation model that is wrapped. Note: this field is non-editable. See also A Simulation Model File |
2 | Description | A description that shortly describes the simulation model for future reference. See also A Simulation Model File |
3 | Class Name | Name of the Class that is used in the smif wrapper. See also Wrapping a Sector Model: Overview |
4 | Path | The location of the python wrapper file. See also Wrapping a Sector Model: Overview |
5 | Inputs | The simulation model inputs with their name, units and temporal-spatial resolution. See also Inputs |
6 | Outputs | The simulation model outputs with their name, units and temporal-spatial resolution. See also Outputs |
7 | Parameters | The simulation model parameters. See also Parameters |
Hints
[A] “Add Input” to open a form to add a new input
[B] “Add Output” to open a form to add a new output
[C] “Add Parameter” to open a form to add a new parameter
[D] “Save” to save changes to this configuration. Click on “Cancel” to leave the configuration without saving.
Inputs¶
Define the collection of inputs required from external sources to run the model. Inputs are defined with a name, spatial resolution, temporal-resolution and units.
inputs:
- name: precipitation
dims:
- country
dtype: float
unit: ml
- name: population
dims:
- country
dtype: int
unit: people
Attribute | Type | Notes |
---|---|---|
name | string | A name for the input variable |
dims | string | A list of dimensions |
dtype | string | The data type |
units | string | The units required for the variable |
absolute_range | tuple | (optional) Raises an error if bounds exceeded |
suggested_range | tuple | (optional) Provides a hint to a user as to sensible ranges |
default_value | float | (optional) The default value for the parameter |
Outputs¶
Define the collection of output model values used for the purpose of metrics, for accounting purposes, such as operational cost and emissions, or as the source of a dependency in another model.
outputs:
- name: cost
dims:
- country
dtype: float
unit: million GBP
- name: energy_demand
dims:
- country
dtype: float
unit: kWh
Outputs are defined with exactly the same attributes as inputs.
Parameters¶
Parameters should all be configured with default values - these may be overridden by narratives when a model is run.
parameters:
- name: clever_water_meter_savings
description: The savings from smart water meters
abs_range:
- 0
- 100
exp_range:
- 3
- 10
default: defaults.csv
dtype: float
unit: '%'
Parameters are defined with exactly the same attributes as inputs and outputs.
Scenarios¶
The config/scenarios
folder contains scenario definitions. Data files for each of the
scenario variants are stored in data/scenarios
.
Here’s an example of a population scenario which can provide consistent data for population and population density, in three variants (high/medium/low):
name: population
description: Growth in UK population
provides:
- name: population
description: ''
dims:
- country
dtype: int
unit: people
- name: population_density
description: ''
dims:
- country
dtype: float
unit: people/km²
variants:
- name: population_low
description: Central Population (Low)
data:
population: population_low.csv
population_density: population_density_low.csv
- name: population_med
description: Central Population (Medium)
data:
population: population_med.csv
population_density: population_density_med.csv
- name: population_high
description: Central Population (High)
data:
population: population_high.csv
population_density: population_density_high.csv
The list of variables that a scenario provides
are defined exactly as model inputs, outputs
and parameters.
The filenames in the data
section within the scenario definition point to CSV
(comma-separated-values) files stored in the data/scenarios
folder. For example:
timestep,country,population
2010,England,52000000
2010,Scotland,5100000
2010,Wales,2900000
2015,England,53000000
2015,Scotland,5300000
2015,Wales,3000000
2020,England,54000000
2020,Scotland,5500000
2020,Wales,3200000
Narratives¶
The config/narratives
folder contains narrative definitions. Data files for each of the
variants are stored in data/narratives
.
A narrative file contains references to 0 or more parameters defined in the simulation models. Parameters might include system-wide constants such as emission coefficients or exchange rates, and parameters used by a single model, such as technology energy efficiencies.
Value specified in the narrative file override the default values specified in the simulation model configuration. If more than one narrative file is selected in the sos model configuration, then values in later files override values in earlier files.
Dimensions¶
Dimensions are used in the metadata that describes model inputs, outputs and parameters.
The config/dimensions
folder contains dimenion definitions. The data/dimensions
folder
contains the list of coordinates which index the dimension, along with optional metadata that
can allow conversion between different dimensions of the same type.
Dimension config includes a name and brief description, along with a path to the file in
data/dimensions
which defines the coordinates elements:
name: annual
description: Single annual interval of 8760 hours
elements: annual_intervals.csv
Spatial and temporal dimensions are commonly used to define inputs or outputs which carry information about a variable which varies over time (intervals) and/or space (locations or regions).
Temporal dimensions¶
A temporal dimension definition specifies the mapping of model timesteps to durations within a year (assuming that each planning timestep models 365 days: no extra day in leap years, no leap seconds).
A dimension definition is a list of one or more intervals, as expected by a model. Each interval is defined by one or more periods. The start and end of each period is measured from the beginning of a year.
Use ISO 8601 [1] duration format to specify periods:
P[n]Y[n]M[n]DT[n]H[n]M[n]S
For example, to define one year-long interval, starting in the first hour of the year and ending in the last hour of the year:
id,interval
1,"[[""PT0H"",""PT8760H""]]"
Or to define a set of seasons, where winter covers January, February and December:
name,interval
spring,"[[""P2M"", ""P5M""]]"
summer,"[[""P5M"", ""P8M""]]"
autumn,"[[""P8M"", ""P11M""]]"
winter,"[[""P0M"", ""P2M""], [""P11M"", ""P12M""]]"
Attribute | Type | Notes |
---|---|---|
id | string | The unique identifier used by the simulation model |
interval | string | list of [start, end] pairs (in CSV format, JSON-serialised and escaped) |
Spatial dimensions¶
Spatial dimensions define the set of regions used to index a model variable. The spatial resolution of the model may be implicit, and even a national model needs to have a national region defined.
The file format must be possible to parse with GDAL, and must contain an attribute “name” to use as an identifier for the region.
Interventions¶
Interventions are the atomic units which comprise the infrastructure systems in the simulation models. Interventions can represent physical assets such as pipes, and lines (edges in a network) or power stations and reservoirs (nodes in a network). Interventions can also represent intangibles which affects the operation of a system, such as a policy.
An exhaustive list of the interventions (often infrastructure assets) should be defined. These are represented internally in the system-of-systems model, collected into a gazateer and allow the framework to reason on infrastructure assets across all sectors.
Interventions are instances of -smif.intervention.Intervention
and are held in
-smif.intervention.InterventionRegister
. Interventions include investments in assets,
supply side efficiency improvements, but not demand side management (these are incorporated in
the strategies).
Define all possible interventions in an *.csv
file in the data/interventions
For example:
name,location,capacity_value,capacity_units,operational_lifetime_value,operational_lifetime_units,technical_lifetime_value,technical_lifetime_units,capital_cost_value,capital_cost_units
small_pumping_station,Oxford,50,ML/day,150,years,50,years,5,million £
large_pumping_station,Oxford,500,ML/day,100,years,50,years,15,million £
After name, columns are optional and will be used for any attributes. Another example:
name,capital_cost_value,capital_cost_unit,technical_lifetime_value,operational_lifetime_value,line_num,from_location,to_location,capacity_value,capacity_unit,length_value,length_unit
ac_line1,5,£/km,5,80,1,1,2,525,MW,50,km
ac_line2,4,£/km,5,80,2,1,3,132,MW,40,km
ac_line3,4.2,£/km,5,80,3,2,4,760,MW,42,km
ac_line4,6,£/km,5,80,4,4,7,1090,MW,60,km
Note that the _value
and _unit
suffixes of the column names are used to unpack the data
internally.
Some attributes may be required:
- technical_lifetime (years are assumed as unit and can be omitted)
Initial Conditions¶
Initial conditions define the interventions to be applied before any decision process starts. Depending on the model, it may be possible to compose the entire system of interest from a list of initial conditions.
name,build_year
large_pumping_station,1975