Getting Started¶
Once you have installed smif, the quickest way to get started is to use the
included test project. You can find the test project in the development version
of the package, in the tests/fixtures/single_run folder.
On the command line, type the following command to list the available model runs:
$ smif list -d smif/tests/fixtures/single_run
20170918_energy_water_short.yml
20170918_energy_water.yml
To run a model run, type the following command:
$ smif run 20170918_energy_water.yml -d smif/tests/fixtures/single_run
Model run complete
Note that the -d directory flag should point to the single_run folder, so
check you are pointing to the correct directory if this doesn’t work first time.
Project Configuration¶
There are three layers of configuration in order to use the simulation modelling integration framework to conduct system-of-system modelling.
A project is the highest level container which holds all the elements required to run models, configure simulation models and define system-of-system models.
The basic folder structure looks like this:
project.yml
/config
/sector_models
energy_demand.yml
energy_supply.yml
/sos_models
energy.yml
/model_runs
20170918_energy.yml
/data
/initial_conditions
energy_demand_existing.yml
energy_supply_existing.yml
/intervals
hourly.csv
annual.csv
/interventions
energy_demand.yml
/narratives
energy_demand_high_tech.yml
central_planning.yml
/regions
lad.shp
/scenarios
population_high.csv
units.yml
The folder structure is divided into a config subfolder and a data
subfolder.
The Configuration Folder¶
This folder holds configuration and metadata on simulation models, system-of-system models and model runs.
The Project File¶
This file holds all the project configuration.
name: "Test Project"
scenario_sets:
- name: raininess
description: 'UK precipitation'
- name: population
description: 'Growth in UK population'
narrative_sets:
- name: technology
description: 'Describes the evolution of technology'
region_definitions:
# shared understanding of regions within the modelled area (e.g. GB)
- name: national
description: ''
filename: uk_nations_shp/regions.shp
- name: oxfordshire
description: ''
filename: oxfordshire/regions.geojson
interval_definitions:
# shared understanding of time intervals within a year
- name: annual
description: ''
filename: annual_intervals.csv
units:
scenarios:
# given input data, required values for every year of a model run
- name: 'Central Population'
description: 'Central Population for the UK'
scenario_set: population
parameters:
- name: population
filename: population.csv
spatial_resolution: national
temporal_resolution: annual
units: million people
- name: 'Central Rainfall'
description: 'Central Rainfall Scenario for the UK'
scenario_set: rainfall
parameters:
- name: raininess
filename: raininess.csv
spatial_resolution: national
temporal_resolution: annual
units: ml
narratives:
- name: 'High Tech Demand Side Management'
description: 'High penetration of SMART technology on the demand side'
filename: high_tech_dsm.yml
narrative_set: technology
We’ll step through this configuration file section by section.
The first line gives the project name, a unique identifier for this project.
name: "Test Project"
The next section lists the scenario sets. These give the categories into which scenarios are collected.
scenario_sets:
- name: raininess
description: 'UK precipitation'
- name: population
description: 'Growth in UK population'
Narrative sets collect together the categories into which narrative files are collected.
narrative_sets:
- name: technology
description: 'Describes the evolution of technology'
Region definitions list the collection of region files and the mapping to a unique name which can be used in scenarios and sector models. Region definitions define the spatial resolution of data.
region_definitions:
- name: national
description: ''
filename: uk_nations_shp/regions.shp
- name: oxfordshire
description: ''
filename: oxfordshire/regions.geojson
Interval definitions list the collection of interval files and the mapping to a unique name which can be used in scenarios and sector models. Interval definitions define the temporal resolution of data.
interval_definitions:
- name: annual
description: ''
filename: annual_intervals.csv
Unit definitions references a file containing custom units, not included in the Pint library default unit register (e.g. non-SI units).
units:
The scenarios section lists the scenarios and corresponding collections of
data associated with scenarios.
scenarios:
- name: 'Central Population'
description: 'Central Population for the UK'
scenario_set: population
parameters:
- name: population
filename: population.csv
spatial_resolution: national
temporal_resolution: annual
units: million people
- name: 'Central Rainfall'
description: 'Central Rainfall Scenario for the UK'
scenario_set: rainfall
parameters:
- name: raininess
filename: raininess.csv
spatial_resolution: national
temporal_resolution: annual
units: ml
The narratives section lists the narratives and mapping to one or more
narrative files
narratives:
- name: 'High Tech Demand Side Management'
description: 'High penetration of SMART technology on the demand side'
filename: high_tech_dsm.yml
narrative_set: technology
A Simulation Model File¶
A simulation model file contains all the configuration data necessary for smif to run the model, and link the model to data sources and sinks. This file also contains a list of parameters, the ‘knobs and dials’ the user wishes to expose to smif which can be adjusted in narratives. Intervention files and initial condition files contain the collections of data that are needed to expose the model to smif’s decision making functionality.
name: water_supply # model name for internal/logging reference
description: 'Simulates the optimal operation of the UK water
supply system'
path: models/water_supply.py # path to python file
classname: WaterSupplySectorModel # implements smif.SectorModel
inputs:
- name: raininess
spatial_resolution: national
temporal_resolution: annual
units: ml
- name: population
spatial_resolution: national
temporal_resolution: annual
units: million people
- name: water_demand
spatial_resolution: national
temporal_resolution: annual
units: Ml
outputs:
- name: cost
spatial_resolution: national
temporal_resolution: annual
units: million GBP
- name: energy_demand
spatial_resolution: national
temporal_resolution: annual
units: kWh
- name: water
spatial_resolution: national
temporal_resolution: annual
units: Ml
interventions:
- water_supply.yml
initial_conditions:
- water_supply_oxford.yml
- reservoirs.yml
parameters:
- name: clever_water_meter_savings
description: The savings from smart water meters
absolute_range: (0, 100)
suggested_range: (3, 10)
default_value: 3
units: '%'
Inputs¶
Define the collection of inputs required from external sources to run the model. Inputs are defined with a name, spatial resolution, temporal-resolution and units.
inputs:
- name: raininess
spatial_resolution: national
temporal_resolution: annual
units: ml
- name: population
spatial_resolution: national
temporal_resolution: annual
units: million people
| Attribute | Type | Notes |
|---|---|---|
| name | string | A unique name within the input defintions |
| spatial_resolution | string | References an entry in the region definitions |
| temporal_resolution | string | References an entry in the interval definitions |
| units | string | References an entry in the unit definitions |
Outputs¶
Define the collection of output model parameters used for the purpose of metrics, for accounting purposes, such as operational cost and emissions, or as the source of a dependency in another model.
outputs:
- name: cost
spatial_resolution: national
temporal_resolution: annual
units: million GBP
- name: energy_demand
spatial_resolution: national
temporal_resolution: annual
units: kWh
| Attribute | Type | Notes |
|---|---|---|
| name | string | A unique name within the output definitions |
| spatial_resolution | string | References an entry in the region definitions |
| temporal_resolution | string | References an entry in the interval definitions |
| units | string | References an entry in the unit definitions |
Parameters¶
parameters:
- name: clever_water_meter_savings
description: The savings from smart water meters
absolute_range: (0, 100)
suggested_range: (3, 10)
default_value: 3
units: '%'
| Attribute | Type | Notes |
|---|---|---|
| name | string | A unique name within the simulation model |
| description | string | Include sources of assumptions around default value |
| absolute_range | tuple | Raises an error if bounds exceeded |
| suggested_range | tuple | Provides a hint to a user as to sensible ranges |
| default_value | float | The default value for the parameter |
| units | string |
A System-of-System Model File¶
A system-of-systems model collects together scenario sets and simulation models. Users define dependencies between scenario and simulation models.
name: energy_water
description: 'The future supply and demand of energy and water for the UK'
scenario_sets: # Select 0 or more of the scenario sets
- population
- rainfall
sector_models: # Select 1 or more of the sector models
- water_supply
- energy_demand
dependencies:
- source_model: rainfall
source_model_output: raininess
sink_model: water_supply
sink_model_input: raininess
- source_model: population
source_model_output: population
sink_model: water_supply
sink_model_input: population
- source_model: water_supply
source_model_output: energy_demand
sink_model: energy_demand
sink_model_input: energy_demand
- source_model: population
source_model_output: population
sink_model: energy_demand
sink_model_input: population
- source_model: energy_demand
source_model_output: water_demand
sink_model: water_supply
sink_model_input: water_demand
max_iterations: 100
convergence_absolute_tolerance: 1e-05
convergence_relative_tolerance: 1e-05
Scenario Sets¶
Scenario sets are the categories in which scenario data are organised. Choosing a scenario set at this points allows different scenario data to be chosen in model runs which share the same system-of-systems model configuration defintion.
scenario_sets: # Select 0 or more of the scenario sets
- population
- rainfall
| Attribute | Type | Notes |
|---|---|---|
| names | list | A list of scenario set names |
Simulation Models¶
This section contains a list of pre-configured simulation models which exist in the current project.
sector_models: # Select 1 or more of the sector models
- water_supply
- energy_demand
| Attribute | Type | Notes |
|---|---|---|
| names | list | A list of simulation model names |
Dependencies¶
In this section, dependencies are defined between sources and sinks.
dependencies:
- source_model: rainfall
source_model_output: raininess
sink_model: water_supply
sink_model_input: raininess
- source_model: population
source_model_output: population
sink_model: water_supply
sink_model_input: population
| Attribute | Type | Notes |
|---|---|---|
| source_model | string | The source model of the data |
| source_model_output | string | The output in the source model |
| sink_model | string | The model which depends on the source |
| sink_model_input | string | The input which should receive the data |
A Model Run File¶
A model run brings together a system-of-systems model definition with timesteps over which planning takes place, and a choice of scenarios and narratives to population the placeholder scenario sets in the system-of-systems model.
name: '20170918_energy_water.yml'
description: 'Combined energy and water under central scenario'
stamp: 2017-09-18T12:53:23+00:00
timesteps:
- 2010
- 2015
- 2020
sos_model: energy_water
decision_module: ''
scenarios:
- ['population', 'Central Population']
- ['rainfall', 'Central Rainfall']
narratives:
- technology:
- 'High Tech Demand Side Management'
Timesteps¶
A list of timesteps define the years in which planning takes place, and the simulation models are executed.
timesteps:
- 2010
- 2015
- 2020
| Attribute | Type | Notes |
|---|---|---|
| timesteps | list | A list of integer years |
Scenarios¶
For each scenario set available in the contained system-of-systems model, one scenario should be chosen.
scenarios:
- ['population', 'Central Population']
- ['rainfall', 'Central Rainfall']
| Attribute | Type | Notes |
|---|---|---|
| scenarios | list | A list of tuples of scenario sets and scenarios |
Narratives¶
For each narrative set available in the project, zero or more available narratives should be chosen.
narratives:
- technology:
- 'High Tech Demand Side Management'
Note that narrative files override the values of parameters in specific simulation models. Selecting a narrative file which overrides parameters in an absent simulation model will have no effect.
| Attribute | Type | Notes |
|---|---|---|
| scenarios | list | A list of mappings between narrative sets and list of narrative files |
Data Folder¶
This folder holds data like information to define the spatial and temporal resolution of data, as well as exogenous environmental data held in scenarios.
Initial Conditions¶
- name: Kielder Water
location: England
capacity:
value: 500
units: ML
operational_lifetime:
value: 300
units: years
economic_lifetime:
value: 150
units: years
capital_cost:
value: 15
units: million £
build_date: 1975
current_level:
value: 3
units: Ml
is_state: true
Interval definitions¶
The attribution of hours in a year to the temporal resolution used in the sectoral model.
Within-year time intervals are specified in yaml files, and as for regions,
specified in the *.yml file in the project/data/intervals folder.
This links a unique name with the definitions of the intervals in a yaml file. The data in the file specify the mapping of model timesteps to durations within a year (assume modelling 365 days: no extra day in leap years, no leap seconds)
Use ISO 8601 [1] duration format to specify periods:
P[n]Y[n]M[n]DT[n]H[n]M[n]S
For example:
id,start,end
1,PT0H,PT8760H
In this example, the interval with id 1 begins in the first hour of the year
and ends in the last hour of the year. This represents one, year-long interval.
| Attribute | Type | Notes |
|---|---|---|
| id | string | The unique identifier used by the simulation model |
| start_hour | string | Period since beginning of year |
| end_hour | string | Period since beginning of year |
Region definitions¶
Define the set of unique regions which are used within the model as polygons. The spatial resolution of the model may be implicit, and even a national model needs to have a national region defined. Inputs and outputs are assigned a model-specific geography from this list allowing automatic conversion from and to these geographies.
Model region files are stored in project/data/region_defintions.
The file format must be possible to parse with GDAL, and must contain an attribute “name” to use as an identifier for the region.
The sets of geographic regions are specified in the project configuration file
using a region_definitions attributes as shown below:
region_definitions:
- name: national
description: ''
filename: uk_nations_shp/regions.shp
- name: oxfordshire
description: ''
filename: oxfordshire/regions.geojson
This links a name, used elsewhere in the configuration with inputs, outputs and scenarios with a file containing the geographic data.
Interventions¶
An Intervention is an investment which has a name (or name), other attributes (such as capital cost and economic lifetime), and location, but no build date.
An intervention is a possible investment, normally an infrastructure asset, the timing of which can be decided by the logic-layer.
An exhaustive list of the interventions (often infrastructure assets)
should be defined.
These are represented internally in the system-of-systems model,
collected into a gazateer and allow the framework to reason on
infrastructure assets across all sectors.
Interventions are instances of Intervention and are
held in InterventionRegister.
Interventions include investments in assets,
supply side efficiency improvements, but not demand side management (these
are incorporated in the strategies).
Define all possible interventions in an *.yml file
in the project/data/interventions For example:
- name: small_pumping_station
location: Oxford
capacity:
value: 50
units: ML/day
operational_lifetime:
value: 150
units: years
economic_lifetime:
value: 50
units: years
capital_cost:
value: 5
units: million £
Narratives¶
energy_demand:
smart_meter_savings: 8
water_supply:
clever_water_meter_savings: 8
Scenarios¶
The scenarios: section of the project configuration file allows
you to define static sources for simulation model dependencies.
In the case of the example project file shown earlier,
the scenarios section lists the scenarios and corresponding collections of
data associated with the scenario sets:
scenarios:
- name: 'Central Population'
description: 'Central Population for the UK'
scenario_set: population
parameters:
- name: population
filename: population.csv
spatial_resolution: national
temporal_resolution: annual
units: million people
- name: 'Central Rainfall'
description: 'Central Rainfall Scenario for the UK'
scenario_set: rainfall
parameters:
- name: raininess
filename: raininess.csv
spatial_resolution: national
temporal_resolution: annual
units: ml
The data files are stored in the ``project/data/scenarios` folder.
The metadata required to define a particular scenario are shown in the table below. It is possible to associate a number of different data sets with the same scenario, so that, for example, choosing the High Population scenario allows users to access both population count and density data in the same or different spatial and temporal resolutions.
| Attribute | Type | Notes |
|---|---|---|
| name | string | |
| description | string | |
| scenario_set | string | |
| parameters | list |
Scenario Parameters¶
The filename in the parameters section within the scenario definition
points to a comma-seperated-values file stored in the ``project/data/scenarios`
folder. For example:
year,region,interval,value
2010,England,1,52000000
2010,Scotland,1,5100000
2010,Wales,1,2900000
2015,England,1,53000000
2015,Scotland,1,5300000
2015,Wales,1,3000000
2020,England,1,54000000
2020,Scotland,1,5500000
2020,Wales,1,3200000
For each entry in the scenario parameters list, the following metadata is required:
| Attribute | Type | Notes |
|---|---|---|
| name | string | |
| spatial_resolution | string | |
| temporal_resolution | string | |
| units | string | |
| filename | string |
Wrapping a Sector Model¶
To integrate a sector model into the system-of-systems model, it is necessary
to write a Python wrapper,
which implements smif.sector_model.SectorModel.
The key methods which need to be overridden are:
smif.sector_model.SectorModel.initialise()smif.sector_model.SectorModel.simulate()smif.sector_model.SectorModel.extract_obj()
The wrapper should be written in a python file, e.g. run.py.
The path to the location of this run.py file should be entered in the
model.yaml file under the path key
(see System-of-Systems Model File above).
To integrate an infrastructure simulation model within the system-of-systems modelling framework, it is also necessary to provide the configuration data. This configuration data includes definitions of the spatial and temporal resolutions of the input and output data to and from the models. This enables the framework to convert data from one spatio-temporal resolution to another.