smif package¶
Submodules¶
smif.decision module¶
The decision module handles the three planning levels
Currently, only pre-specified planning is implemented.
-
class
smif.decision.
Planning
(planned_interventions=None)[source]¶ Bases:
object
Holds the list of planned interventions, where a single planned intervention is an intervention with a build date after which it will be included in the modelled systems.
For example, a small pumping station might be built in Oxford in 2045:
{ 'name': 'small_pumping_station', 'build_date': 2045 }
-
planned_interventions
¶ list
A list of pre-specified planned interventions
-
names
¶ Returns the set of assets defined in the planned interventions
-
timeperiods
¶ Returns the set of build dates defined in the planned interventions
-
smif.intervention module¶
This module handles the collection of interventions and assets in a sector. The set of interventions describes the targets of possible physical (or non-physical) decisions which can be made in the sector.
An Asset is the general term for an existing component of an infrastructure system.
The difference between an Intervention and an Asset, is that the latter exists (it has been “built”), whereas the former describes the potential to build an Asset.
The set of assets defines the ‘state’ of the infrastructure system.
Notes
This module implements:
- initialisation of the set of assets from model config (either a collection of yaml
text files, or a database)
- hold generic list of key/values
- creation of new assets by decision logic (rule-based/optimisation solver)
- maintain or derive set of possible assets
- makes the distinction between known-ahead values and build-time values. Location and date are specified at build time, while cost and capacity are a function of time and location.
- serialisation for passing to models
- ease of access to full generic data structure
- output list of assets for reporting
- write out with legible or traceable keys and units for verification and understanding
Terminology
- name:
- A category of infrastructure intervention (e.g. power station, policy) which holds default attribute/value pairs. These names can be inherited by asset/intervention definitions to reduce the degree of duplicate data entry.
- asset:
- An instance of an intervention, which represents a single investment
decisions which will take place, or has taken place.
Historical interventions are defined as initial conditions, while
future interventions are listed as pre-specified planning.
Both historical and future interventions can make use of names to
ease data entry. Assets must have
location
,build_date
andname
attributes defined. - intervention:
- A potential asset or investment.
Interventions are defined in the same way as for assets,
cannot have a
build_date
defined.
-
class
smif.intervention.
Asset
(name='', data=None, sector='')[source]¶ Bases:
smif.intervention.Intervention
An instance of an intervention with a build date.
Used to represent pre-specified planning and existing infrastructure assets and interventions
Parameters: - name (str, default="") – The type of asset, which should be unique across all sectors
- data (dict, default=None) – The dictionary of asset attributes
- sector (str, default="") – The sector associated with the asset
-
build_date
¶ The build date of this asset instance (if specified - asset types will not have build dates)
-
class
smif.intervention.
AssetRegister
[source]¶ Bases:
smif.intervention.Register
Register each asset type
-
class
smif.intervention.
Intervention
(name='', data=None, sector='')[source]¶ Bases:
smif.intervention.InterventionContainer
An potential investment to send to the logic-layer
An Intervention, is an investment which has a name (or name), other attributes (such as capital cost and economic lifetime), and location, but no build date.
The set of interventions are defined within each sector, and these are collected into an
InterventionRegister
when asmif.controller.SosModel
is instantiated by the controller at runtime.Parameters: - name (str, default="") – The type of asset, which should be unique across all sectors
- data (dict, default=None) – The dictionary of asset attributes
- sector (str, default="") – The sector associated with the asset
-
location
¶ The location of this asset instance (if specified - asset types may not have explicit locations)
-
class
smif.intervention.
InterventionContainer
(name='', data=None, sector='')[source]¶ Bases:
object
An container for asset types, interventions and assets.
An asset’s data is set up to be a flexible, plain data structure.
Parameters: - name (str, default="") – The type of asset, which should be unique across all sectors
- data (dict, default=None) – The dictionary of asset attributes
- sector (str, default="") – The sector associated with the asset
-
static
deterministic_dict_to_str
(data)[source]¶ Return a reproducible string representation of any dict
Parameters: data (dict) – An intervention attributes dictionary Returns: A reproducible string representation Return type: str
-
get_attributes
()[source]¶ Override to return two lists, one containing required attributes, the other containing omitted attributes
Returns: Tuple of lists, one contained required attributes, the other which must be omitted Return type: tuple
-
sector
¶ The name of the sector model this asset is used in.
-
class
smif.intervention.
InterventionRegister
[source]¶ Bases:
smif.intervention.Register
The collection of Intervention objects
An InterventionRegister contains an immutable collection of sector specific assets and decision points which can be decided on by the Logic Layer
- Reads in a collection of interventions defined in each sector model
- Builds an ordered and immutable collection of interventions
- Provides interfaces to
- optimisation/rule-based planning logic
- SectorModel class model wrappers
Key functions:
- outputs a complete list of asset build possibilities (asset type at location) which are (potentially) constrained by the pre-specified planning instructions and existing infrastructure.
- translate a binary vector of build instructions (e.g. from optimisation routine) into Asset objects with human-readable key-value pairs
- translates an immutable collection of Asset objects into a binary vector to pass to the logic-layer.
Notes
Internal data structures
- Intervention_types
- is a 2D array of integers: each entry is an array representing an Intervention type, each integer indexes attribute_possible_values
- attribute_keys
- is a 1D array of strings
- attribute_possible_values
- is a 2D array of simple values, possibly (boolean, integer, float, string, tuple). Each entry is a list of possible values for the attribute at that index.
Invariants
- there must be one name and one list of possible values per attribute
- each Intervention type must list one value for each attribute, and that value must be a valid index into the possible_values array
- each possible_values array should be all of a single type
-
attribute_value_index
(attr_idx, value)[source]¶ Get the index of a possible value for a given attribute index
-
numeric_to_intervention
(numeric_asset)[source]¶ Convert the numeric representation of an asset back to Asset (with legible key/value data)
Parameters: numeric_asset (list) – A list of integers of length self._attribute_keys Returns: An Intervention
objectReturn type: Intervention Examples
Given a (very minimal) possible state of a register:
>>> register = AssetRegister() >>> register._names = [[1,1,1]] >>> register._attribute_keys = ["name", "capacity", "sector"] >>> register._attribute_possible_values = [ ... [None, "water_treatment_plant"], ... [None, {"value": 5, "units": "ML/day"}], ... [None, "water_supply"] ... ]
Calling this function would piece together the asset:
>>> asset = register.numeric_to_asset([1,1,1]) >>> print(asset) Asset("water_treatment_plant", {"name": "water_treatment_plant", "capacity": {"units": "ML/day", "value": 5}, "sector": "water_supply"})
-
register
(intervention)[source]¶ Add a new intervention to the register
Parameters: intervention ( Intervention
) –
smif.optimisation module¶
Solve the optimal planning problem for a system of systems model
-
smif.optimisation.
define_basic_model
(assets, availability_constraint, asset_costs)[source]¶ Define the binary integer planning problem
Parameters: Returns: model – An abstract instance of the incomplete model with no objective function
Return type: pyomo.environ.ConcreteModel
-
smif.optimisation.
feature_vfa_model
(assets, availability_constraint, asset_costs, feature_coefficients, asset_features)[source]¶ Define the value function approximation
Here we assume that the value function approximation is a function of the features of the state, rather than individual assets, or enumeration of all possible states
Parameters: - assets (list) – The list of assets
- availability_constraint (dict) – A dictionary of binary constraints on whether you can build each asset in assets
- asset_costs (dict) – The investment cost of each asset
- features (list) – The set of features
- feature_coefficients (dict) – The regression coefficients for each feature
- asset_features (dict) – The mapping of features to assets
Returns: model – A concrete instance of the model
Return type: pyomo.environ.ConcreteModel
Notes
The use of features decomposes the optimisation problem into several sub-components. The first is to find the clusters of ‘bits’ in the state, which correctly predict minimal cost investments and operation of infrastructure assets.
These clusters can be used to define a feature, adding an entry to the feature set of features and a column to the asset_features matrix, where 1 indicates that the feature includes that asset.
Initially, feature selection could begin by regressing the results from a random sample of investment policies. This could highlight which patterns of investments seem to result in least cost systems. In addition, the attributes of the assets could provide a set of features which at least help categorise the assets (e.g. according to sector, size, location, and asset type).
The binary integer problem is posed as follows:
where
- is an element in the set of assets
- is an element in the set of features
- is the cost of asset
- is the decision to invest in asset
- is the mapping of feature to asset
- is the basis coefficient of asset and feature
-
smif.optimisation.
formulate_model
(asset_register, availability_constraint, feature_coefficients, asset_features)[source]¶
-
smif.optimisation.
linear_vfa_model
(assets, availability_constraint, asset_costs, asset_value)[source]¶ Define the value function approximation
Here we assume a linear relationship (in the asset)
Parameters: Returns: model – A concrete instance of the model
Return type: pyomo.environ.ConcreteModel
-
smif.optimisation.
solve_model
(model, state=None)[source]¶ Solves the model using glpk
Passing in a state as a list of asset names initialises the state of the model, fixing those decision variables.
Parameters: - model (pyomo.environ.ConcreteModel) – A concrete instance of the model
- state (dict, optional, default=None) – A list of assets which were installed in a previous iteration or time period
- returns (pyomo.core.Model.Instance) – An instance of the model populated with results
-
smif.optimisation.
state_vfa_model
(assets, availability_constraint, asset_costs, asset_value, states)[source]¶ Define the value function approximation
Here we assume that the value function approximation is a function of the state, rather than individual assets
Unfortunately, the number of states becomes very large, growing exponentially in the number of assets, and so representing the approximate value function like this is very inefficient as soon as the number of assets increases above 32 (about 4 GB).
Parameters: - assets (list) – The list of assets
- availability_constraint (dict) – A dictionary of binary constraints on whether you can build each asset in assets
- asset_costs (dict) – The investment cost of each asset
- asset_value (dict) – The value function approximation of each asset
- states (dict) – A dictionary where the keys are tuples of entries in assets and an
index of states (
2 ** len(assets)
) and the value is a binary indicator showing the possible combinations
Returns: model – A concrete instance of the model
Return type: pyomo.environ.ConcreteModel
Notes
smif.parameters module¶
Encapsulates the input or output parameters of a sector model, for example:
- name: petrol_price
spatial_resolution: GB
temporal_resolution: annual
- name: diesel_price
spatial_resolution: GB
temporal_resolution: annual
- name: LPG_price
spatial_resolution: GB
temporal_resolution: annual
-
class
smif.parameters.
ModelParameters
(parameters)[source]¶ Bases:
object
A container for all the model inputs
Parameters: inputs (list) – A list of dicts of model parameter name, spatial resolution and temporal resolution -
get_spatial_res
(name)[source]¶ The spatial resolution for parameter name
Parameters: name (str) – The name of a model parameter
-
get_temporal_res
(name)[source]¶ The temporal resolution for parameter name
Parameters: name (str) – The name of a model parameter
-
names
¶
-
parameters
¶ A list of the model parameters
Returns: Return type: smif.parameters.ParameterList
-
-
class
smif.parameters.
Parameter
(name, spatial_resolution, temporal_resolution)¶ Bases:
tuple
-
__getnewargs__
()¶ Return self as a plain tuple. Used by copy and pickle.
-
static
__new__
(_cls, name, spatial_resolution, temporal_resolution)¶ Create new instance of Parameter(name, spatial_resolution, temporal_resolution)
-
__repr__
()¶ Return a nicely formatted representation string
-
name
¶ Alias for field number 0
-
spatial_resolution
¶ Alias for field number 1
-
temporal_resolution
¶ Alias for field number 2
-
smif.sector_model module¶
This module acts as a bridge to the sector models from the controller
The SectorModel
exposes several key methods for running wrapped
sector models. To add a sector model to an instance of the framework,
first implement SectorModel
.
Key Functions¶
This class performs several key functions which ease the integration of sector models into the system-of-systems framework.
The user must implement the various abstract functions throughout the class to
provide an interface to the sector model, which can be called upon by the
framework. From the model’s perspective, SectorModel
provides a bridge
from the sector-specific problem representation to the general representation
which allows reasoning across infrastructure systems.
The key functions include
- converting input/outputs to/from geographies/temporal resolutions
- converting control vectors from the decision layer of the framework, to asset Interventions specific to the sector model
- returning scaler/vector values to the framework to enable measurements of performance, particularly for the purposes of optimisation and rule-based approaches
-
class
smif.sector_model.
SectorModel
[source]¶ Bases:
abc.ABC
A representation of the sector model with inputs and outputs
-
extract_obj
(results)[source]¶ Implement this method to return a scalar value objective function
This method should take the results from the output of the
simulate()
method, process the results, and return a scalar value which can be used as a component of the objective function by the decision layerParameters: results (dict) – A nested dict of the results from the simulate()
methodReturns: A scalar component generated from the simulation model results Return type: float
-
initialise
(initial_conditions)[source]¶ Implement this method to set up the model system
Parameters: initial_conditions (list) – A list of past Interventions, with build dates and locations as necessary to specify the infrastructure system to be modelled.
-
inputs
¶ The inputs to the model
The inputs should be specified in a list. For example:
- name: eletricity_price spatial_resolution: GB temporal_resolution: annual
Parameters: value (list) – A list of dicts of inputs to the model. These includes parameters, assets and exogenous data Returns: Return type: smif.parameters.ModelInputs
-
intervention_names
¶ The names of the interventions
Returns: A list of the names of the interventions Return type: list
-
outputs
¶ The outputs from the model
Parameters: value (list) – A list of dicts of outputs from the model. This may include results and metrics Returns: Return type: smif.parameters.ModelParameters
-
simulate
(decisions, state, data)[source]¶ Implement this method to run the model
Parameters: Returns: A dictionary of the format:
results[parameter] = [SpaceTimeValue(region, interval, value, units), ...]
Return type: Notes
In the results returned from the
simulate()
method:interval
- should reference an id from the interval set corresponding to the output parameter, as specified in model configuration
region
- should reference a region name from the region set corresponding to the output parameter, as specified in model configuration
-
-
class
smif.sector_model.
SectorModelBuilder
(name)[source]¶ Bases:
object
Build the components that make up a sectormodel from the configuration
Parameters: name (str) – The name of the sector model
smif.sos_model module¶
This module coordinates the software components that make up the integration framework.
-
class
smif.sos_model.
ModelSet
(models, sos_model)[source]¶ Bases:
object
Wraps a set of interdependent models
Given a directed graph of dependencies between models, any cyclic dependencies are contained within the strongly-connected components of the graph.
A ModelSet corresponds to the set of models within a single strongly- connected component. If this is a set of one model, it can simply be run deterministically. Otherwise, this class provides the machinery necessary to find a solution to each of the interdependent models.
The current implementation first estimates the outputs for each model in the set, guaranteeing that each model will then be able to run, then begins iterating, running every model in the set at each iteration, monitoring the model outputs over the iterations, and stopping at timeout, divergence or convergence.
Notes
This calls back into
SosModel
quite extensively for state, data, decisions, regions and intervals.-
converged
(timestep)[source]¶ Check whether the results of a set of models have converged.
Returns: converged – True if the results have converged to within a tolerance Return type: bool Raises: DiverganceError
– If the results appear to be diverging
-
-
class
smif.sos_model.
RunMode
[source]¶ Bases:
enum.Enum
Enumerates the operating modes of a SoS model
-
dynamic_optimisation
= <RunMode.dynamic_optimisation: 3>¶
-
sequential_simulation
= <RunMode.sequential_simulation: 1>¶
-
static_optimisation
= <RunMode.static_optimisation: 2>¶
-
static_simulation
= <RunMode.static_simulation: 0>¶
-
-
class
smif.sos_model.
SosModel
[source]¶ Bases:
object
Consists of the collection of timesteps and sector models
This is NISMOD - i.e. the system of system model which brings all of the sector models together. Sector models may be joined through dependencies.
This class is populated at runtime by the
SosModelBuilder
and called fromsmif.cli.run_model()
.-
models
¶ dict
This is a dictionary of
smif.SectorModel
-
initial_conditions
¶ list
List of interventions required to set up the initial system, with any state attributes provided here too
-
static
add_data_series
(list_a, list_b)[source]¶ Given two lists of SpaceTimeValues of identical spatial and temporal resolution, return a single list with matching values added together.
Notes
Assumes a data series is not sparse, i.e. has a value for every region/interval combination
-
determine_running_mode
()[source]¶ Determines from the config in what mode to run the model
Returns: The mode in which to run the model Return type: RunMode
-
get_data
(model, timestep)[source]¶ Gets the data in the required format to pass to the simulate method
Returns: A nested dictionary of the format: data[parameter][region][time_interval] = {value, units}
Return type: dict Notes
Note that the timestep is not passed to the SectorModel in the nested data dictionary. The current timestep is available in
data['timestep']
.
-
get_decisions
(model, timestep)[source]¶ Gets the interventions that correspond to the decisions
Parameters: - model (
smif.sector_model.SectorModel
) – The instance of the sector model wrapper to run - timestep (int) – The current model year
- model (
-
inputs
¶ A dictionary of model names associated with an inputs
Returns: Keys are parameter names, value is a list of sector model names Return type: dict
-
intervention_names
¶ Names (id-like keys) of all known asset type
-
outputs
¶ Model names associated with model outputs & scenarios
Returns: Keys are parameter names, value is a list of sector model names Return type: dict
-
resolution_mapping
¶ Returns the temporal and spatial mapping to an input, output or scenario parameter
Example
The data structure follows
source->parameter->{temporal, spatial}
:{ 'scenario': { 'raininess': { 'temporal_resolution': 'annual', 'spatial_resolution': 'LSOA' } } }
-
results
¶ Get nested dict of model results
Returns: Nested dictionary in the format results[int:year][str:model][str:parameter] => list of SpaceTimeValues Return type: dict
-
run
()[source]¶ Runs the system-of-system model
- Determine run mode
- Determine running order
- Run each sector model
- Return success or failure
-
run_sector_model
(model_name)[source]¶ Runs the sector model
Parameters: model_name (str) – The name of the model, corresponding to the folder name in the models subfolder of the project folder
-
run_sector_model_timestep
(model, timestep)[source]¶ Run the sector model for a specific timestep
Parameters: - model (
smif.sector_model.SectorModel
) – The instance of the sector model wrapper to run - timestep (int) – The year for which to run the model
- model (
-
scenario_data
¶ Get nested dict of scenario data
Returns: Nested dictionary in the format data[year][param] = SpaceTimeValue(region, interval, value, unit)
Return type: dict
-
set_data
(model, timestep, results)[source]¶ Sets results output from model as data available to other/future models
Stores only latest estimated results (i.e. not holding on to iterations here while trying to solve interdependencies)
-
-
class
smif.sos_model.
SosModelBuilder
[source]¶ Bases:
object
Constructs a system-of-systems model
Builds a
SosModel
.Examples
Call
SosModelBuilder.construct()
to populate aSosModel
object andSosModelBuilder.finish()
to return the validated and dependency-checked system-of-systems model.>>> builder = SosModelBuilder() >>> builder.construct(config_data) >>> sos_model = builder.finish()
-
add_initial_conditions
(model_name, initial_conditions)[source]¶ Adds initial conditions (state) for a model
-
add_model
(model)[source]¶ Adds a sector model to the system-of-systems model
Parameters: model ( smif.sector_model.SectorModel
) – A sector model wrapper
-
add_model_data
(model, model_data)[source]¶ Adds sector model data to the system-of-systems model which is convenient to have available at the higher level.
-
add_planning
(planning)[source]¶ Loads the planning logic into the system of systems model
Pre-specified planning interventions are defined at the sector-model level, read in through the SectorModel class, but populate the intervention register in the controller.
Parameters: planning (list) – A list of planning instructions
-
add_resolution_mapping
(resolution_mapping)[source]¶ Parameters: resolution_mapping (dict) – A dictionary containing information on the spatial and temporal resoultion of scenario data Example
The data structure follows
source->parameter->{temporal, spatial}
:{'scenario': { 'raininess': {'temporal_resolution': 'annual', 'spatial_resolution': 'LSOA'}}}
-
add_scenario_data
(data)[source]¶ Load the scenario data into the system of systems model
Expect a dictionary, where each key maps a parameter name to a list of data, each observation with:
- timestep
- value
- units
- region (must use a region id from scenario regions)
- interval (must use an id from scenario time intervals)
Add a dictionary of list of
smif.SpaceTimeValue
named tuples, for ease of iteration:data[year][param] = SpaceTimeValue(region, interval, value, units)
Default region: “national” Default interval: “annual”
-
add_timesteps
(timesteps)[source]¶ Set the timesteps of the system-of-systems model
Parameters: timesteps (list) – A list of timesteps
-
construct
(config_data)[source]¶ Set up the whole SosModel
Parameters: config_data (dict) – A valid system-of-systems model configuration dictionary
-
finish
()[source]¶ Returns a configured system-of-systems model ready for operation
Includes validation steps, e.g. to check dependencies
-
static
intervention_state_from_data
(intervention_data)[source]¶ Unpack an intervention from the initial system to extract StateData
-
load_interval_sets
(interval_sets)[source]¶ Loads the time-interval sets into the system-of-system model
Parameters: interval_sets (list) – A dict, where key is the name of the interval set, and the value the data
-
load_models
(model_data_list)[source]¶ Loads the sector models into the system-of-systems model
Parameters:
-