Wrapping a Simulation Model¶
Overview¶
The Python class smif.model.sector_model.SectorModel
is
script which runs the wrapped model, passes in parameters and writes out results.
The wrapper acts as an interface between the simulation modelling integration framework and the simulation model, keeping all the code necessary to implement the conversion of data types in one place.
In particular, the wrapper must take the smif formatted data, which includes inputs,
parameters, state and pass this data into the wrapped model. After the
simulate()
has run, results from the sector model
must be formatted and passed back into smif.
The handling of data is aided through the use of a set of methods provided by
smif.data_layer.data_handle.DataHandle
, namely:
and
In this section, we describe the process necessary to correctly write this wrapper function, referring to the example project included with the package.
It is difficult to provide exhaustive details for every type of sector model implementation - our decision to leave this largely up to the user is enabled by the flexibility afforded by python. The wrapper can write to a database or structured text file before running a model from a command line prompt, or import a python sector model and pass in parameters values directly. As such, what follows is a recipe of components from which you can construct a wrapper to full integrate your simulation model within smif.
For help or feature requests, please raise issues at the github repository [1] and we will endeavour to provide assistance as resources allow.
Example Wrapper¶
Here’s a reproduction of the example wrapper in the sample project included within smif. In this case, the wrapper doesn’t actually call or run a separate model, but demonstrates calls to the data handler methods necessary to pass data into an external model, and send results back to smif.
"""Energy model
"""
def before_model_run(self, data):
data = data.get_base_timestep_data('energy_demand')
self.logger.info("Energy demand in base year is %s",
data)
def simulate(self, data):
# Get the current timestep
now = data.current_timestep
self.logger.info("EDMWrapper received inputs in %s",
now)
# State
current_interventions = data.get_current_interventions()
self.logger.debug("Current interventions: {}".format(current_interventions.keys()))
# Demonstrates how to get the value for a model parameter
parameter_value = data.get_parameter('smart_meter_savings')
self.logger.info('Smart meter savings: %s', parameter_value)
# Demonstrates how to get the value for a model input
# (defaults to the current time period)
current_energy_demand = data.get_data('energy_demand')
self.logger.info("Current energy demand in %s is %s",
now, current_energy_demand)
# Demonstrates how to get the value for a model input from the base
# timeperiod
base_energy_demand = data.get_base_timestep_data('energy_demand')
base_year = data.base_timestep
self.logger.info("Base year energy demand in %s was %s", base_year,
base_energy_demand)
# Demonstrates how to get the value for a model input from the previous
# timeperiod
if now > base_year:
prev_energy_demand = data.get_previous_timestep_data('energy_demand')
prev_year = data.previous_timestep
self.logger.info("Previous energy demand in %s was %s",
prev_year, prev_energy_demand)
# Pretend to call the 'energy model'
# This code prints out debug logging messages for each input
# defined in the energy_demand configuration
for name in self.inputs:
self.logger.info(
"Read %s for %s",
data.get_data(name),
name
)
# Write pretend results to data handler
data.set_results("cost", np.ones((3, )) * 3)
data.set_results("water_demand", np.ones((3, )) * 3)
self.logger.info("EDMWrapper produced outputs in %s",
now)
The key methods in the SectorModel class which need to be overridden are:
initialise()
simulate()
extract_obj()
The wrapper should be written in a python file, e.g. water_supply.py
. The path to the
location of this file should be entered in the sector model configuration of the project. (see
A Simulation Model File above).
Implementing a simulate method¶
The most common workflow that will need to be implemented in the simulate method is:
- Retrieve model input and parameter data from the data handler
- Write or pass this data to the wrapped model
- Run the model
- Retrieve results from the model
- Write results back to the data handler
Accessing model parameter data¶
Use the get_parameter()
or
get_parameters()
method as shown in the
example:
parameter_value = data.get_parameter('smart_meter_savings')
Note that the name argument passed to the
get_parameter()
is that which is defined in
the sector model configuration file.
Accessing model input data for the current year¶
The method get_data()
allows a user to get
the value for any model input that has been defined in the sector model’s configuration. In
the example, the option year argument is omitted, and it defaults to fetching the data for the
current timestep:
current_energy_demand = data.get_data('energy_demand')
Accessing model input data for the base year¶
To access model input data from the timestep prior to the current timestep, you can use the following argument:
base_energy_demand = data.get_base_timestep_data('energy_demand')
Accessing model input data for a previous year¶
To access model input data from the timestep prior to the current timestep, you can use the following argument:
prev_energy_demand = data.get_previous_timestep_data('energy_demand')
Passing model data directly to a Python model¶
If the wrapped model is a python script or package, then the wrapper can import and instantiate the model, passing in data directly.
# simulate (wrapping toy model)
instance = ExampleWaterSupplySimulationModel()
instance.raininess = raininess
instance.number_of_treatment_plants = number_of_treatment_plants
instance.reservoir_level = reservoir_level
In this example, the example water supply simulation model is instantiated within the simulate
method, data is written to properties of the instantiated class and the run()
method of
the simulation model is called. Finally, (dummy) results are written back to the data handler
using the set_results()
method.
Alternatively, the wrapper could call the model via the command line (see below).
Passing model data in as a command line argument¶
If the model is fairly simple, or requires a parameter value or input data to be passed as an
argument on the command line, use the methods provided by subprocess
to call out to
the model from the wrapper:
parameter = data.get_parameter('command_line_argument')
arguments = ['path/to/model/executable',
'-my_argument={}'.format(parameter)]
output = subprocess.run(arguments, check=True)
Writing data to a text file¶
Again, the exact implementation of writing data to a text file for subsequent reading into the wrapped model will differ on a case-by-case basis. In the following example, we write some data to a comma-separated-values (.csv) file:
with open(path_to_data_file, 'w') as open_file:
fieldnames = ['year', 'PETROL', 'DIESEL', 'LPG',
'ELECTRICITY', 'HYDROGEN', 'HYBRID']
writer = csv.DictWriter(open_file, fieldnames)
writer.writeheader()
now = data.current_timestep
base_year_enum = RelativeTimestep.BASE
base_price_set = {
'year': base_year_enum.resolve_relative_to(now, data.timesteps),
'PETROL': data.get_data('petrol_price', base_year_enum),
'DIESEL': data.get_data('diesel_price', base_year_enum),
'LPG': data.get_data('lpg_price', base_year_enum),
'ELECTRICITY': data.get_data('electricity_price', base_year_enum),
'HYDROGEN': data.get_data('hydrogen_price', base_year_enum),
'HYBRID': data.get_data('hybrid_price', base_year_enum)
}
current_price_set = {
'year': now,
'PETROL': data.get_data('petrol_price'),
'DIESEL': data.get_data('diesel_price'),
'LPG': data.get_data('lpg_price'),
'ELECTRICITY': data.get_data('electricity_price'),
'HYDROGEN': data.get_data('hydrogen_price'),
'HYBRID': data.get_data('hybrid_price')
}
writer.writerow(base_price_set)
writer.writerow(current_price_set)
Writing data to a database¶
The exact implementation of writing input and parameter data will differ on a case-by-case
basis. In the following example, we write model inputs energy_demand
to a postgreSQL
database table ElecLoad
using the psycopg2 library [2]
def simulate(self, data):
# Open a connection to the database
conn = psycopg2.connect("dbname=vagrant user=vagrant")
# Open a cursor to perform database operations
cur = conn.cursor()
# Returns a numpy array whose dimensions are defined by the interval and
# region definitions
elec_data = data.get_data('electricity_demand')
# Build the SQL string
sql = """INSERT INTO "ElecLoad" (Year, Interval, BusID, ElecLoad)
VALUES (%s, %s, %s, %s)"""
# Get the time interval definitions associated with the input
time_intervals = self.inputs[name].get_interval_names()
# Get the region definitions associated with the input
regions = self.inputs[name].get_region_names()
# Iterate over the regions and intervals (columns and rows) of the numpy
# array holding the energy demand data and write each value into the table
for i, region in enumerate(regions):
for j, interval in enumerate(time_intervals):
# This line calls out to a helper method which associates
# electricity grid bus bars to energy demand regions
bus_number = get_bus_number(region)
# Build the tuple to write to the table
insert_data = (data.current_timestep,
interval,
bus_number,
data[i, j])
cur.execute(sql, insert_data)
# Make the changes to the database persistent
conn.commit()
# Close communication with the database
cur.close()
conn.close()
Writing model results to the data handler¶
Writing results back to the data handler is as simple as calling the
set_results()
method:
data.set_results("cost", np.array([1.23, 1.543, 2.355])
The expected format of the data is an n-dimensional numpy array with the dimensions described
by the shape tuple (len(dim), ...)
where there is an entry for each dimension defined in
the model’s output specification.
A model wrapper can reflect on its outputs and their specs:
# find the spec for a given output
spec = self.outputs[output_name]
# spec dimensions
spec.dims # e.g. ['lad', 'month', 'economic_sector']
# spec shape (length of each dimension)
spec.shape # e.g. (370, 12, 46)
# dimension names (labels for each element in a given dimension)
spec.dim_names('lad') # e.g. ['E070001', 'E060002', ...]
# full metadata about dimension elements
spec.dim_elements('lad') # [{'name': 'E070001', 'feature': {'properties': {...}, 'coordinates': {...}}}, ...]
Results are expected to be set for each of the model outputs defined in the output configuration and a warning is raised if these are not present at runtime.
The interval definitions associated with the output can be interrogated from within the
SectorModel class using self.outputs[name].get_interval_names()
and the regions using
self.outputs[name].get_region_names()
and these can then be used to compose the numpy
array.