Wrapping a Simulation Model

Overview

The Python class smif.model.sector_model.SectorModel is script which runs the wrapped model, passes in parameters and writes out results.

The wrapper acts as an interface between the simulation modelling integration framework and the simulation model, keeping all the code necessary to implement the conversion of data types in one place.

In particular, the wrapper must take the smif formatted data, which includes inputs, parameters, state and pass this data into the wrapped model. After the simulate() has run, results from the sector model must be formatted and passed back into smif.

The handling of data is aided through the use of a set of methods provided by smif.data_layer.data_handle.DataHandle, namely:

and

In this section, we describe the process necessary to correctly write this wrapper function, referring to the example project included with the package.

It is difficult to provide exhaustive details for every type of sector model implementation - our decision to leave this largely up to the user is enabled by the flexibility afforded by python. The wrapper can write to a database or structured text file before running a model from a command line prompt, or import a python sector model and pass in parameters values directly. As such, what follows is a recipe of components from which you can construct a wrapper to full integrate your simulation model within smif.

For help or feature requests, please raise issues at the github repository [1] and we will endeavour to provide assistance as resources allow.

Example Wrapper

Here’s a reproduction of the example wrapper in the sample project included within smif. In this case, the wrapper doesn’t actually call or run a separate model, but demonstrates calls to the data handler methods necessary to pass data into an external model, and send results back to smif.

    """Energy model"""

    def before_model_run(self, data):

        data = data.get_base_timestep_data("energy_demand")
        self.logger.info("Energy demand in base year is %s", data)

    def simulate(self, data):

        # Get the current timestep

        now = data.current_timestep
        self.logger.info("EDMWrapper received inputs in %s", now)

        # State

        current_interventions = data.get_current_interventions()

        self.logger.debug(
            "Current interventions: {}".format(current_interventions.keys())
        )

        # Demonstrates how to get the value for a model parameter
        parameter_value = data.get_parameter("smart_meter_savings")
        self.logger.info("Smart meter savings: %s", parameter_value)

        # Demonstrates how to get the value for a model input
        # (defaults to the current time period)
        current_energy_demand = data.get_data("energy_demand")
        self.logger.info(
            "Current energy demand in %s is %s", now, current_energy_demand
        )

        # Demonstrates how to get the value for a model input from the base
        # timeperiod
        base_energy_demand = data.get_base_timestep_data("energy_demand")
        base_year = data.base_timestep
        self.logger.info(
            "Base year energy demand in %s was %s", base_year, base_energy_demand
        )

        # Demonstrates how to get the value for a model input from the previous
        # timeperiod
        if now > base_year:
            prev_energy_demand = data.get_previous_timestep_data("energy_demand")
            prev_year = data.previous_timestep
            self.logger.info(
                "Previous energy demand in %s was %s", prev_year, prev_energy_demand
            )

        # Pretend to call the 'energy model'
        # This code prints out debug logging messages for each input
        # defined in the energy_demand configuration
        for name in self.inputs:
            self.logger.info("Read %s for %s", data.get_data(name), name)

        # Write pretend results to data handler
        data.set_results("cost", np.ones((3,)) * 3)
        data.set_results("water_demand", np.ones((3,)) * 3)

        self.logger.info("EDMWrapper produced outputs in %s", now)

    def extract_obj(self, results):

The key methods in the SectorModel class which need to be overridden are:

The wrapper should be written in a python file, e.g. water_supply.py. The path to the location of this file should be entered in the sector model configuration of the project. (see A Simulation Model File above).

Implementing a simulate method

The most common workflow that will need to be implemented in the simulate method is:

  1. Retrieve model input and parameter data from the data handler
  2. Write or pass this data to the wrapped model
  3. Run the model
  4. Retrieve results from the model
  5. Write results back to the data handler

Accessing model parameter data

Use the get_parameter() or get_parameters() method as shown in the example:

parameter_value = data.get_parameter('smart_meter_savings')

Note that the name argument passed to the get_parameter() is that which is defined in the sector model configuration file.

Accessing model input data for the current year

The method get_data() allows a user to get the value for any model input that has been defined in the sector model’s configuration. In the example, the option year argument is omitted, and it defaults to fetching the data for the current timestep:

current_energy_demand = data.get_data('energy_demand')

Accessing model input data for the base year

To access model input data from the timestep prior to the current timestep, you can use the following argument:

base_energy_demand = data.get_base_timestep_data('energy_demand')

Accessing model input data for a previous year

To access model input data from the timestep prior to the current timestep, you can use the following argument:

prev_energy_demand = data.get_previous_timestep_data('energy_demand')

Passing model data directly to a Python model

If the wrapped model is a python script or package, then the wrapper can import and instantiate the model, passing in data directly.


# Parameters
self.logger.debug(data.get_parameters())

# simulate (wrapping toy model)

In this example, the example water supply simulation model is instantiated within the simulate method, data is written to properties of the instantiated class and the run() method of the simulation model is called. Finally, (dummy) results are written back to the data handler using the set_results() method.

Alternatively, the wrapper could call the model via the command line (see below).

Passing model data in as a command line argument

If the model is fairly simple, or requires a parameter value or input data to be passed as an argument on the command line, use the methods provided by subprocess to call out to the model from the wrapper:

parameter = data.get_parameter('command_line_argument')
arguments = ['path/to/model/executable',
             '-my_argument={}'.format(parameter)]
output = subprocess.run(arguments, check=True)

Writing data to a text file

Again, the exact implementation of writing data to a text file for subsequent reading into the wrapped model will differ on a case-by-case basis. In the following example, we write some data to a comma-separated-values (.csv) file:

with open(path_to_data_file, 'w') as open_file:
    fieldnames = ['year', 'PETROL', 'DIESEL', 'LPG',
                  'ELECTRICITY', 'HYDROGEN', 'HYBRID']
    writer = csv.DictWriter(open_file, fieldnames)
    writer.writeheader()

    now = data.current_timestep
    base_year_enum = RelativeTimestep.BASE

    base_price_set = {
        'year': base_year_enum.resolve_relative_to(now, data.timesteps),
        'PETROL': data.get_data('petrol_price', base_year_enum),
        'DIESEL': data.get_data('diesel_price', base_year_enum),
        'LPG': data.get_data('lpg_price', base_year_enum),
        'ELECTRICITY': data.get_data('electricity_price', base_year_enum),
        'HYDROGEN': data.get_data('hydrogen_price', base_year_enum),
        'HYBRID': data.get_data('hybrid_price', base_year_enum)
    }

    current_price_set = {
        'year': now,
        'PETROL': data.get_data('petrol_price'),
        'DIESEL': data.get_data('diesel_price'),
        'LPG': data.get_data('lpg_price'),
        'ELECTRICITY': data.get_data('electricity_price'),
        'HYDROGEN': data.get_data('hydrogen_price'),
        'HYBRID': data.get_data('hybrid_price')
    }

    writer.writerow(base_price_set)
    writer.writerow(current_price_set)

Writing data to a database

The exact implementation of writing input and parameter data will differ on a case-by-case basis. In the following example, we write model inputs energy_demand to a postgreSQL database table ElecLoad using the psycopg2 library [2]

def simulate(self, data):

    # Open a connection to the database
    conn = psycopg2.connect("dbname=vagrant user=vagrant")
    # Open a cursor to perform database operations
    cur = conn.cursor()

    # Returns a smif.DataArray
    demand = data.get_data('electricity_demand')

    # Convert to DataFrame (could also convert to numpy.ndarray or xarray.DataArray)
    demand_df = demand.as_df().reset_index()  # reset index for easier iteration later on

    # Build the SQL string
    sql = """INSERT INTO "ElecLoad" (Year, Interval, BusID, ElecLoad)
             VALUES (%s, %s, %s, %s)"""

    # Get the dimensions associated with the input
    # e.g. ['bus_region', 'hour']
    dims = demand.dims

    # Get the dimension definitions associated with the input, here regions and intervals
    regions = demand.dim_elements('bus_regions')
    intervals = demand.dim_elements('hours')

    # Iterate over the energy demand data and write each value into the table
    for item in demand_df.itertuples():
            # This line calls out to a helper method which associates
            # electricity grid bus bars to energy demand regions
            bus_number = get_bus_number(item.bus_region)
            # Build the tuple to write to the table
            insert_data = (
                data.current_timestep,
                item.hour,
                bus_number,
                item.electricity_demand
            )
            cur.execute(sql, insert_data)

    # Make the changes to the database persistent
    conn.commit()

    # Close communication with the database
    cur.close()
    conn.close()

Writing model results to the data handler

Writing results back to the data handler is as simple as calling the set_results() method:

data.set_results("cost", np.array([1.23, 1.543, 2.355])

The expected format of the data is an n-dimensional numpy array with the dimensions described by the shape tuple (len(dim), ...) where there is an entry for each dimension defined in the model’s output specification.

A model wrapper can reflect on its outputs and their specs:

# find the spec for a given output
spec = self.outputs[output_name]

# spec dimensions
spec.dims  # e.g. ['lad', 'month', 'economic_sector']

# spec shape (length of each dimension)
spec.shape  # e.g. (370, 12, 46)

# dimension names (labels for each element in a given dimension)
spec.dim_names('lad')  # e.g. ['E070001', 'E060002', ...]

# full metadata about dimension elements
spec.dim_elements('lad')
# e.g. [{'name': 'E070001', 'feature': {'properties': {...}, 'coordinates': {...}}}, ...]

Results are expected to be set for each of the model outputs defined in the output configuration and a warning is raised if these are not present at runtime.