=========================== Wrapping a Simulation Model =========================== Overview -------- The Python class :class:`smif.model.sector_model.SectorModel` is script which runs the wrapped model, passes in parameters and writes out results. The wrapper acts as an interface between the simulation modelling integration framework and the simulation model, keeping all the code necessary to implement the conversion of data types in one place. In particular, the wrapper must take the smif formatted data, which includes inputs, parameters, state and pass this data into the wrapped model. After the :py:meth:`~smif.model.sector_model.SectorModel.simulate` has run, results from the sector model must be formatted and passed back into smif. The handling of data is aided through the use of a set of methods provided by :class:`smif.data_layer.data_handle.DataHandle`, namely: - :py:meth:`~smif.data_layer.data_handle.DataHandle.get_data` - :py:meth:`~smif.data_layer.data_handle.DataHandle.get_parameter` - :py:meth:`~smif.data_layer.data_handle.DataHandle.get_parameters` - :py:meth:`~smif.data_layer.data_handle.DataHandle.get_results` and - :py:meth:`~smif.data_layer.data_handle.DataHandle.set_results` In this section, we describe the process necessary to correctly write this wrapper function, referring to the example project included with the package. It is difficult to provide exhaustive details for every type of sector model implementation - our decision to leave this largely up to the user is enabled by the flexibility afforded by python. The wrapper can write to a database or structured text file before running a model from a command line prompt, or import a python sector model and pass in parameters values directly. As such, what follows is a recipe of components from which you can construct a wrapper to full integrate your simulation model within smif. For help or feature requests, please raise issues at the github repository [1]_ and we will endeavour to provide assistance as resources allow. Example Wrapper ~~~~~~~~~~~~~~~ Here's a reproduction of the example wrapper in the sample project included within smif. In this case, the wrapper doesn't actually call or run a separate model, but demonstrates calls to the data handler methods necessary to pass data into an external model, and send results back to smif. .. literalinclude:: ../src/smif/sample_project/models/energy_demand.py :language: python :lines: 8-70 The key methods in the SectorModel class which need to be overridden are: - :py:meth:`~smif.model.sector_model.SectorModel.initialise` - :py:meth:`~smif.model.sector_model.SectorModel.simulate` - :py:meth:`~smif.model.sector_model.SectorModel.extract_obj` The wrapper should be written in a python file, e.g. ``water_supply.py``. The path to the location of this file should be entered in the sector model configuration of the project. (see A Simulation Model File above). Implementing a `simulate` method -------------------------------- The most common workflow that will need to be implemented in the simulate method is: 1. Retrieve model input and parameter data from the data handler 2. Write or pass this data to the wrapped model 3. Run the model 4. Retrieve results from the model 5. Write results back to the data handler Accessing model parameter data ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Use the :py:meth:`~smif.data_layer.data_handle.DataHandle.get_parameter` or :py:meth:`~smif.data_layer.data_handle.DataHandle.get_parameters` method as shown in the example:: parameter_value = data.get_parameter('smart_meter_savings') Note that the name argument passed to the :py:meth:`~smif.data_layer.data_handle.DataHandle.get_parameter` is that which is defined in the sector model configuration file. Accessing model input data for the current year ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The method :py:meth:`~smif.data_layer.data_handle.DataHandle.get_data()` allows a user to get the value for any model input that has been defined in the sector model's configuration. In the example, the option year argument is omitted, and it defaults to fetching the data for the current timestep:: current_energy_demand = data.get_data('energy_demand') Accessing model input data for the base year ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To access model input data from the timestep prior to the current timestep, you can use the following argument:: base_energy_demand = data.get_base_timestep_data('energy_demand') Accessing model input data for a previous year ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To access model input data from the timestep prior to the current timestep, you can use the following argument:: prev_energy_demand = data.get_previous_timestep_data('energy_demand') Passing model data directly to a Python model ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the wrapped model is a python script or package, then the wrapper can import and instantiate the model, passing in data directly. .. literalinclude:: ../src/smif/sample_project/models/water_supply.py :language: python :lines: 76-80 :dedent: 8 In this example, the example water supply simulation model is instantiated within the simulate method, data is written to properties of the instantiated class and the ``run()`` method of the simulation model is called. Finally, (dummy) results are written back to the data handler using the :py:meth:`~smif.data_layer.data_handle.DataHandle.set_results` method. Alternatively, the wrapper could call the model via the command line (see below). Passing model data in as a command line argument ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If the model is fairly simple, or requires a parameter value or input data to be passed as an argument on the command line, use the methods provided by :py:mod:`subprocess` to call out to the model from the wrapper:: parameter = data.get_parameter('command_line_argument') arguments = ['path/to/model/executable', '-my_argument={}'.format(parameter)] output = subprocess.run(arguments, check=True) Writing data to a text file ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Again, the exact implementation of writing data to a text file for subsequent reading into the wrapped model will differ on a case-by-case basis. In the following example, we write some data to a comma-separated-values (.csv) file:: with open(path_to_data_file, 'w') as open_file: fieldnames = ['year', 'PETROL', 'DIESEL', 'LPG', 'ELECTRICITY', 'HYDROGEN', 'HYBRID'] writer = csv.DictWriter(open_file, fieldnames) writer.writeheader() now = data.current_timestep base_year_enum = RelativeTimestep.BASE base_price_set = { 'year': base_year_enum.resolve_relative_to(now, data.timesteps), 'PETROL': data.get_data('petrol_price', base_year_enum), 'DIESEL': data.get_data('diesel_price', base_year_enum), 'LPG': data.get_data('lpg_price', base_year_enum), 'ELECTRICITY': data.get_data('electricity_price', base_year_enum), 'HYDROGEN': data.get_data('hydrogen_price', base_year_enum), 'HYBRID': data.get_data('hybrid_price', base_year_enum) } current_price_set = { 'year': now, 'PETROL': data.get_data('petrol_price'), 'DIESEL': data.get_data('diesel_price'), 'LPG': data.get_data('lpg_price'), 'ELECTRICITY': data.get_data('electricity_price'), 'HYDROGEN': data.get_data('hydrogen_price'), 'HYBRID': data.get_data('hybrid_price') } writer.writerow(base_price_set) writer.writerow(current_price_set) Writing data to a database ~~~~~~~~~~~~~~~~~~~~~~~~~~ The exact implementation of writing input and parameter data will differ on a case-by-case basis. In the following example, we write model inputs ``energy_demand`` to a postgreSQL database table ``ElecLoad`` using the psycopg2 library [2]_ :: def simulate(self, data): # Open a connection to the database conn = psycopg2.connect("dbname=vagrant user=vagrant") # Open a cursor to perform database operations cur = conn.cursor() # Returns a smif.DataArray demand = data.get_data('electricity_demand') # Convert to DataFrame (could also convert to numpy.ndarray or xarray.DataArray) demand_df = demand.as_df().reset_index() # reset index for easier iteration later on # Build the SQL string sql = """INSERT INTO "ElecLoad" (Year, Interval, BusID, ElecLoad) VALUES (%s, %s, %s, %s)""" # Get the dimensions associated with the input # e.g. ['bus_region', 'hour'] dims = demand.dims # Get the dimension definitions associated with the input, here regions and intervals regions = demand.dim_elements('bus_regions') intervals = demand.dim_elements('hours') # Iterate over the energy demand data and write each value into the table for item in demand_df.itertuples(): # This line calls out to a helper method which associates # electricity grid bus bars to energy demand regions bus_number = get_bus_number(item.bus_region) # Build the tuple to write to the table insert_data = ( data.current_timestep, item.hour, bus_number, item.electricity_demand ) cur.execute(sql, insert_data) # Make the changes to the database persistent conn.commit() # Close communication with the database cur.close() conn.close() Writing model results to the data handler ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Writing results back to the data handler is as simple as calling the :py:meth:`~smif.data_layer.data_handle.DataHandle.set_results` method:: data.set_results("cost", np.array([1.23, 1.543, 2.355]) The expected format of the data is an n-dimensional numpy array with the dimensions described by the shape tuple ``(len(dim), ...)`` where there is an entry for each dimension defined in the model's output specification. A model wrapper can reflect on its outputs and their specs:: # find the spec for a given output spec = self.outputs[output_name] # spec dimensions spec.dims # e.g. ['lad', 'month', 'economic_sector'] # spec shape (length of each dimension) spec.shape # e.g. (370, 12, 46) # dimension names (labels for each element in a given dimension) spec.dim_names('lad') # e.g. ['E070001', 'E060002', ...] # full metadata about dimension elements spec.dim_elements('lad') # e.g. [{'name': 'E070001', 'feature': {'properties': {...}, 'coordinates': {...}}}, ...] Results are expected to be set for each of the model outputs defined in the output configuration and a warning is raised if these are not present at runtime. References ---------- .. [1] https://github.com/nismod/smif/issues .. [2] http://initd.org/psycopg/