smif.data_layer.datafile_interface module

File-backed data interface

Summary

Classes:

DatafileInterface

Read and write interface to YAML / CSV configuration files and intermediate CSV / native-binary data storage.

Reference

class smif.data_layer.datafile_interface.DatafileInterface(base_folder, storage_format='local_binary', timestamp='yyyy_mm_dd_hhmm')

Bases: smif.data_layer.data_interface.DataInterface

Read and write interface to YAML / CSV configuration files and intermediate CSV / native-binary data storage.

Project.yml

Parameters:

• base_folder (str) – The path to the configuration and data files
• storage_format (str) – The format used to store intermediate data (local_csv, local_binary)
• timestamp (str) – The ISO-8601 timestamp that identifies the modelrun (%y%m%dT%H%M%S)

read_units_file_name()

read_sos_model_runs()

Read all system-of-system model runs from Yaml files

sos_model_runs.yml

Returns:A list of sos_model_run dicts Return type:list

read_sos_model_run(sos_model_run_name)

Read a system-of-system model run

Parameters:sos_model_run_name (str) – A sos_model_run name Returns:sos_model_run – A sos_model_run dictionary Return type:dict

write_sos_model_run(sos_model_run)

Write system-of-system model run to Yaml file

Parameters:sos_model_run (dict) – A sos_model_run dictionary

update_sos_model_run(sos_model_run_name, sos_model_run)

Update system-of-system model run in Yaml file

Parameters:

• sos_model_run_name (str) – A sos_model_run name
• sos_model_run (dict) – A sos_model_run dictionary

delete_sos_model_run(sos_model_run_name)

Delete a system-of-system model run

Parameters:sos_model_run_name (str) – A sos_model_run name

read_sos_models()

Read all system-of-system models from Yaml files

Returns:A list of sos_models dicts Return type:list

read_sos_model(sos_model_name)

Read a specific system-of-system model

Parameters:sos_model_name (str) – A sos_model name Returns:sos_model – A sos_model dictionary Return type:dict

write_sos_model(sos_model)

Write system-of-system model to Yaml file

Parameters:sos_model (dict) – A sos_model dictionary

update_sos_model(sos_model_name, sos_model)

Update system-of-system model in Yaml file

Parameters:

• sos_model_name (str) – A sos_model name
• sos_model (dict) – A sos_model dictionary

delete_sos_model(sos_model_name)

Delete a system-of-system model run

Parameters:sos_model_name (str) – A sos_model name

read_sector_models()

Read all sector models from Yaml files

sector_models.yml

Returns:A list of sector_model dicts Return type:list

read_sector_model(sector_model_name)

Read a sector model

Parameters:sector_model_name (str) – A sector_model name Returns:sector_model – A sector_model dictionary Return type:dict

write_sector_model(sector_model)

Write sector model to Yaml file

Parameters:sector_model (dict) – A sector_model dictionary

update_sector_model(sector_model_name, sector_model)

Update sector model in Yaml file

Parameters:

• sector_model_name (str) – A sector_model name
• sector_model (dict) – A sector_model dictionary

delete_sector_model(sector_model_name)

Delete a sector model

Parameters:sector_model_name (str) – A sector_model name

read_interventions(filename)

Read the interventions from filename

Parameters:filename (str) – The name of the intervention yml file to read in

read_initial_conditions(filename)

Read the initial conditions from filename

Parameters:filename (str) – The name of the initial conditions yml file to read in

read_region_definitions()

Read region_definitions from project configuration

Returns:A list of region_definition dicts Return type:list

read_region_definition_data(region_definition_name)

Read region_definition data file into a Fiona feature collection

The file format must be possible to parse with GDAL, and must contain an attribute “name” to use as an identifier for the region_definition.

Parameters:region_definition_name (str) – Name of the region_definition Returns:A list of data from the specified file in a fiona formatted dict Return type:list

write_region_definition(region_definition)

Write region_definition to project configuration

Parameters:region_definition (dict) – A region_definition dict

update_region_definition(region_definition_name, region_definition)

Update region_definition to project configuration

Parameters:

• region_definition_name (str) – Name of the (original) entry
• region_definition (dict) – The updated region_definition dict

read_interval_definitions()

Read interval_definition sets from project configuration

Returns:A list of interval_definition set dicts Return type:list

read_interval_definition_data(interval_definition_name)

Parameters:interval_definition_name (str) – Returns:Interval definition data Return type:dict

Notes

Expects csv file to contain headings of id, start, end

write_interval_definition(interval_definition)

Write interval_definition to project configuration

Parameters:interval_definition (dict) – A interval_definition dict

update_interval_definition(interval_definition_name, interval_definition)

Update interval_definition to project configuration

Parameters:

• interval_definition_name (str) – Name of the (original) entry
• interval_definition (dict) – The updated interval_definition dict

read_scenario_sets()

Read scenario sets from project configuration

Returns:A list of scenario set dicts Return type:list

read_scenario_set(scenario_set_name)

Read a scenario_set

Parameters:scenario_set_name (str) – Name of the scenario_set Returns:Scenario set definition Return type:dict

read_scenario_set_scenario_definitions(scenario_set_name)

Read all scenarios from a certain scenario_set

Parameters:scenario_set_name (str) – Name of the scenario_set Returns:A list of scenarios within the specified ‘scenario_set_name’ Return type:list

read_scenario_definition(scenario_name)

Read scenario definition data

Parameters:scenario_name (str) – Name of the scenario Returns:The scenario definition Return type:dict

write_scenario_set(scenario_set)

Write scenario_set to project configuration

Parameters:scenario_set (dict) – A scenario_set dict

update_scenario_set(scenario_set_name, scenario_set)

Update scenario_set to project configuration

Parameters:

• scenario_set_name (str) – Name of the (original) entry
• scenario_set (dict) – The updated scenario_set dict

delete_scenario_set(scenario_set_name)

Delete scenario_set from project configuration and all scenarios within scenario_set

Parameters:scenario_set_name (str) – A scenario_set name

read_scenarios()

Read scenarios from project configuration

Returns:A list of scenario dicts Return type:list

read_scenario(scenario_name)

Read a scenario

Parameters:scenario_name (str) – Name of the scenario Returns:A scenario dictionary Return type:dict

write_scenario(scenario)

Write scenario to project configuration

Parameters:scenario (dict) – A scenario dict

update_scenario(scenario_name, scenario)

Update scenario to project configuration

Parameters:

• scenario_name (str) – Name of the (original) entry
• scenario (dict) – The updated scenario dict

delete_scenario(scenario_name)

Delete scenario from project configuration

Parameters:scenario_name (str) – A scenario name

read_scenario_data(scenario_name, facet_name, spatial_resolution, temporal_resolution, timestep)

Read scenario data file

Parameters:

• scenario_name (str) – Name of the scenario
• facet_name (str) – Name of the scenario facet to read
• spatial_resolution (str) –
• temporal_resolution (str) –
• timestep (int) –

Returns:

data

Return type:

numpy.ndarray

read_narrative_sets()

Read narrative sets from project configuration

Returns:A list of narrative set dicts Return type:list

read_narrative_set(narrative_set_name)

Read all narratives from a certain narrative_set

Parameters:narrative_set_name (str) – Name of the narrative_set Returns:A narrative_set dictionary Return type:list

write_narrative_set(narrative_set)

Write narrative_set to project configuration

Parameters:narrative_set (dict) – A narrative_set dict

update_narrative_set(narrative_set_name, narrative_set)

Update narrative_set to project configuration

Parameters:

• narrative_set_name (str) – Name of the (original) entry
• narrative_set (dict) – The updated narrative_set dict

delete_narrative_set(narrative_set_name)

Delete narrative_set from project configuration

Parameters:narrative_set_name (str) – A narrative_set name

read_narratives()

Read narrative sets from project configuration

Returns:A list of narrative set dicts Return type:list

read_narrative(narrative_name)

Read all narratives from a certain narrative

Parameters:narrative_name (str) – Name of the narrative Returns:A narrative dictionary Return type:list

write_narrative(narrative)

Write narrative to project configuration

Parameters:narrative (dict) – A narrative dict

update_narrative(narrative_name, narrative)

Update narrative to project configuration

Parameters:

• narrative_name (str) – Name of the (original) entry
• narrative (dict) – The updated narrative dict

delete_narrative(narrative_name)

Delete narrative from project configuration

Parameters:narrative_name (str) – A narrative name

read_narrative_data(narrative_name)

Read narrative data file

Parameters:narrative_name (str) – Name of the narrative Returns:A list with dictionaries containing the contents of ‘narrative_name’ data file Return type:list

read_narrative_definition(narrative_name)

Read the narrative definition

Parameters:narrative_name (str) – Name of the narrative Returns: Return type:dict

read_coefficients(source_name, destination_name)

Reads coefficients from file on disk

Coefficients are uniquely identified by their source/destination names

write_coefficients(source_name, destination_name, data)

Writes coefficients to file on disk

Coefficients are uniquely identified by their source/destination names

read_results(modelrun_id, model_name, output_name, spatial_resolution, temporal_resolution, timestep=None, modelset_iteration=None, decision_iteration=None)

Return path to text file for a given output

Parameters:

• modelrun_id (str) –
• model_name (str) –
• output_name (str) –
• spatial_resolution (str) –
• temporal_resolution (str) –
• timestep (int, optional) –
• modelset_iteration (int, optional) –
• decision_iteration (int, optional) –

Returns:

data

Return type:

numpy.ndarray

write_results(modelrun_id, model_name, output_name, data, spatial_resolution, temporal_resolution, timestep=None, modelset_iteration=None, decision_iteration=None)

Return path to text file for a given output

Parameters:

• modelrun_id (str) –
• model_name (str) –
• output_name (str) –
• data (numpy.ndarray) –
• spatial_resolution (str) –
• temporal_resolution (str) –
• timestep (int, optional) –
• modelset_iteration (int, optional) –
• decision_iteration (int, optional) –

prepare_warm_start(modelrun_id)

Copy the results from the previous modelrun if available

Parameters:modelrun_id (str) – Returns:num Return type:The timestep where the data store was recovered to