smif.data_layer.file.file_data_store module

File-backed data store

Summary

Data:

CSVDataStore CSV text file data store
FileDataStore Abstract file data store
ParquetDataStore Binary file data store

Reference

class smif.data_layer.file.file_data_store.FileDataStore(base_folder)[source]

Bases: smif.data_layer.abstract_data_store.DataStore

Abstract file data store

read_scenario_variant_data(key, spec, timestep=None, timesteps=None)[source]

Read scenario variant data array.

If a single timestep is specified, the spec MAY include ‘timestep’ as a dimension, which should match the timestep specified.

If multiple timesteps are specified, the spec MUST include ‘timestep’ as a dimension, which should match the timesteps specified.

If timestep and timesteps are None, read all available timesteps. Whether or not the spec includes ‘timestep’ as a dimension, the returned DataArray will include a ‘timestep’ dimension with all available timesteps included.

Parameters:
  • key (str) –
  • spec (Spec) –
  • timestep (int (optional)) – If set, read data for single timestep
  • timesteps (list[int] (optional)) – If set, read data for specified timesteps
Returns:

data_array
Return type:

DataArray
write_scenario_variant_data(key, data)[source]

Write data array

Parameters:
scenario_variant_data_exists(key)[source]

Test if scenario variant data exists

Parameters:key (str) –
Returns:
Return type:bool
read_narrative_variant_data(key, spec, timestep=None)[source]

Read data array

Parameters:
  • key (str) –
  • spec (Spec) –
  • timestep (int (optional)) – If None, read data for all timesteps
Returns:

data_array
Return type:

DataArray
write_narrative_variant_data(key, data)[source]

Write data array

Parameters:
narrative_variant_data_exists(key)[source]
read_model_parameter_default(key, spec)[source]

Read data array

Parameters:
Returns:

data_array
Return type:

DataArray
write_model_parameter_default(key, data)[source]

Read data array

Parameters:
Returns:

data_array
Return type:

DataArray
model_parameter_default_data_exists(key)[source]
get_timesteps_from_data(key, spec_dict)[source]
read_interventions(keys)[source]

Read interventions data for key

Parameters:key (str) –
Returns:A dict of intervention dictionaries containing intervention attributes keyed by intervention name
Return type:dict[str, dict]
write_interventions(key, interventions)[source]

Write interventions data for key

Parameters:
interventions_data_exists(key)[source]
read_strategy_interventions(strategy)[source]
write_strategy_interventions(strategy, data)[source]
strategy_data_exists(strategy)[source]
read_initial_conditions(keys)[source]

Read historical interventions for key

Parameters:key (str) –
Returns:
Return type:list[dict]
write_initial_conditions(key, initial_conditions)[source]

Write historical interventions for key

Parameters:
initial_conditions_data_exists(key)[source]
read_state(modelrun_name, timestep, decision_iteration=None)[source]

Read list of (name, build_year) for a given model_run, timestep, decision

Parameters:
  • model_run_name (str) –
  • timestep (int) –
  • decision_iteration (int, optional) –
Returns:
Return type:

list[dict]
write_state(state, modelrun_name, timestep=None, decision_iteration=None)[source]

State is a list of decisions with name and build_year.

State is output from the DecisionManager

Parameters:
  • state (list[dict]) –
  • model_run_name (str) –
  • timestep (int) –
  • decision_iteration (int, optional) –
read_coefficients(source_dim, destination_dim)[source]

Reads coefficients from the store

Coefficients are uniquely identified by their source/destination dimensions. This method and write_coefficients implement caching of conversion coefficients between a single pair of dimensions.

Parameters:
  • source_dim (str) – dimension name
  • destination_dim (str) – dimension name
Returns:
Return type:

numpy.ndarray

Notes

To be called from Adaptor implementations.

write_coefficients(source_dim, destination_dim, data)[source]

Writes coefficients to the store

Coefficients are uniquely identified by their source/destination dimensions. This method and read_coefficients implement caching of conversion coefficients between a single pair of dimensions.

Parameters:
  • source_dim (str) – dimension name
  • destination_dim (str) – dimension name
  • data (numpy.ndarray) –

Notes

To be called from Adaptor implementations.

read_results(modelrun_id, model_name, output_spec, timestep, decision_iteration=None)[source]

Return results of a model from a model_run for a given output at a timestep and decision iteration

Parameters:
  • model_run_id (str) –
  • model_name (str) –
  • output_spec (Spec) –
  • timestep (int, default=None) –
  • decision_iteration (int, default=None) –
Returns:
Return type:

DataArray
write_results(data_array, modelrun_id, model_name, timestep=None, decision_iteration=None)[source]

Write results of a model_name in model_run_name for a given output_name

Parameters:
  • data_array (DataArray) –
  • model_run_id (str) –
  • model_name (str) –
  • timestep (int, optional) –
  • decision_iteration (int, optional) –
delete_results(model_run_name, model_name, output_name, timestep=None, decision_iteration=None)[source]

Delete results for a single timestep/iteration of a model output in a model run

Parameters:
  • model_run_name (str) –
  • model_name (str) –
  • output_name (str) –
  • timestep (int, default=None) –
  • decision_iteration (int, default=None) –
available_results(modelrun_name)[source]

List available results for a given model run

See _get_results_path for path construction.

On the pattern of:
results/<modelrun_name>/<model_name>/ decision_<id>/ output_<output_name>_timestep_<timestep>.csv
class smif.data_layer.file.file_data_store.CSVDataStore(base_folder)[source]

Bases: smif.data_layer.file.file_data_store.FileDataStore

CSV text file data store

class smif.data_layer.file.file_data_store.ParquetDataStore(base_folder)[source]

Bases: smif.data_layer.file.file_data_store.FileDataStore

Binary file data store