smif.controller.run package

Provides all the model run schedulers.

class smif.controller.run.DAFNIRunScheduler(username, password)

Bases: object

The scheduler can run instances of smif as a subprocess and can provide information whether the modelrun is running, is done or has failed.

add(model_run_name, args)

Add a model_run to the Modelrun scheduler.

Parameters:

• model_run_name (str) – Name of the modelrun

• args (dict) – Arguments for the command-line interface

• Exception

• ---------

• Exception – When the modelrun was already started

Notes

DAFNI’s queuing mechanism starts model runs in separate container. This means that it is possible to run multiple modelruns concurrently. This will not cause conflicts.

get_dict_from_json(file_path)

Given a JSON file, will return the dictionary of values within that file. Is developed to wait for a file to exist (for the case of reading vault secrets) to avoid the inherent race condition.

get_scheduler_type()

get_status(model_run_name)

get_yamls(model_run_name, args)

kill(model_run_name)

class smif.controller.run.SubProcessRunScheduler

Bases: object

The scheduler can run instances of smif as a subprocess and can provide information whether the modelrun is running, is done or has failed.

add(model_run_name, args)

Add a model_run to the Modelrun scheduler.

Parameters:

• model_run_name (str) – Name of the modelrun

• args (dict) – Arguments for the command-line interface

• Exception

• ---------

• Exception – When the modelrun was already started

Notes

There is no queuing mechanism implemented, each add will directly start a subprocess. This means that it is possible to run multiple modelruns concurrently. This may cause conflicts, it depends on the implementation whether a certain sector model / wrapper touches the filesystem or other shared resources.

get_scheduler_type()

get_status(model_run_name)

Get the status from the Modelrun scheduler.

Parameters:

model_run_name (str) – Name of the modelrun

Returns:

• dict (A message containing the status, command-line)

• output and error that can be directly sent back over

• the http api.

Notes

Possible status:

unstarted:

Model run was not started

queing:

Model run is waiting to be executed

running:

Model run is running

stopped:

Model run was stopped (killed) by user

done:

Model run was completed succesfully

failed:

Model run completed running with an exit code

kill(model_run_name)

Kill a Modelrun that is already running

Parameters:

model_run_name (str) – Name of the modelrun

Submodules

smif.controller.run.dafni_run_scheduler module

Schedulers are used to run models.

The defaults provided allow model runs to be scheduled as subprocesses, or individual models to be called in series.

Future implementations may interface with common schedulers to queue up models to run in parallel and/or distributed.

The DAFNIRunScheduler currently only works while connected to the RAL vpn.

Posts all of the model run information to the DAFNI API so that a DAFNI worker can start the model run.

class smif.controller.run.dafni_run_scheduler.DAFNIRunScheduler(username, password)

Bases: object

The scheduler can run instances of smif as a subprocess and can provide information whether the modelrun is running, is done or has failed.

add(model_run_name, args)

Add a model_run to the Modelrun scheduler.

Parameters:

• model_run_name (str) – Name of the modelrun

• args (dict) – Arguments for the command-line interface

• Exception

• ---------

• Exception – When the modelrun was already started

Notes

DAFNI’s queuing mechanism starts model runs in separate container. This means that it is possible to run multiple modelruns concurrently. This will not cause conflicts.

get_dict_from_json(file_path)

Given a JSON file, will return the dictionary of values within that file. Is developed to wait for a file to exist (for the case of reading vault secrets) to avoid the inherent race condition.

get_scheduler_type()

get_status(model_run_name)

get_yamls(model_run_name, args)

kill(model_run_name)

smif.controller.run.subprocess_run_scheduler module

Schedulers are used to run models.

The defaults provided allow model runs to be scheduled as subprocesses, or individual models to be called in series.

Future implementations may interface with common schedulers to queue up models to run in parallel and/or distributed.

Calls smif run … on the selected model in a sub process, where … are the options set in the app for info/debug messages, warm start and output format.

class smif.controller.run.subprocess_run_scheduler.SubProcessRunScheduler

Bases: object

The scheduler can run instances of smif as a subprocess and can provide information whether the modelrun is running, is done or has failed.

add(model_run_name, args)

Add a model_run to the Modelrun scheduler.

Parameters:

• model_run_name (str) – Name of the modelrun

• args (dict) – Arguments for the command-line interface

• Exception

• ---------

• Exception – When the modelrun was already started

Notes

There is no queuing mechanism implemented, each add will directly start a subprocess. This means that it is possible to run multiple modelruns concurrently. This may cause conflicts, it depends on the implementation whether a certain sector model / wrapper touches the filesystem or other shared resources.

get_scheduler_type()

get_status(model_run_name)

Get the status from the Modelrun scheduler.

Parameters:

model_run_name (str) – Name of the modelrun

Returns:

• dict (A message containing the status, command-line)

• output and error that can be directly sent back over

• the http api.

Notes

Possible status:

unstarted:

Model run was not started

queing:

Model run is waiting to be executed

running:

Model run is running

stopped:

Model run was stopped (killed) by user

done:

Model run was completed succesfully

failed:

Model run completed running with an exit code

kill(model_run_name)

Kill a Modelrun that is already running

Parameters:

model_run_name (str) – Name of the modelrun