ramble.language namespace

Submodules

ramble.language.application_language module

class ramble.language.application_language.ApplicationMeta(name, bases, attr_dict)

Bases: SharedMeta

ramble.language.application_language.cleanup(name, regex, directory=None, recurse=False, description='', pre=False, post=False, when=None, **kwargs)

Adds a cleanup operation to the application.

This directive defines a cleanup step that removes files matching a regular expression from a specified directory.

Parameters:

• name (str) – Name of the cleanup operation.

• regex (str) – Regex passed to find to match files and directories to be deleted.

• directory (str) – The directory to perform the cleanup in. Defaults to {experiment_run_dir}.

• recurse (bool) – Whether to search for files recursively in subdirectories.

• description (str) – Description of the cleanup operation.

• pre (bool) – Whether to run this cleanup before the main application execution.

• post (bool) – Whether to run this cleanup after the main application execution.

• when (list | None) – List of when conditions to apply to this directive.

ramble.language.application_language.executable(name, template, when=None, **kwargs)

Adds an executable to this application

Defines a new executable that can be used to configure workloads and experiments with.

Executables may or may not use MPI.

Required Args:

name (str): Name of the executable template (list[str] | str): The template command this executable should generate from

Optional Args:

use_mpi or mpi (bool): determines if this executable should be

wrapped with an mpirun like command or not.

variables (dict): Dictionary of variable definitions to use for this

executable only

redirect (str): Optional, sets the path for outputs to be written to.

defaults to {log_file}

output_capture (str): Optional, Declare which output (stdout, stderr,

both) to capture. Defaults to stdout

run_in_background (bool): Optional, Declare if the command should

run in the background. Defaults to False

when (list | None): List of when conditions to apply to directive

ramble.language.application_language.input_file(name, url, description, target_dir='{workload_input_dir}', sha256=None, extension=None, expand=True, when=None, **kwargs)

Adds an input file definition to this application

Defines a new input file. An input file must define it’s name, and a url where the input can be fetched from.

Parameters:

• url (str) – Path to the input file / archive

• description (str) – Description of this input file

• target_dir (str) – Optional, the directory where the archive will be expanded. Defaults to the ‘{workload_input_dir}’ + os.sep + ‘{input_name}’

• sha256 (str) – Optional, the expected sha256 checksum for the input file

• extension (str) – Optiona, the extension to use for the input, if it isn’t part of the file name.

• expand (bool) – Optional. Whether the input should be expanded or not. Defaults to True

• when (list | None) – List of when conditions to apply to directive

ramble.language.application_language.license_name(name, **kwargs)

Add a new license name directive, to specify license name in a declarative way.

Parameters:

name (str) – name to use during license lookup and propagation

ramble.language.application_language.stage_files(src=None, dst=None, stages=None, name=None, method='user-defined', when=None, **kwargs)

Adds an executable that stages an input file or directory.

Defines a new executable that copies or links a file or directory from a source to a destination. This is useful for staging input files that are not managed by the input_file directive.

The staging method is controlled by the stage_method configuration option, which can be set to ‘cp’, ‘rsync’, ‘symbolic_link’, ‘hard_link’, or ‘install’ (for files only).

Parameters:

• src (str | None) – The source path of the file or directory.

• dst (str | None) – The destination path. If src is passed in, and dst is not, dst defaults to the experiment_run_dir.

• stages (list(tuple(str, str)) | None) – A list of tuples describing pairs of src, dest locations to stage.

• name (str | None) – The name of the executable. Defaults to ‘stage-files’.

• method (str) – The method to use for this stage. Can be one of: “user-defined”, “cp”, “rsync”, “symbolic_link”, “hard_link”, “install”

• when (list | None) – List of when conditions to apply to this directive.

ramble.language.application_language.workload(name, executables=None, executable=None, input=None, inputs=None, tags=None, when=None, **kwargs)

Adds a workload to this application

Defines a new workload that can be used within the context of its application.

Parameters:

• executable (str) – The name of an executable to be used

• executables (str) – A list of executable names to be used

• input (str) – Optional, name of an input be used

• inputs (str) – Optional, A list of input names that will be used

One of executable, or executables is required as an input argument.

ramble.language.application_language.workload_group(name, workloads=None, mode=None, when=None, **kwargs)

Adds a workload group to this application

Defines a new workload group that can be used within the context of its application.

Parameters:

• name (str) – The name of the group

• workloads (list(str) | None) – A list of workloads to be grouped

ramble.language.application_language.workload_variable(name, default=None, description='', values=None, strict: bool = True, workload=None, workloads=None, workload_group=None, workload_defaults=None, expandable: bool = True, track_used: bool = True, when=None, environment_variable_name=None, **kwargs)

Define a new variable to be used in experiments

Defines a new variable that can be defined within the experiments.yaml config file, to control various aspects of an experiment.

These are specific to each workload.

Parameters:

• name (str) – Name of variable to define

• default – Default value of variable definition

• description (str) – Description of variable’s purpose

• values (list) – Optional list of suggested values for this variable

• strict (bool) – If True (the default) and values is not None, the variable’s value will be validated against the values list.

• workload (str) – Single workload this variable is used in

• workloads (list) – List of modes this variable is used in

• workload_group (str) – Name of workload group this variable is used in.

• workload_defaults (dict) – Dictionary mapping workload names to default values. Mututally exclusive with workload, workloads, workload_group, and default.

• expandable (bool) – True if the variable should be expanded, False if not.

• track_used (bool) – True if the variable should be tracked as used, False if not. Can help with allowing lists without vectorizing experiments.

• when (list | None) – List of when conditions to apply to directive

• environment_variable_name (str | None) – If not None, an environment variable of this name will be defined with the value of this variable.

ramble.language.language_base module

This package contains the underlying implementation for the language directives, which are to allow functions to be invoked at class level

exception ramble.language.language_base.DirectiveError(message: str, long_message: str | None = None)

Bases: RambleError

This is raised when something is wrong with a language directive.

class ramble.language.language_base.DirectiveMeta(name, bases, attr_dict)

Bases: ABCMeta

Flushes the directives that were temporarily stored in the staging area into the package.

classmethod directive(dicts=None, init_value=None)

Decorator for Ramble directives.

Ramble directives allow you to modify an object while it is being defined, e.g. to add version or dependency information. Directives are one of the key pieces of Ramble’s object “language”, which is embedded in python.

Here’s an example directive:

@directive(dicts='workloads')
workload('workload_name', ...):
    ...

This directive allows you write:

class Foo(ApplicationBase):
    workload(...)

The @directive decorator handles a couple things for you:

1. Adds the class scope (app) as an initial parameter when called, like a class method would. This allows you to modify a package from within a directive, while the package is still being defined.

2. It automatically adds a dictionary called “workloads” to the package so that you can refer to app.workloads.

The (dicts='workloads') part ensures that ALL applications in Ramble will have a workloads attribute after they’re constructed, and that if no directive actually modified it, it will just be an empty dict.

The (init_value={}) part allows objects in Ramble to define what the type of the attribute defined by the dicts argument will be. This allows (dicts="variables", init_value=[]) which makes the attribute a list instead of a dict, which is the default.

This is just a modular way to add storage attributes to the Application class, and it’s how Ramble gets information from the applications to the core.

pop_default_args() → dict

pop_from_context() → str

push_default_args() → None

push_to_context() → None

ramble.language.language_helpers module

ramble.language.language_helpers.add_variable_validator(obj, var_name, var_values, when_list, wl_name=None)

Adds a validator to an object to ensure a variable’s value is in a list of values.

ramble.language.language_helpers.are_when_compatible(when_set1, when_set2)

Determine if two sets of when conditions are compatible

Parameters:

• when_set1 (list) – First set of when conditions

• when_set2 (list) – Second set of when conditions

Returns:

True if they are compatible, False otherwise

Return type:

(bool)

ramble.language.language_helpers.build_when_list(when_arg: str | List[str] | None, obj: Any, directive_id: str, directive_name: str) → List[str]

Construct list of when conditions based on a directives input argument Also, validate that when is passed in with the right type.

Parameters:

• when_arg (str | list(str)) – Single or list of string conditions that were input into the calling directive.

• obj – A ramble object (i.e. application, modifier, etc..)

• directive_id (str) – Directive identifier. The calling directive can define what is used here, but it should be something that can help users identify where errors from this method originate from.

• directive_name (str) – Name of the calling directive

Returns:

List of strings, for all of the when conditions.

ramble.language.language_helpers.check_definition(single_type, multiple_type, single_arg_name, multiple_arg_name, directive_name)

Sanity check definitions before merging or require

Parameters:

• single_type – Single string for type name

• multiple_type – List of strings for type names, may contain wildcards

• multiple_pattern_match – List of strings to match against patterns in multiple_type

• single_arg_name – String name of the single_type argument in the directive

• multiple_arg_name – String name of the multiple_type argument in the directive

• directive_name – Name of the directive requiring a type

Returns:

List of all type names (Merged if both single_type and multiple_type definitions are valid)

ramble.language.language_helpers.expand_patterns(merged_types: list, multiple_pattern_match: list | dict)

Expand wildcard patterns within a list of names

This method takes an input list containing wildcard patterns and expands the wildcard with values matching a list of names. Returns a list containing matching names and any inputs with zero matches.

If multiple_pattern_match is a dict keyed on ‘when’, it checks the input against patterns in all ‘when’ conditions, without evaluating them, and returns a list containing names that match under any when condition, and any inputs with zero matches.

Parameters:

• merged_types – List of strings for type names, may contain wildcards

• multiple_pattern_match – List of strings (optional: nested in when_set dict) to match against patterns in merged_types

Returns:

List of expanded patterns matching the names list plus patterns

not found in the names list.

ramble.language.language_helpers.is_when_impossible(when_list)

Determine if a single list of when conditions is self-contradictory

Parameters:

when_list (list) – list of when conditions

Returns:

True and a message if it is impossible, False and None otherwise

Return type:

(bool, str)

ramble.language.language_helpers.merge_conditions(obj, directive_name: str, single_arg_name: str | None = None, multiple_arg_name: str | None = None, **kwargs) → List[List[str]]

Merge conditions for a type in a directive, and converts all conditions to when conditions

If single/multiple values are provided, this method will validate that they are properly defined, and will merge them into a list of when conditions for each mode.

Parameters:

• obj – Object instance

• directive_name – Name of the calling directive

• single_arg_name – Name of the singular kwarg being required

• multiple_arg_name – Name of the plural kwarg being required

Kwargs:

when (list | None): List of when conditions to apply to directive mode (str | None): Modifier mode to be applied as a when condition modes (list(str) | None): List of modifier modes to be applied as when conditions

Returns:

List of lists of strings, where each inner list is a list of when conditions for a mode.

ramble.language.language_helpers.merge_definitions(single_type, multiple_type, multiple_pattern_match, single_arg_name, multiple_arg_name, directive_name)

Merge definitions of a type

This method will merge two optional definitions of single_type and multiple_type.

Parameters:

• single_type – Single string for type name

• multiple_type – List of strings for type names, may contain wildcards

• multiple_pattern_match – List of strings to match against patterns in multiple_type

• single_arg_name – String name of the single_type argument in the directive

• multiple_arg_name – String name of the multiple_type argument in the directive

• directive_name – Name of the directive requiring a type

Returns:

List of all type names (Merged if both single_type and multiple_type definitions are valid)

ramble.language.language_helpers.require_definition(single_type, multiple_type, multiple_pattern_match, single_arg_name, multiple_arg_name, directive_name)

Require at least one definition for a type in a directive

This method will validate that single_type / multiple_type are properly defined.

It will raise an error if at least one type is not defined, or if either are the incorrect type.

Parameters:

• single_type – Single string for type name

• multiple_type – List of strings for type names, may contain wildcards

• multiple_pattern_match – List of strings to match against patterns in multiple_type

• single_arg_name – String name of the single_type argument in the directive

• multiple_arg_name – String name of the multiple_type argument in the directive

• directive_name – Name of the directive requiring a type

Returns:

List of all type names (Merged if both single_type and multiple_type definitions are valid)

ramble.language.modifier_language module

class ramble.language.modifier_language.ModifierMeta(name, bases, attr_dict)

Bases: SharedMeta

ramble.language.modifier_language.default_mode(name, **kwargs)

Define a default mode for this modifier.

The default mode will be used if modifier mode is not specified in an experiment.

Parameters:

name (str) – Name of mode to be used as default

ramble.language.modifier_language.env_var_modification(name, modification=None, method='set', mode=None, modes=None, when=None, **kwargs)

Define an environment variable modifier

Environment variable modifications modify the values of environment variables within the application instance.

Parameters:

• name (str) – The name of the environment variable that will be modified

• modification (str) – The value of the modification

• method (str) – The method of the modification.

• mode (str | None) – Name of mode this env_var_modification should apply in

• modes (list(str) | None) – List of mode names this env_var_modification should apply in

• when (list | None) – List of when conditions this env_var_modification should apply in

Supported values for method are:

• set: Defines the variable to equal the modification value

• unset: Removes any definition of the variable from the environment

• prepend: Prepends the modification to the beginning of the variable. Always uses the separator ‘:’

• append: Appends the modification value to the end of the value. Allows a keyword argument of ‘separator’ to define the delimiter between values.

ramble.language.modifier_language.executable_modifier(name, usage_filter=None, when=None, **kwargs)

Register an executable modifier

Executable modifiers can modify various aspects of non-builtin application executable definitions.

These behave similarly to builtins, in that a python method defines the actual modifications

For example:

executable_modifier('write_exec_name')

def write_exec_name(self, executable_name, executable, app_inst=None):
  prepend_execs = []
  append_execs = [ExecutableCommand(
      template='echo "{executable_name}"',
      mpi=False,
      redirect='{log_file}',
      output_capture=OUTPUT_CAPTURE.DEFAULT
  )]

  return prepend_execs, append_execs

Would append the echo “{executable_name}” to every non-builtin executable in an experiment.

Executable modifiers are allowed to modify the input executable in place. Executable modifiers must return two lists of executables.

Parameters:

• name (str) – Name of executable modifier to use. Should be the name of a class method.

• usage_filter (str) – Filters the application of this executable modifier. Modifiers can register filters to select how to apply this. Valid default options include: None, “once”, “first_mpi”, “all_mpi”

• when (list | None) – List of when conditions this executable modifier should apply in

Each executable modifier needs to return:

prepend_execs (list(CommandExecutable)): List of executables to inject

before the base executable

append_execs (list(CommandExecutable)): List of executables to inject

after the base executable

ramble.language.modifier_language.mode(name, description, **kwargs)

Define a new mode for this modifier.

Modes allow a modifier to bundle a set of modifications together.

NOTE: A mode ‘disabled’ is defined by default for all modifiers.

Parameters:

• name (str) – Name of the mode to define

• description (str) – Description of the new mode

ramble.language.modifier_language.modifier_conflict(conflict_type, when=None, **kwargs)

Define a conflict with other modifiers on the same experiment.

Allowed values are defined in the MODIFIER_CONFLICT class in conflicts.py

Parameters:

conflict_type – Either a string or integer based on the options in ramble.util.conflicts.MODIFIER_CONFLICT

ramble.language.modifier_language.modifier_variable(name: str, default, description: str, values: list | None = None, mode: str | None = None, modes: list | None = None, expandable: bool = True, track_used: bool = False, when=None, **kwargs)

Define a variable for this modifier

Parameters:

• name (str) – Name of variable to define

• default – Default value of variable definition

• description (str) – Description of variable’s purpose

• values (list) – Optional list of suggested values for this variable

• mode (str) – Single mode this variable is used in, if no mode/modes are specified, will apply to all modes.

• modes (list) – List of modes this variable is used in, if no mode/modes are specified, will apply to all modes.

• expandable (bool) – True if the variable should be expanded, False if not.

• track_used (bool) – True if the variable should be tracked as used, False if not. Can help with allowing lists without vectorizing experiments.

• when (list | None) – List of when conditions to apply to directive

ramble.language.modifier_language.package_manager_requirement(command: str, validation_type: str, mode: str | None = None, modes: list | None = None, regex=None, package_manager: str = '*', when=None, **kwargs)

Define a requirement when this modifier is used in an experiment with a package manager.

This allows modifiers to inject additional requirements on packages managers. These can be used to ensure package manager commands return specific values.

Parameters:

• command (str) – Package manager command to execute, when evaluating the requirement

• validation_type (str) – Type of validation to perform on output of command. Valid types are: ‘empty’, ‘not_empty’, ‘contains_regex’, ‘does_not_contain_regex’

• mode (str) – Usage mode this requirement should apply to

• modes (list(str)) – List of usage modes this requirement should apply to

• regex (str) – Regular expression to use when validation_type is either ‘contains_regex’ or ‘does_no_contain_regex’

• package_manager (str) – Name of the package manager this requirement applies to

• when (list | None) – List of when conditions this requirement should apply to

ramble.language.modifier_language.variable_modification(name, modification, method='set', mode=None, modes=None, separator=' ', when=None, **kwargs)

Define a new variable modification for a mode in this modifier.

A variable modification will apply a change to a defined variable within an experiment.

Parameters:

• name (str) – The variable to modify

• modification (str) – The value to modify ‘name’ with

• method (str) – How the modification should be applied

• mode (str) – Single mode to group this modification into

• modes (str) – List of modes to group this modification into

• separator (str) – Optional separator to use when modifying with ‘append’ or ‘prepend’ methods.

• when (list | None) – List of when conditions this modification should apply in

Supported values are ‘append’, ‘prepend’, and ‘set’:

‘append’ will add the modification to the end of ‘name’ ‘prepend’ will add the modification to the beginning of ‘name’ ‘set’ (Default) will overwrite ‘name’ with the modification

ramble.language.package_manager_language module

class ramble.language.package_manager_language.PackageManagerMeta(name, bases, attr_dict)

Bases: SharedMeta

ramble.language.package_manager_language.package_manager_family(*names: str, **kwargs)

Add a new family to this package manager

Parameters:

name (str) – Name of family to apply to this package manager

ramble.language.package_manager_language.package_manager_variable(name: str, default, description: str, values: list | None = None, expandable: bool = True, track_used: bool = False, when=None, **kwargs)

Define a variable for this package manager

Parameters:

• name (str) – Name of variable to define

• default – Default value of variable definition

• description (str) – Description of variable’s purpose

• values (list) – Optional list of suggested values for this variable

• expandable (bool) – True if the variable should be expanded, False if not.

• track_used (bool) – True if the variable should be tracked as used, False if not. Can help with allowing lists without vectorizing

• when (list | None) – List of when conditions to apply to directive

ramble.language.platform_language module

class ramble.language.platform_language.PlatformMeta(name, bases, attr_dict)

Bases: SharedMeta

ramble.language.platform_language.platform_family(*names: str, **kwargs)

Add a new family to this platform

Parameters:

name (str) – Name of family to apply to this platform

ramble.language.shared_language module

class ramble.language.shared_language.SharedMeta(name, bases, attr_dict)

Bases: DirectiveMeta

ramble.language.shared_language.archive_pattern(pattern, **kwargs)

Adds a file pattern to be archived in addition to figure of merit logs

Defines a new file pattern that will be archived during workspace archival. Archival will only happen for files that match the pattern when archival is being performed.

Parameters:

pattern (str) – Pattern that refers to files to archive

ramble.language.shared_language.auxiliary_software_file(name, src_path, dest_path, when=None, **kwargs)

Defines an auxiliary software file

Parameters:

• name (str) – Name of the auxiliary file

• src_path (str) – Source path of the auxiliary file

• dest_path (str) – Destination path of the auxiliary file

ramble.language.shared_language.class_family(*names: str, **kwargs)

Add a new family to this object

Parameters:

names (str) – Name of family to apply to this object

ramble.language.shared_language.command_variable(name, command, dry_run_value, description='', expandable=True, track_used=True, when=None, **kwargs)

Defines an variable, backed by a command

Parameters:

• name (str) – Name of the variable

• command (str) – Command to execute to get the variable value

• dry_run_value (str) – Value to use when part of a dry-run

• description (str) – Description of variable

ramble.language.shared_language.conflict(conflict_spec: str, when: str | List[str] | None = None, msg: str | None = None, **kwargs)

Defines a conflict for this object.

Parameters:

• conflict_spec – The trigger condition (e.g., +variant, compiler=gcc)

• when – Optional conditional under which the conflict occurs

• msg – Optional custom error/warning message

ramble.language.shared_language.default_args(**kwargs)

ramble.language.shared_language.define_compiler(name, pkg_spec, compiler_spec=None, compiler=None, package_manager=None, inject_if_missing=False, when=None, **kwargs)

Defines the compiler that will be used with this object

Adds a new compiler spec to this object. Software specs should reference a compiler that has been added.

Parameters:

• name (str) – Name of compiler package

• pkg_spec (str) – Package spec to install compiler

• compiler_spec (str) – Compiler spec (if different from pkg_spec)

• compiler (str) – Package name to use for compilation

• package_manager (str) – Glob supported pattern to match package managers this compiler applies to

• inject_if_missing (bool) – Whether the package should be defined if a matching package is not already defined

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.environment_variable(name, value, description, method='set', append_separator=',', workload=None, workloads=None, workload_group=None, when=None, **kwargs)

Define an environment variable to be used in experiments. Workload args are only applicable to application definitions.

Parameters:

• name (str) – Name of environment variable to define

• value (str) – Value to set env-var to

• description (str) – Description of the env-var

• method (str) – The method to use when defining the env-var. Can be “set”, “append”, or “prepend”

• workload (str) – Name of app workload this env-var should be added to

• workloads (list(str)) – List of app workload names this env-var should be added to

• workload_group (str) – Name of app workload group this env-var should be added to

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.figure_of_merit(name, fom_regex=None, group_name=None, log_file='{log_file}', units='', contexts=None, fom_type: FomType = FomType.UNDEFINED, when=None, fom_map_key=None, **kwargs)

Adds a figure of merit to track for this object

Defines a new figure of merit.

Parameters:

• name (str) – High level name of the figure of merit

• log_file (str) – File the figure of merit can be extracted from

• fom_regex (str) – A regular expression using named groups to extract the FOM

• group_name (str) – The name of the group that the FOM should be pulled from

• units (str) – The units associated with the FOM

• contexts (list(str) | None) – List of contexts (defined by figure_of_merit_context) this figure of merit should exist in.

• fom_type (ramble.util.foms.FomType) – The type of figure of merit

• when (list | None) – List of when conditions to apply to directive

• fom_map_key – If supplied, this is treated as an in-memory (as opposed to file-based) figure of merit, and its value is extracted using this key

ramble.language.shared_language.figure_of_merit_context(name, regex, output_format, when=None, **kwargs)

Defines a context for figures of merit

Defines a new context to contain figures of merit.

Parameters:

• name (str) – High level name of the context. Can be referred to in the figure of merit

• regex (str) – Regular expression, using group names, to match a context.

• output_format (str) – String, using python keywords {group_name} to extract group names from context regular expression.

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.formatted_executable(name: str, commands: list, prefix: str = '', indentation: int = 0, join_separator: str = '\n', when=None, **kwargs)

Define a new formatted execution for this object

Parameters:

• name – Name of the new formatted executable

• prefix – Prefix for each line of the formatted executable

• indentation – Number of spaces to indent before the prefix of each line

• join_separator – String to use when separating the commands during formatting

• commands – List of commands to expand when generating the formatted executable

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.maintainers(*names: str, **kwargs)

Add a new maintainer directive, to specify maintainers in a declarative way.

Parameters:

names (str) – GitHub username(s) for the maintainer. Can provide multiple names as separate arguments.

ramble.language.shared_language.package_manager_config(name, config, package_manager=None, when=None, **kwargs)

Defines a config option to set within a package manager

Define a new config which will be passed to a package manager. The resulting experiment instance will pass the config to the package manager, which will control the logic of applying it.

Parameters:

• name (str) – Name of this configuration

• config (str) – Configuration option to set

• package_manager (str) – Name of the package manager this config should be used with

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.register_builtin(name, required=True, injection_method='prepend', depends_on=None, dependents=None, when=None, **kwargs)

Register a builtin

Builtins are methods that return lists of strings. These methods represent a way to write python code to generate executables for building up workloads.

Manual injection of a builtins can be performed through modifying the execution order in the internals config section.

Modifier builtins are named: modifier_builtin::modifier_name::method_name.

Application modifiers are named: builtin::method_name.

Package manager builtins are named: package_manager_builtin::package_manager_name::method_name.

As an example, if the following builtin was defined:

register_builtin('example_builtin', required=True)
def example_builtin(self):
  ...

Its fully qualified name would be: * modifier_builtin::test-modifier::example_builtin when defined in a modifier named test-modifier * builtin::example_builtin when defined in an application

The ‘required’ attribute marks a builtin as required for all workloads. This will ensure the builtin is added to the workload if it is not explicitly added. If required builtins are not explicitly added to a workload, they are injected into the list of executables, based on the injection_method attribute.

The ‘injection_method’ attribute controls where the builtin will be injected into the executable list. Options are: - ‘prepend’ – This builtin will be injected before the executables for the experiment - ‘append’ – This builtin will be injected after the executables for the experiment

NOTE: When specifying explicit dependencies, cycles can be created which will cause an error when trying to construct the final executable order. One possible way to resolve those issues is to make sure that builtins which depend on each other have the same injection method (or at least do not have conflicting injection methods). As an example, if builtin a has an injection method of prepend, and builtin b lists a as a dependent but has an injection method of append, then this will create a cycle. If b has it’s injection method updated to be prepend the cycle will be resolved.

Parameters:

• name (str) – Name of builtin (should be the name of a class method) to register

• required (bool) – Whether the builtin will be auto-injected or not

• injection_method (str) – The method of injecting the builtin. Can be ‘prepend’ or ‘append’

• depends_on (list(str) | None) – The names of builtins this builtin depends on (and must execute after).

• dependents (list(str) | None) – The names of builtins that should come after this builtin

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.register_phase(name, pipeline=None, run_before=None, run_after=None, when=None, **kwargs)

Register a phase

Phases are portions of a pipeline that will execute when executing a full pipeline.

Registering a phase allows an object to know what the phases dependencies are, to ensure the execution order is correct.

If called multiple times, the dependencies are combined together. Only one instance of a phase will show up in the resulting dependency list for a phase.

Parameters:

• name (str) – The name of the phase. Phases are functions named ‘_<phase>’.

• pipeline (str) – The name of the pipeline this phase should be registered into.

• run_before (list(str) | None) – A list of phase names this phase should run before

• run_after (list(str) | None) – A list of phase names this phase should run after

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.register_template(name: str, src_path: str, dest_path: str | None = None, define_var: bool = True, extra_vars: dict | None = None, extra_vars_func: str | None = None, output_perm=None, when: List[str] | None = None, **kwargs)

Directive to define an object-specific template to be rendered into experiment run_dir.

For instance, register_template(name=”foo”, src_path=”foo.tpl”, dest_path=”foo.sh”) expects a “foo.tpl” template defined alongside the object source, and uses that to render a file under “{experiment_run_dir}/foo.sh”. The rendered path can also be referenced with the foo variable name.

Parameters:

• name – The name of the template. It is also used as the variable name that an experiment can use to reference the rendered path, if define_var is true.

• src_path – The location of the template. It can either point to an absolute or a relative path. It knows how to resolve workspace paths such as $workspace_shared. A relative path is relative to the containing directory of the object source.

• dest_path – If present, the location of the rendered output. It can either point to an absolute or a relative path. It knows how to resolve workspace paths such as $workspace_shared. A relative path is relative to the experiment_run_dir. If not given, it will use the same name as the template (optionally drop the .tpl extension) and placed under experiment_run_dir.

• define_var – Controls if a variable named name should be defined.

• extra_vars – If present, the variable dict is used as extra variables to render the template.

• extra_vars_func – If present, the name of the function to call to return a dict of extra variables used to render the template. This option is combined together with and takes precedence over extra_vars, if both are present.

• output_perm – The chmod mask for the rendered output file.

• when – List of when conditions to apply to directive

ramble.language.shared_language.register_validator(name: str, predicate: str, message: str, fail_on_invalid: bool = True, when=None, **kwargs)

Directive to define a validator for the object.

Parameters:

• name – The name of the validator.

• predicate – The expression to be evaluated (expanded). If it expands to False, then the validation fails.

• message – The message given when the validation fails. This can contain Ramble variables.

• fail_on_invalid – When true, this fails the experiment setup, otherwise it logs a warning. The default is True.

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.required_package(name, package_manager=None, when=None, **kwargs)

Defines a new spack package that is required for this object to function properly.

Parameters:

• name (str) – Name of required package

• package_manager (str) – Glob package manager name to apply this required package to

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.required_variable(var: str, results_level='variable', description=None, mode=None, modes=None, when=None, **kwargs)

Mark a variable as being required by this modifier

Parameters:

• var (str) – Variable name to mark as required

• results_level (str) – ‘variable’ or ‘key’. If ‘key’, variable is promoted to a key within JSON or YAML formatted results.

• description (str | None) – Description of the required variable.

• mode (str | None) – mode that the required check should be applied to. The default None means apply to all modes.

• modes (list[str] | None) – modes that the required check should be applied to. The default None means apply to all modes.

• when (list | None) – List of when conditions to apply the required check.

ramble.language.shared_language.software_spec(name, pkg_spec, compiler_spec=None, compiler=None, package_manager=None, inject_if_missing=False, when=None, **kwargs)

Defines a new software spec needed for this object.

Adds a new software spec that this object needs to execute properly.

Specs can be described as an mpi spec, which means they will depend on the MPI library within the resulting spack environment.

Parameters:

• name (str) – Name of package

• pkg_spec (str) – Package spec to install package

• compiler_spec (str) – Spec to use if this package will be used as a compiler for another package

• compiler (str) – Package name to use as compiler for compiling this package

• package_manager (str) – Glob supported pattern to match package managers this package applies to

• inject_if_missing (bool) – Whether the package should be added to experiment environments automatically or not.

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.strict_versions(strict: bool = True, **kwargs)

Directive to specify if the object has strict versioning. If true, only known versions can be used in experiments.

Parameters:

strict (bool) – Whether strict versioning is enabled.

ramble.language.shared_language.success_criteria(name, mode, match=None, file='{log_file}', fom_name=None, fom_context='null', formula=None, anti_match=None, when=None, **kwargs)

Defines a success criteria used by experiments of this object

Adds a new success criteria to this object definition.

These will be checked during the analyze step to see if a job exited properly.

Parameters:

• name (str) – The name of this success criteria

• mode (str) – The type of success criteria that will be validated Valid values are: ‘string’, ‘application_function’, and ‘fom_comparison’

• match (str) – For mode=’string’. Value to check indicate success (if found, it would mark success)

• file (str) – For mode=’string’. File success criteria should be located in

• fom_name (str) – For mode=’fom_comparison’. Name of fom for a criteria. Accepts globbing.

• fom_context (str) – For mode=’fom_comparison’. Context the fom is contained in. Accepts globbing.

• formula (str) – For mode=’fom_comparison’. Formula to use to evaluate success. ‘{value}’ keyword is set as the value of the FOM.

• anti_match (str) – For mode=’string’. Value to check indicate failure. This setting and match are mutually exclusive.

• when (list | None) – List of when conditions to apply to directive

ramble.language.shared_language.tags(*values: str, **kwargs)

Add a new tag directive, to specify tags in a declarative way.

Parameters:

values (str) – Values to mark as a tag. Can provide multiple values as separate arguments.

ramble.language.shared_language.target_shells(shell_support_pattern=None, **kwargs)

Directive to specify supported shells.

If not specified, i.e., not directly specified or inherited from the base, then it assumes all shells are supported.

Parameters:

shell_support_pattern (str) – The glob pattern that is used to match with the configured shell

ramble.language.shared_language.variable(name: str, default, description: str, values: list | None = None, strict: bool = True, expandable: bool = True, track_used: bool = False, when=None, error_context='variable', environment_variable_name: str | None = None, **kwargs)

Define a variable for this modifier

Parameters:

• name (str) – Name of variable to define

• default – Default value of variable definition

• description (str) – Description of variable’s purpose

• values (list) – Optional list of suggested values for this variable

• strict (bool) – If True (the default) and values is not None, the variable’s value will be validated against the values list.

• expandable (bool) – True if the variable should be expanded, False if not.

• track_used (bool) – True if the variable should be tracked as used, False if not. Can help with allowing lists without vectorizing experiments.

• when (list | None) – List of when conditions to apply to directive

• environment_variable_name (str | None) – If not None, an environment variable of this name will be defined with the value of this variable.

ramble.language.shared_language.variant(name: str, default: Any | None = None, description: str = '', values: Sequence | Callable[[Any], bool] | None = None, **kwargs)

ramble.language.shared_language.version(number: str, description: str = '', preferred: bool = False, **kwargs)

Define a new version in the input object

Parameters:

• number – Version number (Python packaging version format)

• description – Description of this version

• preferred – Mark this version as preferred. Only one version can be preferred.

ramble.language.shared_language.when(condition)

ramble.language.system_language module

class ramble.language.system_language.SystemMeta(name, bases, attr_dict)

Bases: SharedMeta

ramble.language.system_language.available_platforms(platforms, **kwargs)

Sets the available platforms for this system

Parameters:

platforms (list) – List of available platforms

ramble.language.system_language.default_package_manager(name, **kwargs)

Sets the default package manager for this system

Parameters:

name (str) – The name of the default package manager

ramble.language.system_language.default_platform(name, **kwargs)

Sets the default platform for this system

Parameters:

name (str) – The name of the default platform

ramble.language.system_language.default_workflow_manager(name, **kwargs)

Sets the default workflow manager for this system

Parameters:

name (str) – The name of the default workflow manager

ramble.language.system_language.platform_variable_map(variable_name, var_map, **kwargs)

Defines a mapping of platform to variable values

Parameters:

• variable_name (str) – The name of the variable

• var_map (dict) – Mapping of platform name to variable value

ramble.language.system_language.system_family(*names: str, **kwargs)

Add a new family to this system

Parameters:

name (str) – Name of family to apply to this system

ramble.language.system_language.variable_defaults(variable_definitions, when=None, **kwargs)

Defines default values for variables

Parameters:

• variable_definitions (dict) – Mapping of variable name to value

• when (list | None) – List of when conditions to apply to directive

ramble.language.workflow_manager_language module

class ramble.language.workflow_manager_language.WorkflowManagerMeta(name, bases, attr_dict)

Bases: SharedMeta

ramble.language.workflow_manager_language.workflow_manager_family(*names: str, **kwargs)

Add a new family to this workflow manager

Parameters:

name (str) – Name of family to apply to this workflow manager

ramble.language.workflow_manager_language.workflow_manager_variable(name: str, default, description: str, values: list | None = None, expandable: bool = True, track_used: bool = False, when=None, **kwargs)

Define a variable for this wm :param name: Name of variable :param default: Default value if the variable is not defined :param description: Description of the variable :param values: Optional list of suggested values for this variable :param expandable: True if the variable should be expanded, False if not. :type expandable: bool :param track_used: True if the variable should be tracked as used,

False if not. Can help with allowing lists without vectorizing

Parameters:

when (list | None) – List of when conditions to apply to directive