ramble package

ramble.ramble_version = '0.6.0'

String containing Ramble version joined with .’s

ramble.ramble_version_info = (0, 6, 0)

major, minor, patch version for Ramble, in a tuple

Subpackages

Submodules

ramble.appkit module

appkit is a set of useful build tools and directives for applications.

Everything in this module is automatically imported into Ramble application files.

ramble.caches module

Caches used by Ramble to store data

class ramble.caches.MirrorCache(root)[source]

Bases: object

store(fetcher, relative_dest)[source]

Fetch and relocate the fetcher’s target into our mirror cache.

Symlink a human readable path in our mirror to the actual storage location.

ramble.caches.fetch_cache = <ramble.fetch_strategy.FsCache object>

Ramble’s local cache for downloaded source archives

ramble.caches.fetch_cache_location()[source]

Filesystem cache of downloaded archives.

This prevents Ramble from repeatedly fetch the same files when using them within multiple workspaces.

ramble.caches.misc_cache = <ramble.util.file_cache.FileCache object>

Ramble’s cache for small data

ramble.config module

This module implements Ramble’s configuration file handling.

This implements Ramble’s configuration system, which handles merging multiple scopes with different levels of precedence.

The scopes are:

  1. default

  2. system

  3. site

  4. user

Important functions in this module are:

get_config reads in YAML data for a particular scope and returns it. Callers can then modify the data and write it back with update_config.

When read in, Ramble validates configurations with jsonschemas. The schemas are in submodules of ramble.schema.

exception ramble.config.ConfigError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Superclass for all ramble config related errors.

exception ramble.config.ConfigFileError(message: str, long_message: str | None = None)[source]

Bases: ConfigError

Issue reading or accessing a configuration file.

exception ramble.config.ConfigFormatError(validation_error, data, filename=None, line=None)[source]

Bases: ConfigError

Raised when a configuration format does not match its schema.

class ramble.config.ConfigScope(name, path)[source]

Bases: object

This class represents a configuration scope.

A scope is one directory containing named configuration files. Each file is a config “section” (e.g., mirrors, compilers, etc).

clear()[source]

Empty cached config information.

get_section(section)[source]
get_section_filename(section)[source]
property is_platform_dependent
exception ramble.config.ConfigSectionError(message: str, long_message: str | None = None)[source]

Bases: ConfigError

Error for referring to a bad config section name in a configuration.

class ramble.config.Configuration(*scopes)[source]

Bases: object

A full Ramble configuration, from a hierarchy of config files.

This class makes it easy to add a new scope on top of an existing one.

clear_caches()[source]

Clears the caches for configuration files,

This will cause files to be re-read upon the next request.

property file_scopes

List of writable scopes with an associated file.

get(path, default=None, scope=None)[source]

Get a config section or a single value from one.

Accepts a path syntax that allows us to grab nested config map entries. Getting the ‘config’ section would look like:

ramble.config.get('config')

and the dirty section in the config scope would be:

ramble.config.get('config:dirty')

We use : as the separator, like YAML objects.

get_config(section, scope=None)[source]

Get configuration settings for a section.

If scope is None or not provided, return the merged contents of all of ramble’s configuration scopes. If scope is provided, return only the configuration as specified in that scope.

This off the top-level name from the YAML section. That is, for a YAML config file that looks like this:

config:
  install_tree: $ramble/opt/ramble
  module_roots:
    lmod:   $ramble/share/ramble/lmod

get_config('config') will return:

{ 'install_tree': '$ramble/opt/ramble',
  'module_roots: {
      'lmod': '$ramble/share/ramble/lmod'
  }
}
get_config_filename(scope, section)[source]

For some scope and section, get the name of the configuration file.

highest_precedence_non_platform_scope()[source]

Non-internal non-platform scope with highest precedence

Platform-specific scopes are of the form scope/platform

highest_precedence_scope()[source]

Non-internal scope with highest precedence.

matching_scopes(reg_expr)[source]

List of all scopes whose names match the provided regular expression.

For example, matching_scopes(r’^command’) will return all scopes whose names begin with command.

pop_scope()[source]

Remove the highest precedence scope and return it.

print_section(section, blame=False)[source]

Print a configuration to stdout.

push_scope(scope)[source]

Add a higher precedence scope to the Configuration.

remove_scope(scope_name)[source]

Remove scope by name; has no effect when scope_name does not exist

set(path, value, scope=None)[source]

Convenience function for setting single values in config files.

Accepts the path syntax described in get().

update_config(section, update_data, scope=None, force=False)[source]

Update the configuration file for a particular scope.

Overwrites contents of a section in a scope with update_data, then writes out the config file.

update_data should have the top-level section name stripped off (it will be re-added). Data itself can be a list, dict, or any other yaml-ish structure.

Configuration scopes that are still written in an old schema format will fail to update unless force is True.

Parameters:

  • section (str) – section of the configuration to be updated

  • update_data (dict) – data to be used for the update

  • scope (str) – scope to be updated

  • force (str) – force the update

class ramble.config.ImmutableConfigScope(name, path)[source]

Bases: ConfigScope

A configuration scope that cannot be written to.

This is used for ConfigScopes passed on the command line.

class ramble.config.InternalConfigScope(name, data=None)[source]

Bases: ConfigScope

An internal configuration scope that is not persisted to a file.

This is for Ramble internal use so that command-line options and config file settings are accessed the same way, and Ramble can easily override settings from files.

clear()[source]

Empty cached config information.

get_section(section)[source]

Just reads from an internal dictionary.

get_section_filename(section)[source]
class ramble.config.SingleFileScope(name, path, schema, yaml_path=None)[source]

Bases: ConfigScope

This class represents a configuration scope in a single YAML file.

get_section(section)[source]
get_section_filename(section)[source]
property is_platform_dependent
ramble.config.add(fullpath, scope=None)[source]

Add the given configuration to the specified config scope. Add accepts a path. If you want to add from a filename, use add_from_file

ramble.config.add_from_file(filename, scope=None)[source]

Add updates to a config from a filename

ramble.config.command_line_scopes: List[str] = []

configuration scopes added on the command line set by ramble.main.main().

ramble.config.config = <ramble.config.Configuration object>

This is the singleton configuration instance for ramble.

ramble.config.config_defaults = {'config': {'build_jobs': 2, 'build_stage': '$tempdir/ramble-stage', 'checksum': True, 'concretizer': 'clingo', 'connect_timeout': 10, 'debug': False, 'dirty': False, 'disable_passthrough': False, 'disable_progress_bar': False, 'enable_strict_versions': True, 'input_cache': '$ramble/var/ramble/cache', 'license_dir': '/home/docs/checkouts/readthedocs.org/user_builds/ramble-dev-dapomeroy/checkouts/latest/lib/ramble/docs/_ramble_root/etc/spack/licenses', 'n_repeats': '0', 'pip': {'install': {'flags': []}}, 'repeat_success_strict': True, 'report_dirs': '~/.ramble/reports', 'shell': 'bash', 'spack': {'concretize': {'flags': '--fresh'}, 'install': {'flags': '--fresh'}}, 'upload': {'push_failed': True}, 'verify_ssl': True, 'workspace_dirs': ['$ramble/var/ramble/workspaces']}}

Hard-coded default values for some key configuration options. This ensures that Ramble will still work even if config.yaml in the defaults scope is removed.

ramble.config.configuration_paths = (('defaults', '/home/docs/checkouts/readthedocs.org/user_builds/ramble-dev-dapomeroy/checkouts/latest/lib/ramble/docs/_ramble_root/etc/ramble/defaults'), ('system', '/etc/ramble'), ('site', '/home/docs/checkouts/readthedocs.org/user_builds/ramble-dev-dapomeroy/checkouts/latest/lib/ramble/docs/_ramble_root/etc/ramble'), ('user', '/home/docs/.ramble'))

Builtin paths to configuration files in ramble

ramble.config.default_list_scope()[source]

Return the config scope that is listed by default.

Commands that list configuration list all scopes (merged) by default.

ramble.config.default_modify_scope(section='config')[source]

Return the config scope that commands should modify by default.

Commands that modify configuration by default modify the highest priority scope.

Parameters:

section (bool) – Section for which to get the default scope. If this is not ‘experiments’, a general (non-workspace) scope is used.

ramble.config.ensure_latest_format_fn(section)[source]

Return a function that takes as input a dictionary read from a configuration file and update it to the latest format.

The function returns True if there was any update, False otherwise.

Parameters:

section (str) – section of the configuration e.g. “applications”, “config”, etc.

ramble.config.first_existing(dictionary, keys)[source]

Get the value of the first key in keys that is in the dictionary.

ramble.config.get(path, default=None, scope=None)[source]

Module-level wrapper for Configuration.get().

ramble.config.get_valid_type(path)[source]

Returns an instance of a type that will pass validation for path.

The instance is created by calling the constructor with no arguments. If multiple types will satisfy validation for data at the configuration path given, the priority order is list, dict, str, bool, int, float.

ramble.config.load_config(*args, **kwargs)[source]

Load YAML using Ramble’s custom Loader that warns on duplicate keys.

ramble.config.merge_yaml(dest, source)[source]

Merges source into dest; entries in source take precedence over dest.

This routine may modify dest and should be assigned to dest, in case dest was None to begin with, e.g.:

dest = merge_yaml(dest, source)

In the result, elements from lists from source will appear before elements of lists from dest. Likewise, when iterating over keys or items in merged OrderedDict objects, keys from source will appear before keys from dest.

Config file authors can optionally end any attribute in a dict with :: instead of :, and the key will override that of the parent instead of merging.

ramble.config.override(path_or_scope, value=None)[source]

Simple way to override config settings within a context.

Parameters:

  • path_or_scope (ConfigScope | str) – scope or single option to override

  • value (object | None) – value for the single option

Temporarily push a scope on the current configuration, then remove it after the context completes. If a single option is provided, create an internal config scope for it and push/pop that scope.

ramble.config.overrides_base_name = 'overrides-'

Base name for the (internal) overrides scope.

ramble.config.process_config_path(path)[source]
ramble.config.read_config_file(filename, schema=None)[source]

Read a YAML configuration file.

User can provide a schema for validation. If no schema is provided, we will infer the schema from the top-level key.

ramble.config.scopes()[source]

Convenience function to get list of configuration scopes.

ramble.config.scopes_metavar = '{defaults,system,site,user}'

metavar to use for commands that accept scopes this is shorter and more readable than listing all choices

ramble.config.section_schemas: Dict[str, Dict[str, Any]] = {'applications': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'applications': {'additionalProperties': {'additionalProperties': False, 'default': '{}', 'properties': {'chained_experiments': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'command': {'type': 'string'}, 'inherit_variables': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'name': {'type': 'string'}, 'order': {'type': 'string'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}}, 'type': 'object'}, 'type': 'array'}, 'env_vars': {'append': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'var-separator': {'type': 'string'}, 'vars': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'prepend': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'set': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'unset': {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}}, 'formatted_executables': {'additionalProperties': {'additionalProperties': False, 'default': {}, 'properties': {'commands': {'default': ['{unformatted_command}'], 'items': {'type': 'string'}, 'type': 'array'}, 'indentation': {'default': 0, 'type': 'number'}, 'join_separator': {'default': '\n', 'type': 'string'}, 'prefix': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'internals': {'additionalProperties': False, 'default': {}, 'properties': {'custom_executables': {'additionalProperties': {'default': {'force': False, 'output_capture': OUTPUT_CAPTURE.ALL, 'redirect': '{log_file}', 'template': [], 'use_mpi': False, 'variables': {}}, 'properties': {'force': {'type': 'boolean'}, 'redirect': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'template': {'anyOf': [{'type': 'string'}, {'type': 'number'}, {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}]}, 'use_mpi': {'type': 'boolean'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'executable_injection': {'default': [], 'items': {'additionalProperties': {'relative_to': {'type': 'string'}}, 'default': {}, 'properties': {'name': {'type': 'string'}, 'order': {'default': 'after', 'type': 'string'}}, 'type': 'object'}, 'type': 'array'}, 'executables': {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}}, 'type': 'object'}, 'modifiers': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'mode': {'type': 'string'}, 'name': {'type': 'string'}, 'on_executable': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'type': 'object'}, 'type': 'array'}, 'success_criteria': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'anti_match': {'default': None, 'type': 'string'}, 'file': {'default': None, 'type': 'string'}, 'fom_context': {'default': None, 'type': 'string'}, 'fom_name': {'default': None, 'type': 'string'}, 'formula': {'default': None, 'type': 'string'}, 'match': {'default': None, 'type': 'string'}, 'mode': {'type': 'string'}, 'name': {'type': 'string'}}, 'type': 'object'}, 'type': 'array'}, 'tables': {'default': [], 'items': {'additionalProperties': False, 'properties': {'autocolumns': {'items': {'additionalProperties': False, 'properties': {'context_name': {'type': 'string'}, 'figure_of_merit': {'type': 'string'}, 'figure_of_merit_origin_type': {'default': None, 'type': 'string'}, 'name': {'type': 'string'}, 'sort_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name', 'figure_of_merit'], 'type': 'object'}, 'type': 'array'}, 'columns': {'items': {'additionalProperties': False, 'properties': {'expression': {'default': None, 'type': 'string'}, 'figure_of_merit': {'default': None, 'type': 'string'}, 'figure_of_merit_context': {'default': None, 'type': 'string'}, 'figure_of_merit_origin_type': {'default': None, 'type': 'string'}, 'name': {'type': 'string'}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name'], 'type': 'object'}, 'type': 'array'}, 'group_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'group_method': {'default': 'max', 'type': 'string'}, 'name': {'type': 'string'}, 'sort_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'transpose': {'default': False, 'type': 'boolean'}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name'], 'type': 'object'}, 'type': 'array'}, 'tags': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'template': {'type': 'boolean'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}, 'variants': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}, 'workloads': {'additionalProperties': {'additionalProperties': False, 'default': {}, 'properties': {'chained_experiments': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'command': {'type': 'string'}, 'inherit_variables': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'name': {'type': 'string'}, 'order': {'type': 'string'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}}, 'type': 'object'}, 'type': 'array'}, 'env_vars': {'append': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'var-separator': {'type': 'string'}, 'vars': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'prepend': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'set': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'unset': {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}}, 'experiments': {'additionalProperties': {'additionalProperties': False, 'default': {}, 'properties': {'chained_experiments': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'command': {'type': 'string'}, 'inherit_variables': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'name': {'type': 'string'}, 'order': {'type': 'string'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}}, 'type': 'object'}, 'type': 'array'}, 'env_vars': {'append': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'var-separator': {'type': 'string'}, 'vars': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'prepend': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'set': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'unset': {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}}, 'exclude': {'additionalProperties': False, 'default': {}, 'properties': {'matrices': {'default': [], 'items': {'anyOf': [{'default': [], 'items': {'type': 'string'}, 'type': 'array'}, {'additionalProperties': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'default': {}, 'properties': {}, 'type': 'object'}]}, 'type': 'array'}, 'matrix': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}, 'where': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'zips': {'additionalProperties': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'default': {}, 'properties': {}, 'type': 'object'}}, 'type': 'object'}, 'formatted_executables': {'additionalProperties': {'additionalProperties': False, 'default': {}, 'properties': {'commands': {'default': ['{unformatted_command}'], 'items': {'type': 'string'}, 'type': 'array'}, 'indentation': {'default': 0, 'type': 'number'}, 'join_separator': {'default': '\n', 'type': 'string'}, 'prefix': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'internals': {'additionalProperties': False, 'default': {}, 'properties': {'custom_executables': {'additionalProperties': {'default': {'force': False, 'output_capture': OUTPUT_CAPTURE.ALL, 'redirect': '{log_file}', 'template': [], 'use_mpi': False, 'variables': {}}, 'properties': {'force': {'type': 'boolean'}, 'redirect': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'template': {'anyOf': [{'type': 'string'}, {'type': 'number'}, {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}]}, 'use_mpi': {'type': 'boolean'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'executable_injection': {'default': [], 'items': {'additionalProperties': {'relative_to': {'type': 'string'}}, 'default': {}, 'properties': {'name': {'type': 'string'}, 'order': {'default': 'after', 'type': 'string'}}, 'type': 'object'}, 'type': 'array'}, 'executables': {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}}, 'type': 'object'}, 'matrices': {'default': [], 'items': {'anyOf': [{'default': [], 'items': {'type': 'string'}, 'type': 'array'}, {'additionalProperties': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'default': {}, 'properties': {}, 'type': 'object'}]}, 'type': 'array'}, 'matrix': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'modifiers': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'mode': {'type': 'string'}, 'name': {'type': 'string'}, 'on_executable': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'type': 'object'}, 'type': 'array'}, 'n_repeats': {'anyOf': [{'type': 'string'}, {'type': 'number'}], 'default': 0}, 'success_criteria': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'anti_match': {'default': None, 'type': 'string'}, 'file': {'default': None, 'type': 'string'}, 'fom_context': {'default': None, 'type': 'string'}, 'fom_name': {'default': None, 'type': 'string'}, 'formula': {'default': None, 'type': 'string'}, 'match': {'default': None, 'type': 'string'}, 'mode': {'type': 'string'}, 'name': {'type': 'string'}}, 'type': 'object'}, 'type': 'array'}, 'tables': {'default': [], 'items': {'additionalProperties': False, 'properties': {'autocolumns': {'items': {'additionalProperties': False, 'properties': {'context_name': {'type': 'string'}, 'figure_of_merit': {'type': 'string'}, 'figure_of_merit_origin_type': {'default': None, 'type': 'string'}, 'name': {'type': 'string'}, 'sort_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name', 'figure_of_merit'], 'type': 'object'}, 'type': 'array'}, 'columns': {'items': {'additionalProperties': False, 'properties': {'expression': {'default': None, 'type': 'string'}, 'figure_of_merit': {'default': None, 'type': 'string'}, 'figure_of_merit_context': {'default': None, 'type': 'string'}, 'figure_of_merit_origin_type': {'default': None, 'type': 'string'}, 'name': {'type': 'string'}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name'], 'type': 'object'}, 'type': 'array'}, 'group_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'group_method': {'default': 'max', 'type': 'string'}, 'name': {'type': 'string'}, 'sort_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'transpose': {'default': False, 'type': 'boolean'}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name'], 'type': 'object'}, 'type': 'array'}, 'tags': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'template': {'type': 'boolean'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}, 'variants': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}, 'zips': {'additionalProperties': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'default': {}, 'properties': {}, 'type': 'object'}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'formatted_executables': {'additionalProperties': {'additionalProperties': False, 'default': {}, 'properties': {'commands': {'default': ['{unformatted_command}'], 'items': {'type': 'string'}, 'type': 'array'}, 'indentation': {'default': 0, 'type': 'number'}, 'join_separator': {'default': '\n', 'type': 'string'}, 'prefix': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'internals': {'additionalProperties': False, 'default': {}, 'properties': {'custom_executables': {'additionalProperties': {'default': {'force': False, 'output_capture': OUTPUT_CAPTURE.ALL, 'redirect': '{log_file}', 'template': [], 'use_mpi': False, 'variables': {}}, 'properties': {'force': {'type': 'boolean'}, 'redirect': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'template': {'anyOf': [{'type': 'string'}, {'type': 'number'}, {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}]}, 'use_mpi': {'type': 'boolean'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'executable_injection': {'default': [], 'items': {'additionalProperties': {'relative_to': {'type': 'string'}}, 'default': {}, 'properties': {'name': {'type': 'string'}, 'order': {'default': 'after', 'type': 'string'}}, 'type': 'object'}, 'type': 'array'}, 'executables': {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}}, 'type': 'object'}, 'modifiers': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'mode': {'type': 'string'}, 'name': {'type': 'string'}, 'on_executable': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'type': 'object'}, 'type': 'array'}, 'success_criteria': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'anti_match': {'default': None, 'type': 'string'}, 'file': {'default': None, 'type': 'string'}, 'fom_context': {'default': None, 'type': 'string'}, 'fom_name': {'default': None, 'type': 'string'}, 'formula': {'default': None, 'type': 'string'}, 'match': {'default': None, 'type': 'string'}, 'mode': {'type': 'string'}, 'name': {'type': 'string'}}, 'type': 'object'}, 'type': 'array'}, 'tables': {'default': [], 'items': {'additionalProperties': False, 'properties': {'autocolumns': {'items': {'additionalProperties': False, 'properties': {'context_name': {'type': 'string'}, 'figure_of_merit': {'type': 'string'}, 'figure_of_merit_origin_type': {'default': None, 'type': 'string'}, 'name': {'type': 'string'}, 'sort_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name', 'figure_of_merit'], 'type': 'object'}, 'type': 'array'}, 'columns': {'items': {'additionalProperties': False, 'properties': {'expression': {'default': None, 'type': 'string'}, 'figure_of_merit': {'default': None, 'type': 'string'}, 'figure_of_merit_context': {'default': None, 'type': 'string'}, 'figure_of_merit_origin_type': {'default': None, 'type': 'string'}, 'name': {'type': 'string'}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name'], 'type': 'object'}, 'type': 'array'}, 'group_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'group_method': {'default': 'max', 'type': 'string'}, 'name': {'type': 'string'}, 'sort_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'transpose': {'default': False, 'type': 'boolean'}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name'], 'type': 'object'}, 'type': 'array'}, 'tags': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'template': {'type': 'boolean'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}, 'variants': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}, 'zips': {'additionalProperties': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'default': {}, 'properties': {}, 'type': 'object'}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'zips': {'additionalProperties': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'default': {}, 'properties': {}, 'type': 'object'}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}}, 'title': 'Ramble application configuration file schema', 'type': 'object'}, 'base_application_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'base_application_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble base application repository configuration file schema', 'type': 'object'}, 'base_class_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'base_class_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble base class repository configuration file schema', 'type': 'object'}, 'base_modifier_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'base_modifier_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble base modifier repository configuration file schema', 'type': 'object'}, 'base_package_manager_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'base_package_manager_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble base package manager repository configuration file schema', 'type': 'object'}, 'base_platform_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'base_platform_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble base platform repository configuration file schema', 'type': 'object'}, 'base_system_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'base_system_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble base system repository configuration file schema', 'type': 'object'}, 'base_workflow_manager_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'base_workflow_manager_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble base workflow manager repository configuration file schema', 'type': 'object'}, 'config': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'config': {'aggregate_warnings': {'default': False, 'type': 'boolean'}, 'default': {}, 'deprecatedProperties': {'error': False, 'message': 'config:module_roots has been replaced by modules:[module set]:roots and is ignored', 'properties': ['module_roots']}, 'disable_logger': {'default': False, 'type': 'boolean'}, 'disable_passthrough': {'default': False, 'type': 'boolean'}, 'disable_progress_bar': {'default': False, 'type': 'boolean'}, 'enable_strict_versions': {'default': True, 'type': 'boolean'}, 'enable_workspace_prompt': {'default': False, 'type': 'boolean'}, 'input_cache': {'default': '$ramble/var/ramble/cache', 'type': 'string'}, 'n_repeats': {'default': '0', 'type': 'string'}, 'overwrite_inventories': {'default': False, 'type': 'boolean'}, 'pip': {'additionalProperties': False, 'default': {'install': {'flags': []}}, 'properties': {'install': {'additionalProperties': False, 'properties': {'flags': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'type': 'object'}}, 'type': 'object'}, 'properties': {'additional_external_search_paths': {'items': {'type': 'string'}, 'type': 'array'}, 'allow_sgid': {'type': 'boolean'}, 'binary_index_root': {'type': 'string'}, 'build_jobs': {'minimum': 1, 'type': 'integer'}, 'build_language': {'type': 'string'}, 'build_stage': {'oneOf': [{'type': 'string'}, {'items': {'type': 'string'}, 'type': 'array'}]}, 'ccache': {'type': 'boolean'}, 'checksum': {'type': 'boolean'}, 'concretizer': {'enum': ['original', 'clingo'], 'type': 'string'}, 'connect_timeout': {'minimum': 0, 'type': 'integer'}, 'db_lock_timeout': {'minimum': 1, 'type': 'integer'}, 'debug': {'type': 'boolean'}, 'deprecated': {'type': 'boolean'}, 'dirty': {'type': 'boolean'}, 'extensions': {'items': {'type': 'string'}, 'type': 'array'}, 'install_hash_length': {'minimum': 1, 'type': 'integer'}, 'install_missing_compilers': {'type': 'boolean'}, 'install_path_scheme': {'type': 'string'}, 'install_tree': {'anyOf': [{'properties': {'padded_length': {'oneOf': [{'minimum': 0, 'type': 'integer'}, {'type': 'boolean'}]}, 'projections': {'patternProperties': {'all|\\w[\\w-]*': {'type': 'string'}}, 'type': 'object'}, 'root': {'type': 'string'}}, 'type': 'object'}, {'type': 'string'}]}, 'license_dir': {'type': 'string'}, 'locks': {'type': 'boolean'}, 'misc_cache': {'type': 'string'}, 'package_lock_timeout': {'anyOf': [{'minimum': 1, 'type': 'integer'}, {'type': 'null'}]}, 'shared_linking': {'enum': ['rpath', 'runpath'], 'type': 'string'}, 'source_cache': {'type': 'string'}, 'suppress_gpg_warnings': {'type': 'boolean'}, 'template_dirs': {'items': {'type': 'string'}, 'type': 'array'}, 'test_stage': {'type': 'string'}, 'url_fetch_method': {'enum': ['urllib', 'curl'], 'type': 'string'}, 'verify_ssl': {'type': 'boolean'}}, 'repeat_success_strict': {'default': True, 'type': 'boolean'}, 'report_dirs': {'default': '~/.ramble/reports', 'type': 'string'}, 'shell': {'enum': ['sh', 'bash', 'csh', 'tcsh', 'fish'], 'type': 'string'}, 'spack': {'additionalProperties': False, 'default': {'concretize': {'flags': '--fresh'}, 'install': {'flags': '--fresh'}}, 'flags': {}, 'properties': {'buildcache': {'additionalProperties': False, 'default': {'flags': '', 'prefix': ''}, 'properties': {'flags': {'default': '', 'type': 'string'}, 'prefix': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'compiler_find': {'default': {'flags': '', 'prefix': ''}, 'properties': {'flags': {'default': '', 'type': 'string'}, 'prefix': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'concretize': {'additionalProperties': False, 'default': {'flags': '--fresh', 'prefix': ''}, 'properties': {'flags': {'default': '--fresh', 'type': 'string'}, 'prefix': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'env_create': {'additionalProperties': False, 'default': {'flags': ''}, 'properties': {'flags': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'env_view': {'additionalProperties': False, 'default': {'link_type': 'symlink'}, 'properties': {'link_type': {'default': 'symlink', 'type': 'string'}}, 'type': 'object'}, 'global': {'additionalProperties': False, 'default': {'flags': ''}, 'properties': {'flags': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'install': {'additionalProperties': False, 'default': {'flags': '--fresh', 'prefix': ''}, 'properties': {'flags': {'default': '--fresh', 'type': 'string'}, 'prefix': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'resolve_variables_in_subprocesses': {'additionalProperties': False, 'default': False, 'type': 'boolean'}}, 'type': 'object'}, 'stage_method': {'default': 'cp', 'enum': ['cp', 'rsync', 'symbolic_link', 'hard_link'], 'type': 'string'}, 'suppress_warnings': {'default': False, 'type': 'boolean'}, 'type': 'object', 'upload': {'properties': {'push_failed': {'default': True, 'type': 'boolean'}, 'type': {'default': 'BigQuery', 'enum': ['BigQuery', 'PrintOnly', 'SQLite'], 'type': 'string'}, 'uri': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'user': {'default': '', 'type': 'string'}, 'workspace_dirs': {'default': '$ramble/var/ramble/workspaces', 'items': {'type': 'string'}, 'type': ['string', 'array']}}}, 'title': 'Ramble core configuration file schema', 'type': 'object'}, 'env_vars': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'env_vars': {'append': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'var-separator': {'type': 'string'}, 'vars': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'prepend': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'set': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'unset': {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}}}, 'title': 'Ramble environment variable configuration file schema', 'type': 'object'}, 'formatted_executables': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'formatted_executables': {'additionalProperties': {'additionalProperties': False, 'default': {}, 'properties': {'commands': {'default': ['{unformatted_command}'], 'items': {'type': 'string'}, 'type': 'array'}, 'indentation': {'default': 0, 'type': 'number'}, 'join_separator': {'default': '\n', 'type': 'string'}, 'prefix': {'default': '', 'type': 'string'}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}}, 'title': 'Ramble formatted executables configuration file schema', 'type': 'object'}, 'internals': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'internals': {'additionalProperties': False, 'default': {}, 'properties': {'custom_executables': {'additionalProperties': {'default': {'force': False, 'output_capture': OUTPUT_CAPTURE.ALL, 'redirect': '{log_file}', 'template': [], 'use_mpi': False, 'variables': {}}, 'properties': {'force': {'type': 'boolean'}, 'redirect': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'template': {'anyOf': [{'type': 'string'}, {'type': 'number'}, {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}]}, 'use_mpi': {'type': 'boolean'}, 'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'executable_injection': {'default': [], 'items': {'additionalProperties': {'relative_to': {'type': 'string'}}, 'default': {}, 'properties': {'name': {'type': 'string'}, 'order': {'default': 'after', 'type': 'string'}}, 'type': 'object'}, 'type': 'array'}, 'executables': {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}}, 'type': 'object'}}, 'title': 'Ramble internals configuration file schema', 'type': 'object'}, 'licenses': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'licenses': {'additionalProperties': {'additionalProperties': False, 'default': {}, 'properties': {'append': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'var-separator': {'type': 'string'}, 'vars': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'prepend': {'items': {'additionalProperties': {}, 'properties': {'paths': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}}, 'type': 'object'}, 'type': 'array'}, 'set': {'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}}, 'type': 'object'}, 'unset': {'default': [], 'items': {'anyOf': [{'type': 'string'}, {'type': 'number'}]}, 'type': 'array'}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}}, 'title': 'Ramble licenses configuration file schema', 'type': 'object'}, 'mirrors': {'$schema': 'http://json-schema.org/draft-07/schema#', 'additionalProperties': False, 'properties': {'mirrors': {'additionalProperties': False, 'default': {}, 'patternProperties': {'\\w[\\w-]*': {'anyOf': [{'type': 'string'}, {'properties': {'fetch': {'type': ['string', 'object']}, 'push': {'type': ['string', 'object']}}, 'required': ['fetch', 'push'], 'type': 'object'}]}}, 'type': 'object'}}, 'title': 'Ramble mirror configuration file schema', 'type': 'object'}, 'modifier_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'modifier_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble modifier repository configuration file schema', 'type': 'object'}, 'modifiers': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'modifiers': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'mode': {'type': 'string'}, 'name': {'type': 'string'}, 'on_executable': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'type': 'object'}, 'type': 'array'}}, 'title': 'Ramble application configuration file schema', 'type': 'object'}, 'package_manager_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'package_manager_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble package manager repository configuration file schema', 'type': 'object'}, 'platform_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'platform_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble platform repository configuration file schema', 'type': 'object'}, 'repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble repository configuration file schema', 'type': 'object'}, 'software': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'software': {'additionalProperties': False, 'default': {}, 'properties': {'environments': {'additionalProperties': {'additionalProperties': {'type': 'string'}, 'default': {}, 'properties': {'external_env': {'default': None, 'type': 'string'}, 'packages': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'type': 'object'}, 'default': {}, 'properties': {}, 'type': 'object'}, 'packages': {'additionalProperties': {'additionalProperties': True, 'default': {}, 'properties': {'compiler': {'default': None, 'type': 'string'}, 'compiler_spec': {'default': None, 'type': 'string'}, 'pkg_spec': {'type': 'string'}}, 'type': 'object'}, 'type': 'object'}}, 'type': 'object'}}, 'title': 'Software configuration file schema', 'type': 'object'}, 'success_criteria': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'success_criteria': {'default': [], 'items': {'additionalProperties': False, 'default': {}, 'properties': {'anti_match': {'default': None, 'type': 'string'}, 'file': {'default': None, 'type': 'string'}, 'fom_context': {'default': None, 'type': 'string'}, 'fom_name': {'default': None, 'type': 'string'}, 'formula': {'default': None, 'type': 'string'}, 'match': {'default': None, 'type': 'string'}, 'mode': {'type': 'string'}, 'name': {'type': 'string'}}, 'type': 'object'}, 'type': 'array'}}, 'title': 'Ramble success criteria configuration file schema', 'type': 'object'}, 'system_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'system_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble system repository configuration file schema', 'type': 'object'}, 'tables': {'$schema': 'http://json-schema.org/schema#', 'properties': {'tables': {'default': [], 'items': {'additionalProperties': False, 'properties': {'autocolumns': {'items': {'additionalProperties': False, 'properties': {'context_name': {'type': 'string'}, 'figure_of_merit': {'type': 'string'}, 'figure_of_merit_origin_type': {'default': None, 'type': 'string'}, 'name': {'type': 'string'}, 'sort_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name', 'figure_of_merit'], 'type': 'object'}, 'type': 'array'}, 'columns': {'items': {'additionalProperties': False, 'properties': {'expression': {'default': None, 'type': 'string'}, 'figure_of_merit': {'default': None, 'type': 'string'}, 'figure_of_merit_context': {'default': None, 'type': 'string'}, 'figure_of_merit_origin_type': {'default': None, 'type': 'string'}, 'name': {'type': 'string'}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name'], 'type': 'object'}, 'type': 'array'}, 'group_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'group_method': {'default': 'max', 'type': 'string'}, 'name': {'type': 'string'}, 'sort_by': {'default': None, 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'transpose': {'default': False, 'type': 'boolean'}, 'where': {'items': {'type': 'string'}, 'type': 'array'}}, 'required': ['name'], 'type': 'object'}, 'type': 'array'}}, 'title': 'Ramble tables configuration file schema', 'type': 'object'}, 'variables': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'variables': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}}, 'title': 'Ramble variables configuration file schema', 'type': 'object'}, 'variants': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'variants': {'additionalProperties': True, 'default': {}, 'properties': {}, 'type': ['object', 'null']}}, 'title': 'Ramble variants configuration file schema', 'type': 'object'}, 'workflow_manager_repos': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'workflow_manager_repos': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}}, 'title': 'Ramble workflow manager repository configuration file schema', 'type': 'object'}, 'zips': {'$schema': 'http://json-schema.org/schema#', 'additionalProperties': False, 'properties': {'zips': {'additionalProperties': {'default': [], 'items': {'type': 'string'}, 'type': 'array'}, 'default': {}, 'properties': {}, 'type': 'object'}}, 'title': 'Ramble zips configuration file schema', 'type': 'object'}}

Dict from section names -> schema for that section

ramble.config.set(path, value, scope=None)[source]

Convenience function for setting single values in config files.

Accepts the path syntax described in get().

ramble.config.use_configuration(*scopes_or_paths)[source]

Use the configuration scopes passed as arguments within the context manager.

Parameters:

*scopes_or_paths – scope objects or paths to be used

Returns:

Configuration object associated with the scopes passed as arguments

ramble.config.validate(data, schema, filename=None)[source]

Validate data read in from a Ramble YAML file.

Parameters:

  • data (dict | list) – data read from a Ramble YAML file

  • schema (dict | list) – jsonschema to validate data

This leverages the line information (start_mark, end_mark) stored on Ramble YAML structures.

ramble.context module

class ramble.context.Context[source]

Bases: object

Class to represent a context

This class contains variable definitions to store any individual context (such as application, workload, or experiment) and logic to merge in additional contexts by order of precedence.

property escaped_name
merge_context(in_context)[source]

Merges another Context into this Context.

output_mapping = {'chained_experiments': 'chained_experiments', 'env_variables': 'env_vars', 'exclude': 'exclude', 'formatted_executables': 'formatted_executables', 'internals': 'internals', 'matrices': 'matrices', 'modifiers': 'modifiers', 'n_repeats': 'n_repeats', 'success_criteria': 'success_criteria', 'tables': 'tables', 'tags': 'tags', 'template': 'template', 'variables': 'variables', 'variants': 'variants', 'zips': 'zips'}
to_workspace_config(application_spec, workload_name)[source]
ramble.context.create_context_from_dict(context_name, in_dict)[source]

Creates a new Context object from an input dictionary

Dictionaries should follow the below format:

in_dict = {
    'env_vars': [],
    'variables': {},
    'variants': {},
    'internals': {},
    'template': '',
    'chained_experiments': [],
    'modifiers': [],
    'context_name': '',
    'exclude': {},
    'zips': {},
    'matrices': {} or [],
    'tags': [],
    'n_repeats': ''
}
Parameters:

  • context_name (str) – The name of the context (e.g., application name)

  • in_dict (dict) – A dictionary representing the variable definitions

Returns:

Context(object)

ramble.error module

exception ramble.error.ApplicationError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Exception that is raised by applications

exception ramble.error.ChainCycleDetectedError(message: str, long_message: str | None = None)[source]

Bases: ApplicationError

Exception raised when a cycle is detected in a defined experiment chain

exception ramble.error.ConflictingModifiersError(message: str, long_message: str | None = None)[source]

Bases: ModifierError

Exception raised when two modifiers on the same experiment conflict

exception ramble.error.DirectiveError(message: str, long_message: str | None = None)[source]

Bases: RambleError

This is raised when something is wrong with a language directive.

exception ramble.error.ExecutableNameError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Exception raised when a name collision in executables happens

exception ramble.error.FormattedExecutableError(message: str, long_message: str | None = None)[source]

Bases: ApplicationError

Exception raise when there are issues defining formatted executables

exception ramble.error.InvalidChainError(message: str, long_message: str | None = None)[source]

Bases: ApplicationError

Exception raised when a invalid chained experiment is defined

exception ramble.error.InvalidModeError(message: str, long_message: str | None = None)[source]

Bases: ModifierError

Exception raised when an invalid mode is passed

exception ramble.error.ModifierError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Exception that is raised by modifiers

exception ramble.error.ObjectValidationError(message: str, long_message: str | None = None)[source]

Bases: ApplicationError

Error when an object validator fails

exception ramble.error.PackageManagerError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Exception that is raised by package managers

exception ramble.error.RambleCommandError[source]

Bases: Exception

Raised when RambleCommand execution fails.

exception ramble.error.RambleError(message: str, long_message: str | None = None)[source]

Bases: Exception

This is the superclass for all Ramble errors. Subclasses can be found in the modules they have to do with.

die() → None[source]
property long_message: str | None
message: str
print_context() → None[source]

Print extended debug information about this exception.

This is usually printed when the top-level Ramble error handler calls die(), but it can be called separately beforehand if a lower-level error handler needs to print error context and continue without raising the exception to the top level.

printed: bool
traceback: str | None
exception ramble.error.SpecError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Superclass for all errors that occur while constructing specs.

ramble.error.debug = False

whether we should write stack traces or short error messages this is module-scoped because it needs to be set very early

ramble.expander module

exception ramble.expander.ApplicationNotDefinedError(message: str, long_message: str | None = None)[source]

Bases: ExpanderError

Raised when an application is not defined properly

class ramble.expander.Expander(variables, experiment_set, no_expand_vars=None)[source]

Bases: object

A class that will track and expand keyword arguments

This class will track variables and their definitions, to allow for expansion within string.

The variables can come from workspace variables, software stack variables, and experiment variables.

Additionally, math will be evaluated as part of expansion.

add_no_expand_var(var: str)[source]

Add a new variable to the no expand set

Parameters:

var (str) – Variable that should not expand

property application_input_dir
property application_name
property application_namespace
property application_run_dir
property application_spec
property application_version
copy()[source]
property env_path
eval_math(node, expansion_vars=None)[source]

Evaluate math from parsing the AST

Does not assume a specific type of operands. Some operators will generate floating point, while others will generate integers (if the inputs are integers).

evaluate_predicate(in_str, extra_vars=None, merge_used_stage: bool = True)[source]

Evaluate a predicate by expanding and evaluating math contained in a string

Parameters:

  • in_str – String representing predicate that should be evaluated

  • extra_vars – Variable definitions to use with highest precedence

Returns:

True or False, based on the evaluation of in_str

Return type:

bool

expand_lists(var)[source]

Expand a variable into a list if possible

If expanding a variable would generate a list, this function will return a list. If any error case happens, this function will return the unmodified input value.

NOTE: This function is generally called early in the expansion. This allows lists to be generated before rendering experiments, but does not support pulling a list from a different experiment.

expand_var(var: str, extra_vars: Dict | None = None, allow_passthrough: bool = True, typed: bool = False, merge_used_stage: bool = True, replace_escaped_braces: bool | None = None)[source]

Perform expansion of a string

Expand a string by building up a dict of all expansion variables.

Parameters:

  • var (str) – String variable to expand

  • extra_vars (dict) – Variable definitions to use with highest precedence

  • allow_passthrough (bool) – Whether the string is allowed to have keywords after expansion

  • typed (bool) – Whether the return type should be typed or not

  • merge_used_stage (bool) – Whether tracked variables are merged into the used variable set or not.

  • replace_escaped_braces (bool) – Whether escaped curly braces are replaced as part of expansion or not.

expand_var_name(var_name: str, extra_vars: Dict | None = None, allow_passthrough: bool = True, typed: bool = False, merge_used_stage: bool = True, replace_escaped_braces: bool | None = None)[source]

Convert a variable name to an expansion string, and expand it

Take a variable name (var) and convert it to an expansion string by calling the expansion_str function. Pass the expansion string into expand_var, and return the result.

Parameters:

  • var_name (str) – String name of variable to expand

  • extra_vars (dict) – Variable definitions to use with highest precedence

  • allow_passthrough (bool) – Whether the string is allowed to have keywords after expansion

  • typed (bool) – Whether the return type should be typed or not

  • merge_used_stage (bool) – Whether tracked variables are merged into the used variable set or not.

  • replace_escaped_braces (bool) – Whether escaped curly braces are replaced as part of expansion or not.

static expansion_str(in_str)[source]
property experiment_name
property experiment_namespace
property experiment_run_dir
flush_used_variable_stage()[source]
property license_input_dir
merge_used_variable_stage()[source]
perform_math_eval(in_str, expansion_vars=None)[source]

Attempt to evaluate in_str

Parameters:

in_str (str) – string representing math to attempt to evaluate

Returns:

(str) either the evaluation of in_str (if successful) or in_str unmodified (if unsuccessful)

preserve_escaped_braces()[source]
satisfies(reqs: str | List[str] | FrozenSet[str] | None = None, variant_set=None, extra_vars=None, merge_used_stage: bool = True)[source]

Determine an experiment’s variants satisfy a query

Parameters:

  • reqs – List of string requirements to check if experiment satisfies

  • extra_vars – Variable definitions to use with highest precedence

  • merged_used_stage – Whether used variables are merged into the set of used variables or not.

Returns:

True or False, based if the experiment’s variants satisfy

the input requirement.

Return type:

(bool)

set_no_expand_vars(no_expand_vars)[source]
property workload_input_dir
property workload_name
property workload_namespace
property workload_run_dir
exception ramble.expander.ExpanderError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Raised when an error happens within an expander

class ramble.expander.ExpansionDelimiter[source]

Bases: object

Class representing the delimiters for ramble expansion strings

escape = '\\'
left = '{'
right = '}'
class ramble.expander.ExpansionGraph(in_str)[source]

Bases: object

Class representing a graph of ExpansionNodes

walk(in_node=None)[source]

Perform a DFS walk of the nodes in the graph

Parameters:

in_node (ExpansionNode) – node to begin the walk from, if not set uses self.root

Yields:

(ExpansionNode) – nodes following a DFS traversal of the graph

class ramble.expander.ExpansionNode(left_idx, right_idx)[source]

Bases: object

Class representing a node in a ramble expansion graph

add_children(children)[source]

Add children to this node

Parameters:

children (ExpansionNode | list) – nodes to adds as children of self

define_value(expansion_dict, allow_passthrough=True, expansion_func=<class 'str'>, evaluation_func=<built-in function eval>, no_expand_vars=None, used_vars=None, replace_escaped_braces=None)[source]

Define the value for this node.

Construct the value of self. This builds up a string representation of self, and performs evaluation and formatting of the resulting string. This includes extracting the values of the children nodes, and replacing their values in the proper positions in self’s string.

Stores the resulting value in self.value

Parameters:

  • expansion_dict (dict) – variable definitions to use for expanding detected matches

  • allow_passthrough (bool) – if true, expansion is allowed to fail. if false, failed expansion raises an error.

  • expansion_func (func) – function to use for expansion of nested variable definitions

  • evaluation_func (func) – function to use for evaluating math of strings

  • no_expand_vars (set) – set of variable names that should never be expanded

  • replace_escaped_braces (bool) – Whether escaped curly braces are replaced as part of expansion or not.

relative_indices(relative_to)[source]

Compute node indices relative to another node

Parameters:

relative_to (ExpansionNode) – node to shift current node’s indices relative to

Returns:

(tuple) indices of shifted match set

exception ramble.expander.ExperimentNotDefinedError(message: str, long_message: str | None = None)[source]

Bases: ExpanderError

Raised when an experiment is not defined properly

exception ramble.expander.MathEvaluationError(message: str, long_message: str | None = None)[source]

Bases: ExpanderError

Raised when an error happens while evaluating math during expansion

exception ramble.expander.RamblePassthroughError(message: str, long_message: str | None = None)[source]

Bases: ExpanderError

Raised when passthrough is disabled and variables fail to expand

exception ramble.expander.RambleSyntaxError(message: str, long_message: str | None = None)[source]

Bases: ExpanderError

Raised when a syntax error happens within variable definitions

class ramble.expander.VformatDelimiter[source]

Bases: object

Class representing the delimiters for the string.Formatter class

left = '{'
right = '}'
exception ramble.expander.WorkloadNotDefinedError(message: str, long_message: str | None = None)[source]

Bases: ExpanderError

Raised when a workload is not defined properly

ramble.expander.raise_passthrough_error(in_str, out_str)[source]

Raise an error when passthrough is disabled but variables are not all expanded

ramble.experiment_result module

class ramble.experiment_result.ExperimentResult(app_inst)[source]

Bases: object

Class containing results and related metadata of an experiment

cache_file_name = 'ramble_results_cache.json'
finalize(workspace)[source]
from_dict(in_dict: dict)[source]

Convert a dict back into a results object

Parameters:

in_dict (dict) – Input dictionary of results from a cache

read_cache(workspace, app_inst) → bool[source]
to_dict()[source]

Generate a dict for encoders (json, yaml) and uploaders.

The generated dict preserves the existing serialized format so that previous result files work as expected.

write_cache(app_inst)[source]
class ramble.experiment_result.ExperimentStatus(*values)[source]

Bases: str, Enum

CANCELLED = 'CANCELLED'
COMPLETE = 'COMPLETE'
FAILED = 'FAILED'
RUNNING = 'RUNNING'
SETUP = 'SETUP'
SUBMITTED = 'SUBMITTED'
SUCCESS = 'SUCCESS'
TIMEOUT = 'TIMEOUT'
UNKNOWN = 'UNKNOWN'
UNQUEUED = 'UNQUEUED'
UNRESOLVED = 'UNRESOLVED'

ramble.experiment_set module

class ramble.experiment_set.ExperimentSet(workspace)[source]

Bases: object

Class to represent a full set of experiments

This class contains logic to take sets of variable definitions and generate experiments from the variable hierarchy.

Experiments are housed in the internal self.experiments dictionary. Keys of this dictionary are experiment names, while values are application instances.

add_chained_experiment(name, instance)[source]
all_experiment_tags()[source]

Aggregate all tags from experiments in this experiment set

Returns:

A set of all tags from the experiment set.

Return type:

(set)

all_experiments()[source]

Iterator over all experiments in this set

property application_namespace

Property to return application namespace (application spec)

build_experiment_chains()[source]
define_scoped_tables(workload_names, experiment_template_name)[source]
property experiment_namespace

Property to return experiment namespace

Experiment namespaces are of the form: application_spec.workload_name.experiment_name

filtered_experiments(filters)[source]

Return a filtered set of all experiments based on a logical expression

Exclusion takes overrides inclusion. If conflicting filters are provided which both include, and exclude the same experiment, the experiment will be excluded.

Parameters:

expression – A logical expression to evaluate, with each experiment

Yields:

exp – The name of the experiment, if expression results in True inst: An application instance representing the experiment

get_config_env_vars(workspace)[source]
get_config_vars(workspace)[source]
get_experiment(experiment)[source]
get_var_from_experiment(experiment, variable)[source]

Lookup a variable in a given experiment

Does not error if invalid values are passed in, to allow @ symbol to pass through to rendered content.

Parameters:

  • experiment – A fully qualified experiment name (application.workload.experiment)

  • variable – Name of variable to look up

keywords

Create experiment set class

num_experiments()[source]

Return the number of total experiments in this set

num_filtered_experiments(filters)[source]

Return the number of filtered experiments in this set

read_config_vars(workspace)[source]
render_experiment_set(app_name, workload_name, experiment_context, warn_validation=True, die_on_validate_error=True, chained=False) → list[source]

Render a set of experiments for a specific application and workload

This method will render a new set of experiments for a given app (input with app_name) and workload (input with workload_name, but could be an indirect variable reference). It takes a context object for the experiment, and will process any vectors and matrices to create multiple experiments.

If chained=True these are added to this experiment set’s chained_experiments list, rather than the base experiments list. Upon completion, all rendered experiment instances are returned in a list, to allow further processing. For example, if one wants to render chained experiments from the child experiment.

Parameters:

  • app_name (str) – Name of application to render experiments for

  • workload_name (str) – Name, or template, of workload(s) to render experiments for

  • experiment_context (ramble.context.Context) – Context object for the set of experiments to render

  • warn_validation (bool) – Whether validation warnings should print or not

  • die_on_validate_error (bool) – Whether validation errors should be fatal or not

  • chained (bool) – Whether the experiments are chained experiments or not

Returns:

List of application instances from the rendered set of experiments

Return type:

(list)

search_primary_experiments(pattern)[source]

Search primary experiments using a glob syntax.

NOTE: This does not search experiments defined in an experiment chain

set_application_context(app_context)[source]

Set up current application context

set_base_var(var, val)[source]

Set a base variable definition

set_experiment_context(experiment_context, warn_validation=True, die_on_validate_error=True, chained=False) → list[source]

Set up current experiment context

set_workload_context(workload_context)[source]

Set up current workload context

template_experiments()[source]

Iterator over template experiments in this set

property workload_namespace

Property to return workload namespace

Workload namespaces are of the form: application_spec.workload_name

exception ramble.experiment_set.RambleExperimentSetError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Super class for all experiment set errors

exception ramble.experiment_set.RambleVariableDefinitionError(message: str, long_message: str | None = None)[source]

Bases: RambleExperimentSetError

Error when a ramble variable definition is invalid

ramble.fetch_strategy module

Fetch strategies are used to download source code into a staging area in order to build it. They need to define the following methods:

  • fetch()

    This should attempt to download/check out source from somewhere.

  • check()

    Apply a checksum to the downloaded source code, e.g. for an archive. May not do anything if the fetch method was safe to begin with.

  • expand()

    Expand (e.g., an archive) downloaded file to source, with the standard stage source path as the destination directory.

  • reset()

    Restore original state of downloaded code. Used by clean commands. This may just remove the expanded source and re-expand an archive, or it may run something like git reset –hard.

  • archive()

    Archive a source directory, e.g. for creating a mirror.

class ramble.fetch_strategy.BundleFetchStrategy(**kwargs)[source]

Bases: FetchStrategy

Fetch strategy associated with bundle, or no-code, packages.

Having a basic fetch strategy is a requirement for executing post-install hooks. Consequently, this class provides the API but does little more than log messages.

TODO: Remove this class by refactoring resource handling and the link between composite stages and composite fetch strategies (see #11981).

property cachable

Report False as there is no code to cache.

fetch()[source]

Simply report success – there is no code to fetch.

mirror_id()[source]

BundlePackages don’t have a mirror id.

source_id()[source]

BundlePackages don’t have a source id.

url_attr: str | None = ''

There is no associated URL keyword in version() for no-code packages but this property is required for some strategy-related functions (e.g., check_pkg_attributes).

class ramble.fetch_strategy.CacheURLFetchStrategy(url=None, checksum=None, **kwargs)[source]

Bases: URLFetchStrategy

The resource associated with a cache URL may be out of date.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

exception ramble.fetch_strategy.ChecksumError(message: str, long_message: str | None = None)[source]

Bases: FetchError

Raised when archive fails to checksum.

class ramble.fetch_strategy.CvsFetchStrategy(**kwargs)[source]

Bases: VCSFetchStrategy

Fetch strategy that gets source code from a CVS repository.

Use like this in a package:

version(‘name’,

cvs=’:pserver:anonymous@www.example.com:/cvsroot%module=modulename’)

Optionally, you can provide a branch and/or a date for the URL:

version(‘name’,

cvs=’:pserver:anonymous@www.example.com:/cvsroot%module=modulename’, branch=’branchname’, date=’date’)

Repositories are checked out into the standard stage source path directory.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

property cvs
fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str] = ['branch', 'date']
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str().

url_attr: str | None = 'cvs'

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

exception ramble.fetch_strategy.FailedDownloadError(url, msg='')[source]

Bases: FetchError

Raised when a download fails.

exception ramble.fetch_strategy.FetchError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Superclass for fetcher errors.

class ramble.fetch_strategy.FetchStrategy(**kwargs)[source]

Bases: object

Superclass of all fetch strategies.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

check()[source]

Checksum the archive fetched by this FetchStrategy.

expand()[source]

Expand the downloaded archive into the stage source path.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

classmethod matches(args)[source]

Predicate that matches fetch strategies to arguments of the version directive.

Parameters:

args – arguments of the version directive

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str] = []
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str().

url: str | None = None
url_attr: str | None = None

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class ramble.fetch_strategy.FetchStrategyComposite[source]

Bases: Composite

Composite for a FetchStrategy object.

classmethod matches(args)

Predicate that matches fetch strategies to arguments of the version directive.

Parameters:

args – arguments of the version directive

source_id()[source]
class ramble.fetch_strategy.FsCache(root)[source]

Bases: object

destroy()[source]
fetcher(target_path, digest, **kwargs)[source]
store(fetcher, relative_dest)[source]
class ramble.fetch_strategy.GCSFetchStrategy(*args, **kwargs)[source]

Bases: URLFetchStrategy

FetchStrategy that pulls from a GCS bucket.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

url_attr: str | None = 'gs'

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class ramble.fetch_strategy.GitFetchStrategy(**kwargs)[source]

Bases: VCSFetchStrategy

Fetch strategy that gets source code from a git repository. Use like this in a package:

version(‘name’, git=’https://github.com/project/repo.git’)

Optionally, you can provide a branch, or commit to check out, e.g.:

version(‘1.1’, git=’https://github.com/project/repo.git’, tag=’v1.1’)

You can use these three optional attributes in addition to git:

  • branch: Particular branch to build from (default is the

    repository’s default branch)

  • tag: Particular tag to check out

  • commit: Particular commit hash in the repo

Repositories are cloned into the standard stage source path directory.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

clone(dest=None, commit=None, branch=None, tag=None, bare=False)[source]

Clone a repository to a path.

This method handles cloning from git, but does not require a stage.

Parameters:

  • dest (str | None) – The path into which the code is cloned. If None, requires a stage and uses the stage’s source path.

  • commit (str | None) – A commit to fetch from the remote. Only one of commit, branch, and tag may be non-None.

  • branch (str | None) – A branch to fetch from the remote.

  • tag (str | None) – A tag to fetch from the remote.

  • bare (bool) – Execute a “bare” git clone (–bare option to git)

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

get_full_repo: bool = False
property git
property git_version
git_version_re = 'git version (\\S+)'
mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str] = ['tag', 'branch', 'commit', 'submodules', 'get_full_repo', 'submodules_delete']
protocol_supports_shallow_clone()[source]

Shallow clone operations (–depth #) are not supported by the basic HTTP protocol or by no-protocol file specifications. Use (e.g.) https:// or file:// instead.

reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str().

submodules: bool = False
submodules_delete: bool = False
url_attr: str | None = 'git'

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

static version_from_git(git_exe)[source]

Given a git executable, return the Version (this will fail if the output cannot be parsed into a valid Version).

class ramble.fetch_strategy.HgFetchStrategy(**kwargs)[source]

Bases: VCSFetchStrategy

Fetch strategy that gets source code from a Mercurial repository. Use like this in a package:

version(‘name’, hg=’https://jay.grs.rwth-aachen.de/hg/lwm2’)

Optionally, you can provide a branch, or revision to check out, e.g.:

version(‘torus’,

hg=’https://jay.grs.rwth-aachen.de/hg/lwm2’, branch=’torus’)

You can use the optional ‘revision’ attribute to check out a branch, tag, or particular revision in hg. To prevent non-reproducible builds, using a moving target like a branch is discouraged.

  • revision: Particular revision, branch, or tag.

Repositories are cloned into the standard stage source path directory.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

property hg

Returns: Executable: the hg executable

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str] = ['revision']
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str().

url_attr: str | None = 'hg'

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

exception ramble.fetch_strategy.InvalidArgsError(**args)[source]

Bases: FetchError

Raised when a version can’t be deduced from a set of arguments.

exception ramble.fetch_strategy.NoArchiveFileError(message: str, long_message: str | None = None)[source]

Bases: FetchError

“Raised when an archive file is expected but none exists.

exception ramble.fetch_strategy.NoCacheError(message: str, long_message: str | None = None)[source]

Bases: FetchError

Raised when there is no cached archive for a package.

exception ramble.fetch_strategy.NoDigestError(message: str, long_message: str | None = None)[source]

Bases: FetchError

Raised after attempt to checksum when URL has no digest.

exception ramble.fetch_strategy.NoStageError(method)[source]

Bases: FetchError

Raised when fetch operations are called before set_stage().

class ramble.fetch_strategy.S3FetchStrategy(*args, **kwargs)[source]

Bases: URLFetchStrategy

FetchStrategy that pulls from an S3 bucket.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

url_attr: str | None = 's3'

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class ramble.fetch_strategy.SvnFetchStrategy(**kwargs)[source]

Bases: VCSFetchStrategy

Fetch strategy that gets source code from a subversion repository.

Use like this in a package:

version(‘name’, svn=’http://www.example.com/svn/trunk’)

Optionally, you can provide a revision for the URL:

version(‘name’, svn=’http://www.example.com/svn/trunk’,

revision=’1641’)

Repositories are checked out into the standard stage source path directory.

archive(destination)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str] = ['revision']
reset()[source]

Revert to freshly downloaded state.

For archive files, this may just re-expand the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str().

property svn
url_attr: str | None = 'svn'

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class ramble.fetch_strategy.URLFetchStrategy(url=None, checksum=None, **kwargs)[source]

Bases: FetchStrategy

URLFetchStrategy pulls source code from a URL for an archive, check the archive against a checksum, and decompresses the archive.

The destination for the resulting file(s) is the standard stage path.

archive(destination)[source]

Just moves this archive to the destination.

property archive_file

Path to the source archive within this stage directory.

property cachable

Whether fetcher is capable of caching the resource it retrieves.

This generally is determined by whether the resource is identifiably associated with a specific package version.

Returns:

True if can cache, False otherwise.

Return type:

bool

property candidate_urls
check()[source]

Check the downloaded archive against a checksum digest. No-op if this stage checks code out of a repository.

property curl
expand()[source]

Expand the downloaded archive into the stage source path.

fetch()[source]

Fetch source code archive or repo.

Returns:

True on success, False on failure.

Return type:

bool

mirror_id()[source]

This is a unique ID for a source that is intended to help identify reuse of resources across packages.

It is unique like source-id, but it does not include the package name and is not necessarily easy for a human to create themselves.

optional_attrs: List[str] = ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512', 'checksum']
reset()[source]

Removes the source path if it exists, then re-expands the archive.

source_id()[source]

A unique ID for the source.

It is intended that a human could easily generate this themselves using the information available to them in the Spack package.

The returned value is added to the content which determines the full hash for a package using str().

url_attr: str | None = 'url'

The URL attribute must be specified either at the package class level, or as a keyword argument to version(). It is used to distinguish fetchers for different versions in the package DSL.

class ramble.fetch_strategy.VCSFetchStrategy(**kwargs)[source]

Bases: FetchStrategy

Superclass for version control system fetch strategies.

Like all fetchers, VCS fetchers are identified by the attributes passed to the version directive. The optional_attrs for a VCS fetch strategy represent types of revisions, e.g. tags, branches, commits, etc.

The required attributes (git, svn, etc.) are used to specify the URL and to distinguish a VCS fetch strategy from a URL fetch strategy.

archive(destination, **kwargs)[source]

Create an archive of the downloaded data for a mirror.

For downloaded files, this should preserve the checksum of the original file. For repositories, it should just create an expandable tarball out of the downloaded repository.

branch: str | None = None
check()[source]

Checksum the archive fetched by this FetchStrategy.

commit: str | None = None
expand()[source]

Expand the downloaded archive into the stage source path.

revision: str | None = None
tag: str | None = None
ramble.fetch_strategy.all_strategies = [<class 'ramble.fetch_strategy.BundleFetchStrategy'>, <class 'ramble.fetch_strategy.URLFetchStrategy'>, <class 'ramble.fetch_strategy.CacheURLFetchStrategy'>, <class 'ramble.fetch_strategy.GitFetchStrategy'>, <class 'ramble.fetch_strategy.CvsFetchStrategy'>, <class 'ramble.fetch_strategy.SvnFetchStrategy'>, <class 'ramble.fetch_strategy.HgFetchStrategy'>, <class 'ramble.fetch_strategy.S3FetchStrategy'>, <class 'ramble.fetch_strategy.GCSFetchStrategy'>]

List of all fetch strategies, created by FetchStrategy metaclass.

ramble.fetch_strategy.fetcher(cls)[source]

Decorator used to register fetch strategies.

ramble.fetch_strategy.from_kwargs(**kwargs)[source]

Construct an appropriate FetchStrategy from the given keyword arguments.

Parameters:

**kwargs – dictionary of keyword arguments, e.g. from a version() directive in a package.

Returns:

The fetch strategy that matches the args, based

on attribute names (e.g., git, hg, etc.)

Return type:

FetchStrategy

Raises:

FetchError – If no fetch_strategy matches the args.

ramble.fetch_strategy.from_url_scheme(url, *args, **kwargs)[source]

Finds a suitable FetchStrategy by matching its url_attr with the scheme in the given url.

ramble.fetch_strategy.stable_target(fetcher)[source]

Returns whether the fetcher target is expected to have a stable checksum. This is only true if the target is a preexisting archive file.

ramble.fetch_strategy.warn_content_type_mismatch(subject, content_type='HTML')[source]

ramble.filters module

class ramble.filters.Filters(phase_filters: List[str] | None = None, include_where_filters: List[List[str]] | None = None, exclude_where_filters: List[List[str]] | None = None, tags: List[List[str]] | None = None)[source]

Bases: object

Object containing filters for limiting various operations in Ramble

ramble.graphs module

class ramble.graphs.AttributeGraph(obj_inst)[source]

Bases: object

add_node(node)[source]

Add a node to the graph

Parameters:

node (ramble.util.graph.GraphNode) – Node to add into graph

define_edges(node, dep_nodes=None, internal_order=False)[source]

Define graph edges

Process dependencies, and internal orderings (inside the node object) to define new graph edges.

Parameters:
get_node(key)[source]

Given a key, return the node containing this key

Parameters:

key (str) – Name of key to find in the graph

Returns:

Node representing the key

requested. Returns None if the key isn’t found.

Return type:

(ramble.util.graph.GraphNode)

node_type = 'object'
update_graph(node, dep_nodes=None, internal_order=False)[source]

Update the graph with a new node and / or new dependencies.

Given a node, and list of dependencies, define new edges in the graph. If the node is new, also construct a new phase node.

Parameters:
walk()[source]

Walk the graph in topological ordering and yield each node.

Construct a topological ordering of the current graph, walk it, and yield each node one by one.

Yields:

node (ramble.util.graph.GraphNode) – Each node in the graph

class ramble.graphs.ExecutableGraph(exec_order, executables, builtin_objects, builtin_groups, obj_inst)[source]

Bases: AttributeGraph

Graph that handles command executables and builtins

inject_executable(exec_name, injection_order, relative)[source]

Inject an executable into the graph

Parameters:

  • exec_name (str) – Name of executable to inject

  • injection_order (str) – Order for injection. Can be ‘before’ or ‘after’

  • relative (str) – Name of executable to inject relative to. Can be None to inject relative to the whole set of executables.

node_type = 'command executable'
class supported_injection_orders(*values)

Bases: Enum

after = 2
before = 1
class ramble.graphs.FormattedExecutableGraph(formatted_execs: dict, obj_inst)[source]

Bases: AttributeGraph

Graph that handles formatted executables

node_type = 'formatted executable'
exception ramble.graphs.GraphCycleError(message: str, long_message: str | None = None)[source]

Bases: GraphError

Exception raised when a cycle is detected in a graph

exception ramble.graphs.GraphError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Exception raised with errors in a graph type

exception ramble.graphs.GraphNodeAmbiguousError(message: str, long_message: str | None = None)[source]

Bases: GraphError

Exception raised when the given name can be resolved to non-unique nodes

exception ramble.graphs.GraphNodeNotFoundError(message: str, long_message: str | None = None)[source]

Bases: GraphError

Exception raised when the given name cannot be resolved to a node

class ramble.graphs.PhaseGraph(phase_definitions, obj_inst)[source]

Bases: AttributeGraph

add_node(node, obj_inst=None)[source]

Add a new phase node to the graph

Extract the phase function from the object instance, and inject a new node into the graph.

Parameters:
node_type = 'phase'
update_graph(phase_name, dependencies=None, internal_order=False, obj_inst=None)[source]

Update the graph with a new phase and / or new dependencies.

Given a phase name, and list of dependencies, define new edges in the graph. If the phase is new, also construct a new phase node.

Parameters:

  • phase_name (str) – Name of the phase to inject or modify

  • dependencies (list(str) | None) – List of phase names to inject dependencies on

  • internal_order (bool) – True to process internal dependencies, False to skip

  • obj_inst (object) – Application or modifier instance to extract phase function from

ramble.keywords module

class ramble.keywords.Keywords(extra_keys=None)[source]

Bases: object

Class to represent known ramble keywords.

Each keyword contains a dictionary of its attributes. Currently, these include: - type - level

Valid types are identified by the ‘key_type’ variable as an enum. Valid levels are identified by the ‘output_level’.

Current key types are:

  • Reserved: Ramble defines these, and a user should not be allowed to define them

  • Optional: Ramble can function with a definition from the user but it isn’t required

  • Required: Ramble requires a definition for these. Ramble will try to set sensible defaults, but it might not be possible always.

Current levels are:
  • Key: Ramble defines this as a top level variable. When results are

    output, these are hoisted to a set of variables that are guaranteed to be in the output. These are non-application specific inputs that define a Ramble experiment.

  • Variable: These are considered standard variables. They might be

    derived from the values of entries with the level key. In results, they are presented in the variables section. These may include application specific inputs to further configure the experiment.

RAMBLE_STATUS: str
all_required_keys()[source]

Yield all required keys

Yields:

(str) – Key name

application_input_dir: str
application_name: str
application_namespace: str
application_run_dir: str
application_spec: str
application_version: str
batch_submit: str
check_required_keys(definitions, warn_validation=True, die_on_validate_error=True)[source]

Check a dictionary of variable definitions for all required keywords

check_reserved_keys(definitions)[source]

Check a dictionary of variable definitions for reserved keywords

copy()[source]
env_name: str
env_path: str
err_file: str
experiment_hash: str
experiment_index: str
experiment_name: str
experiment_namespace: str
experiment_run_dir: str
experiment_status: str
experiment_template_name: str
experiments_file: str
input_name: str
is_key_level(key)[source]

Check if key is part of the key level

is_required(key)[source]

Check if a key is required

is_reserved(key)[source]

Check if a key is reserved

is_valid(key)[source]

Check if a key is valid as a known keyword

license_input_dir: str
log_dir: str
log_file: str
mpi_command: str
n_nodes: str
n_ranks: str
n_threads: str
processes_per_node: str
repeat_index: str
simplified_application_namespace: str
simplified_experiment_namespace: str
simplified_workload_namespace: str
spec_name: str
unformatted_command: str
unformatted_command_without_logs: str
update_keys(extra_keys)[source]
workload_input_dir: str
workload_name: str
workload_namespace: str
workload_run_dir: str
workload_template_name: str
workspace: str
workspace_archives: str
workspace_configs: str
workspace_deployments: str
workspace_experiments: str
workspace_inputs: str
workspace_logs: str
workspace_name: str
workspace_root: str
workspace_shared: str
workspace_software: str
exception ramble.keywords.RambleKeywordError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Superclass for all errors to do with Ramble Keywords

class ramble.keywords.key_type(*values)

Bases: Enum

optional = 2
required = 3
reserved = 1
class ramble.keywords.output_level(*values)

Bases: Enum

key = 1
variable = 2

ramble.main module

This is the implementation of the Ramble command line executable.

In a normal Ramble installation, this is invoked from the bin/ramble script after the system path is set up.

class ramble.main.RambleArgumentParser(prog=None, usage=None, description=None, epilog=None, parents=[], formatter_class=<class 'argparse.HelpFormatter'>, prefix_chars='-', fromfile_prefix_chars=None, argument_default=None, conflict_handler='error', add_help=True, allow_abbrev=True, exit_on_error=True)[source]

Bases: ArgumentParser

add_command(cmd_name)[source]

Add one subcommand to this parser.

add_subparsers(**kwargs)[source]

Ensure that sensible defaults are propagated to subparsers

format_help(level='short')[source]
format_help_sections(level)[source]

Format help on sections for a particular verbosity level.

Parameters:

level (str) – ‘short’ or ‘long’ (more commands shown for long)

class ramble.main.RambleCommand(command_name)[source]

Bases: object

Callable object that invokes a ramble command (for testing).

Example usage:

list = RambleCommand('list')
list('hostname')

Use this to invoke Ramble commands directly from Python and check their output.

class ramble.main.RambleHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)[source]

Bases: RawTextHelpFormatter

add_arguments(actions)[source]
ramble.main.add_all_commands(parser)[source]

Add all ramble subcommands to the parser.

ramble.main.finish_parse_and_run(parser, cmd_name, main_args, workspace_format_error)[source]

Finish parsing after we know the command to run.

ramble.main.index_commands()[source]

create an index of commands by section for this help level

ramble.main.intro_by_level = {'long': 'Complete list of ramble commands:', 'short': 'These are common ramble commands:'}

intro text for help at different levels

ramble.main.levels = ['short', 'long']

help levels in order of detail (i.e., number of commands shown)

ramble.main.main(argv=None)[source]

This is the entry point for the Ramble command.

main() itself is just an error handler – it handles errors for everything in Ramble that makes it to the top level.

The logic is all in _main().

Parameters:

argv (list | None) – command line arguments, NOT including the executable name. If None, parses from sys.argv.

ramble.main.make_argument_parser(**kwargs)[source]

Create an basic argument parser without any subcommands added.

ramble.main.mock_repositories(objects)[source]
ramble.main.options_by_level = {'long': 'all', 'short': ['h', 'k', 'V', 'color', 'd']}

control top-level ramble options shown in basic vs. advanced help

ramble.main.ramble_working_dir = None

Recorded directory where ramble command was originally invoked

ramble.main.required_command_properties = ['level', 'section', 'description']

Properties that commands are required to set.

ramble.main.resolve_alias(cmd_name, cmd)[source]

Resolves aliases in the given command.

Parameters:

  • cmd_name – command name.

  • cmd – command line arguments.

Returns:

new command name and arguments.

ramble.main.section_descriptions = {'admin': 'administration', 'basic': 'query applications', 'config': 'configuration', 'developer': 'developer', 'help': 'more help', 'system': 'system'}

Longer text for each section, to show in help

ramble.main.section_order = {'basic': ['list', 'info', 'find']}

preferential command order for some sections (e.g., build pipeline is in execution order, not alphabetical)

ramble.main.send_warning_to_tty(message, *args)[source]

Redirects messages to ramble.util.logger.logger.warn.

ramble.main.set_working_dir()[source]

Change the working directory to getcwd, or ramble prefix if no cwd.

ramble.main.setup_main_options(args)[source]

Configure ramble globals based on the basic options.

ramble.main.stat_names = {'calls': (((1, -1),), 'call count'), 'cumtime': (((3, -1),), 'cumulative time'), 'cumulative': (((3, -1),), 'cumulative time'), 'filename': (((4, 1),), 'file name'), 'line': (((5, 1),), 'line number'), 'module': (((4, 1),), 'file name'), 'name': (((6, 1),), 'function name'), 'ncalls': (((1, -1),), 'call count'), 'nfl': (((6, 1), (4, 1), (5, 1)), 'name/file/line'), 'pcalls': (((0, -1),), 'primitive call count'), 'stdname': (((7, 1),), 'standard name'), 'time': (((2, -1),), 'internal time'), 'tottime': (((2, -1),), 'internal time')}

names of profile statistics

ramble.mirror module

This file contains code for creating ramble mirror directories. A mirror is an organized hierarchy containing specially named archive files. This enabled ramble to know where to find files in a mirror if the main server for a particular input is down. Or, if the computer where ramble is run is not connected to the internet, it allows ramble to download inputs directly from a mirror (e.g., on an intranet).

class ramble.mirror.Mirror(fetch_url, push_url=None, name=None)[source]

Bases: object

Represents a named location for storing input tarballs.

Mirrors have a fetch_url that indicate where and how artifacts are fetched from them, and a push_url that indicate where and how artifacts are pushed to them. These two URLs are usually the same.

display(max_len=0)[source]
property fetch_url
static from_dict(d, name=None)[source]
static from_json(stream, name=None)[source]
static from_yaml(stream, name=None)[source]
property name
property push_url
to_dict()[source]
to_json(stream=None)[source]
to_yaml(stream=None)[source]
class ramble.mirror.MirrorCollection(mirrors=None, scope=None)[source]

Bases: Mapping

A mapping of mirror names to mirrors.

display()[source]
static from_dict(d)[source]
lookup(name_or_url)[source]

Looks up and returns a Mirror.

If this MirrorCollection contains a named Mirror under the name [name_or_url], then that mirror is returned. Otherwise, [name_or_url] is assumed to be a mirror URL, and an anonymous mirror with the given URL is returned.

to_dict(recursive=False)[source]
to_json(stream=None)[source]
to_yaml(stream=None)[source]
exception ramble.mirror.MirrorError(msg, long_msg=None)[source]

Bases: RambleError

Superclass of all mirror-creation related errors.

class ramble.mirror.MirrorReference(cosmetic_path, global_path=None)[source]

Bases: object

A MirrorReference stores the relative paths where you can store a resource in a mirror directory.

The appropriate storage location is given by storage_path. The cosmetic_path property provides a reference that a human could generate themselves based on reading the details of the input.

A user can iterate over a MirrorReference object to get all the possible names that might be used to refer to the resource in a mirror; this includes names generated by previous naming schemes that are no-longer reported by storage_path or cosmetic_path.

property storage_path
class ramble.mirror.MirrorStats[source]

Bases: object

added(resource)[source]
already_existed(resource)[source]
error(resource)[source]
stats()[source]
ramble.mirror.add(name, url, scope)[source]

Add a named mirror in the given scope

ramble.mirror.mirror_archive_paths(fetcher, per_input_ref)[source]

Returns a MirrorReference object which keeps track of the relative storage path of the resource associated with the specified fetcher.

ramble.mirror.remove(name, scope)[source]

Remove the named mirror in the given scope

ramble.modkit module

modkit is a set of useful modules to import when writing modifiers

ramble.namespace module

class ramble.namespace.namespace[source]

Bases: object

Class of namespace variables

application = 'applications'
chained_experiments = 'chained_experiments'
command = 'command'
commands = 'commands'
compiler = 'compiler'
compiler_spec = 'compiler_spec'
custom_executables = 'custom_executables'
env_var = 'env_vars'
environments = 'environments'
exclude = 'exclude'
executable_injection = 'executable_injection'
executables = 'executables'
experiment = 'experiments'
external_env = 'external_env'
formatted_executables = 'formatted_executables'
include = 'include'
indentation = 'indentation'
inherit_variables = 'inherit_variables'
internals = 'internals'
join_separator = 'join_separator'
matrices = 'matrices'
matrix = 'matrix'
metadata = 'metadata'
modifiers = 'modifiers'
n_repeats = 'n_repeats'
package_manager = 'package_manager'
packages = 'packages'
pkg_spec = 'pkg_spec'
platform = 'platform'
prefix = 'prefix'
ramble = 'ramble'
software = 'software'
success = 'success_criteria'
system = 'system'
tables = 'tables'
tags = 'tags'
template = 'template'
variables = 'variables'
variants = 'variants'
version = 'version'
where = 'where'
workflow_manager = 'workflow_manager'
workload = 'workloads'
workspace = 'workspace'
zips = 'zips'

ramble.paths module

Defines paths that are part of Ramble’s directory structure.

Do not import other ramble modules here. This module is used throughout Ramble and should bring in a minimal number of external dependencies.

ramble.paths.bin_path: str = '/home/docs/checkouts/readthedocs.org/user_builds/ramble-dev-dapomeroy/checkouts/latest/lib/ramble/docs/_ramble_root/bin'

bin directory in the ramble prefix

ramble.paths.prefix: str = '/home/docs/checkouts/readthedocs.org/user_builds/ramble-dev-dapomeroy/checkouts/latest/lib/ramble/docs/_ramble_root'

This file lives in $prefix/lib/ramble/ramble/__file__

ramble.paths.ramble_root: str = '/home/docs/checkouts/readthedocs.org/user_builds/ramble-dev-dapomeroy/checkouts/latest/lib/ramble/docs/_ramble_root'

synonym for prefix

ramble.paths.ramble_script: str = '/home/docs/checkouts/readthedocs.org/user_builds/ramble-dev-dapomeroy/checkouts/latest/lib/ramble/docs/_ramble_root/bin/ramble'

The ramble script itself

ramble.paths.sbang_script: str = '/home/docs/checkouts/readthedocs.org/user_builds/ramble-dev-dapomeroy/checkouts/latest/lib/ramble/docs/_ramble_root/bin/sbang'

The sbang script in the ramble installation

ramble.paths.user_config_path: str = '/home/docs/.ramble'

User configuration location

ramble.pipeline module

class ramble.pipeline.AnalyzePipeline(workspace, filters, output_formats=None, upload=False, print_results=False, summary_only=False, fom_origin_types=None)[source]

Bases: Pipeline

Class for the analyze pipeline

name = 'analyze'
class ramble.pipeline.ArchivePipeline(workspace, filters, create_tar=False, archive_prefix=None, upload_url=None, include_secrets=False, archive_patterns=None)[source]

Bases: Pipeline

Class for the archive pipeline

name = 'archive'
class ramble.pipeline.ExecutePipeline(workspace, filters, executor='{batch_submit}', suppress_per_experiment_prints=True, suppress_run_header=False)[source]

Bases: Pipeline

class for the execute (on) pipeline

name = 'execute'
class ramble.pipeline.LogsPipeline(workspace, filters, first_only=False, suppress_per_experiment_prints=True, suppress_run_header=True)[source]

Bases: Pipeline

class for the logs pipeline

name = 'logs'
class ramble.pipeline.MirrorPipeline(workspace, filters, mirror_path=None)[source]

Bases: Pipeline

Class for the mirror pipeline

name = 'mirror'
class ramble.pipeline.Pipeline(workspace, filters)[source]

Bases: object

Base Class for all pipeline objects

property experiment_set
name = 'base'
run()[source]

Run the full pipeline

class ramble.pipeline.PushDeploymentPipeline(workspace, filters, create_tar=False, upload_url=None, deployment_name=None)[source]

Bases: Pipeline

class for the prepare-deployment pipeline

index_filename = 'index.json'
index_namespace = 'deployment_files'
legacy_object_repo_name = 'object_repo'
name = 'pushdeployment'
tar_extension = '.tar.gz'
class ramble.pipeline.PushToCachePipeline(workspace, filters, spack_cache_path=None)[source]

Bases: Pipeline

Class for the pushtocache pipeline

name = 'pushtocache'
class ramble.pipeline.SetupPipeline(workspace, filters)[source]

Bases: Pipeline

Class for the setup pipeline

name = 'setup'
ramble.pipeline.pipeline_class(name)[source]

Factory for determining a pipeline class from its name

class ramble.pipeline.pipelines(*values)

Bases: Enum

analyze = 1
archive = 2
execute = 6
logs = 8
mirror = 3
pushdeployment = 7
pushtocache = 5
setup = 4

ramble.pkgmankit module

pkgmankit is a set of useful modules to import when writing package managers

ramble.platkit module

ramble.renderer module

class ramble.renderer.RenderGroup(obj_type=None, action='create')[source]

Bases: object

copy_contents(in_group)[source]

Copy contents of in_group into self

from_dict(name_template, in_dict)[source]

Extract RenderGroup definitions from a dictionary

Dictionaries should follow the below format:

in_dict = {

‘variables’: {}, ‘zips’: {}, ‘matrix’: [], ‘matrices’: {} or [] }

Parameters:

  • name_template – The name template for the objects this group represents

  • in_dict – A dictionary representing the group definitions

Returns:

True if anything was extracted from the dictionary

Return type:

bool

class ramble.renderer.Renderer[source]

Bases: object

render_objects(render_group, exclude_where=None, ignore_used=True, fatal=True)[source]

Render objects based on the input variables and matrices

Internally collects all matrix and vector variables.

First, zips are created. Zips extract vector variables, and group them into a higher level name.

Next, matrices are processed. Matrices consume vector variables, or explicit zips.

Vectors in the same matrix are crossed, sibling matrices are zipped. All matrices are required to result in the same number of elements, but not be the same shape.

Matrix elements are only allowed to be the names of variables. These variables are required to be vectors.

After matrices are processed, any remaining vectors are zipped together. All vectors are required to be of the same size.

The resulting zip of vectors is then crossed with all of the matrices to build a final list of objects.

After processing the expansion logic, this function yields a dictionary of variable definitions, one for each object that would be rendered.

If n_repeats is defined in input variables, this function yields one base and n copies of the rendered variable dictionary.

Yields:

  • a dictionary of variables for single object definition

  • a Repeats object indicating if rendered object is a repeat and its index

ramble.repeats module

class ramble.repeats.Repeats[source]

Bases: object

Class to represent configuration of experiment repeats

set_repeat_index(index)[source]

Set the index for an instance of a repeated experiment

set_repeats(is_repeat_base, n_repeats)[source]

Set if this instance will use repeats

Parameters:

  • is_repeat_base (bool) – True if this is the base experiment of a repeat set, False for singletons and for repeats

  • n_repeats (int) – 0 for singletons and repeats, positive integer for base experiment

ramble.reports module

class ramble.reports.ComparisonPlot(spec, normalize, report_dir_path, exp_results, logx, logy, split_by, simplify_names=False, where=None)[source]

Bases: PlotGenerator

draw(perf_measure, scale_var, series, pdf_report)[source]
generate_plot_data(pdf_report)[source]
plot_type = 'comparison'
class ramble.reports.FomPlot(spec, normalize, report_dir_path, exp_results, logx, logy, split_by, simplify_names=False, where=None)[source]

Bases: PlotGenerator

draw(perf_measure, scale_var, series, unit, pdf_report)[source]
generate_plot_data(pdf_report)[source]
plot_type = 'foms'
class ramble.reports.MultiLinePlot(spec, normalize, report_dir_path, exp_results, logx, logy, split_by, simplify_names=False, where=None)[source]

Bases: ScalingPlotGenerator

default_better()[source]
draw_multiline(perf_measure, scale_var, pdf_report, y_label)[source]
generate_plot_data(pdf_report)[source]

Creates a dataframe for plotting line charts with scaling var on x axis, and performance variable on y axis.

normalize_data(data, scale_to_index=True, to_col='normalized_fom_value', from_col='fom_value')[source]
plot_type = 'multi_line'
series_to_plot: List[str] = []
class ramble.reports.PlotFactory[source]

Bases: object

create_plot_generator(args, report_dir_path, exp_results)[source]
determine_plot_type(args)[source]
class ramble.reports.PlotGenerator(spec, normalize, report_dir_path, exp_results, logx, logy, split_by, simplify_names=False, where=None)[source]

Bases: object

add_minmax_data(selected_data, min_data, max_data, scale_var)[source]

When using summary statistics from repeats, adds columns fom_value_min and fom_value_max to the selected data.

add_to_inventory(filename)[source]

Adds a filename to the inventory.

Parameters:

filename – filename to add to inventory.

draw(perf_measure, scale_var, series, pdf_report, y_label=None)[source]
draw_filler(perf_measure, scale_var, series, exception, pdf_report)[source]

Draws a filler figure in cases where a chart cannot be drawn due to errors.

get_inventory_path()[source]
normalize_data(data, scale_to_index=False, to_col='normalized_fom_value', from_col='fom_value')[source]
validate_spec(chart_spec, result_index)[source]

Validates that the FOMs and variables in the chart spec are in the results data.

write(fig, filename, pdf_report)[source]
write_inventory()[source]
class ramble.reports.ReportVars(*values)[source]

Bases: Enum

APP_NAME = 'application_name'
BETTER_DIRECTION = 'better_direction'
CONTEXT_NAME = 'context_name'
EXP_NAME = 'experiment_name'
EXP_NS = 'experiment_namespace'
FOM_NAME = 'fom_name'
FOM_ORIGIN = 'fom_origin'
FOM_ORIGIN_TYPE = 'fom_origin_type'
FOM_UNITS = 'fom_units'
FOM_VALUE = 'fom_value'
FOM_VALUE_MAX = 'fom_value_max'
FOM_VALUE_MIN = 'fom_value_min'
IDEAL_PERF_VALUE = 'ideal_perf_value'
NORMALIZED_FOM_VALUE = 'normalized_fom_value'
SERIES = 'series'
WL_NAME = 'workload_name'
WL_NS = 'workload_namespace'
class ramble.reports.ScalingPlotGenerator(spec, normalize, report_dir_path, exp_results, logx, logy, split_by, simplify_names=False, where=None)[source]

Bases: PlotGenerator

add_idealized_data(raw_results, selected_data)[source]
default_better()[source]
generate_plot_data(pdf_report)[source]

Creates a dataframe for plotting line charts with scaling var on x axis, and performance variable on y axis.

prep_draw(perf_measure, scale_var)[source]
validate_data(data)[source]
validate_spec(chart_spec, result_index)[source]

Validates that the FOMs and variables in the chart spec are in the results data.

class ramble.reports.StrongScalingPlot(spec, normalize, report_dir_path, exp_results, logx, logy, split_by, simplify_names=False, where=None)[source]

Bases: ScalingPlotGenerator

add_idealized_data(raw_results, selected_data)[source]
default_better()[source]
draw(perf_measure, scale_var, series, pdf_report)[source]
normalize_data(data, scale_to_index=True, to_col='normalized_fom_value', from_col='fom_value')[source]
plot_type = 'strong_scaling'
class ramble.reports.WeakScalingPlot(spec, normalize, report_dir_path, exp_results, logx, logy, split_by, simplify_names=False, where=None)[source]

Bases: ScalingPlotGenerator

add_idealized_data(raw_results, selected_data)[source]
draw(perf_measure, scale_var, series, pdf_report)[source]
plot_type = 'weak_scaling'
ramble.reports.clean_redundant_prefixes(name, application_name, workload_name)[source]

Strip application and workload names from a name string, handling case, dots, underscores, and hyphens.

ramble.reports.extract_data(experiments: List[dict], foms: List[str], variables: List[str], where_query=None)[source]

Extracts data from the experiments dicts and returns it as a Pandas DataFrame.

Parameters:

  • experiments – List of experiment dictionaries containing results to extract

  • foms – List of FOMs to extract from experiments

  • variables – List of variables to extract from experiments

  • where_query – Pandas query to constrain results

Returns:

Pandas DataFrame containing extracted data

ramble.reports.filter_exp_results(experiments: list)[source]

Filters a list of experiment results to remove failed experiments and duplicate data.

When repeats are used, this removes individual repeats and returns only the summary statistics.

ramble.reports.generate_result_index(experiments: list, all_vars=False, where_query=None)[source]

Creates an index from the results in the list of experiments

Index format is: {

“applications”: {
application_name: {
workload: {

“Contexts”: set(), “FOMs”: set(), “Template Variables”: set(),

}

}

} “modifiers”: {

modifier_name: {

“Contexts”: set(), “FOMs”: set(),

}

(all other object types)

}

ramble.reports.get_all_foms(result_index)[source]
ramble.reports.get_all_vars(result_index)[source]
ramble.reports.get_common_stripped_prefix(original_values, simplified_values)[source]

Find the longest common prefix stripped from original_values to get simplified_values.

ramble.reports.get_direction_suffix(self)[source]
ramble.reports.get_reports_path()[source]

Returns current directory of ramble-created reports

ramble.reports.is_key_to_skip(key_name: str)[source]

Check if a results dict key should be skipped for indexing and analysis.

The purpose of this is to ignore non-variables and reduce clutter in the results index. Some values in the results index, like paths and commands, have limited utility for analysis or are derived from variables that are available separately.

ramble.reports.is_repeat_child(experiment)[source]
ramble.reports.make_report(experiments: list, ws_name, args)[source]
ramble.reports.simplify_experiment_names(df, index_col=None)[source]

Simplify the index or a column of a dataframe by stripping redundant application/workload names and common prefixes/suffixes.

ramble.reports.simplify_names(names)[source]

Simplify a list of dot-separated names by stripping the longest common prefix and suffix parts.

ramble.reports.to_numeric_if_possible(series)[source]

Try to convert a Pandas series to numeric, or return the series unchanged.

ramble.repository module

exception ramble.repository.BadRepoError(message: str, long_message: str | None = None)[source]

Bases: RepoError

Raised when repo layout is invalid.

exception ramble.repository.FailedConstructorError(name, exc_type, exc_obj, exc_tb, object_type=None)[source]

Bases: RepoError

Raised when an object’s class constructor fails.

class ramble.repository.FastObjectChecker(objects_path, object_file_name, object_type)[source]

Bases: Mapping

Cache that maps object names to the stats obtained on the ‘.py’ files associated with them.

For each repository a cache is maintained at class level, and shared among all instances referring to it. Update of the global cache is done lazily during instance initialization.

invalidate()[source]

Regenerate cache for this checker.

last_mtime()[source]
exception ramble.repository.IndexError(message: str, long_message: str | None = None)[source]

Bases: RepoError

Raised when there’s an error with an index.

class ramble.repository.Indexer(object_type=ObjectTypes.applications)[source]

Bases: object

Adaptor for indexes that need to be generated when repos are updated.

create()[source]
needs_update(pkg)[source]

Whether an update is needed when the application file hasn’t changed.

Returns:

True if this application needs its index

updated, False otherwise.

Return type:

(bool)

We already automatically update indexes when object files change, but other files (like patches) may change underneath the object file. This method can be used to check additional object-specific files whenever they’re loaded, to tell the RepoIndex to update the index just for that object.

abstractmethod read(stream)[source]

Read this index from a provided file object.

set_object_type(object_type)[source]
abstractmethod update(obj_fullname)[source]

Update the index in memory with information about an object.

abstractmethod write(stream)[source]

Write the index to a file object.

exception ramble.repository.InvalidNamespaceError(message: str, long_message: str | None = None)[source]

Bases: RepoError

Raised when an invalid namespace is encountered.

ramble.repository.NOT_PROVIDED = <object object>

Guaranteed unused default value for some functions.

exception ramble.repository.NoRepoConfiguredError(message: str, long_message: str | None = None)[source]

Bases: RepoError

Raised when there are no repositories configured.

class ramble.repository.ObjectNamespace(namespace)[source]

Bases: ModuleType

Allow lazy loading of modules.

class ramble.repository.ObjectTypes(*values)

Bases: Enum

applications = 1
base_applications = 8
base_classes = 7
base_modifiers = 9
base_package_managers = 10
base_platforms = 13
base_systems = 12
base_workflow_managers = 11
modifiers = 2
package_managers = 3
platforms = 6
systems = 5
workflow_managers = 4
class ramble.repository.Repo(root, object_type=ObjectTypes.applications)[source]

Bases: object

Class representing a object repository in the filesystem.

Each object repository must have a top-level configuration file called repo.yaml.

Currently, repo.yaml this must define:

namespace:

A Python namespace where the repository’s objects should live.

all_object_classes()[source]

Iterator over all object classes in the repository.

Use this with care, because loading objects is slow.

all_object_names()[source]

Returns a sorted list of all object names in the Repo.

all_objects()[source]

Iterator over all objects in the repository.

Use this with care, because loading objects is slow.

dirname_for_object_name(obj_name)[source]

Get the directory name for a particular object. This is the directory that contains its object.py file.

dump_provenance(spec, path)[source]

Dump provenance information for a spec to a particular path.

This dumps the object file. Raises UnknownObjectError if not found.

exists(obj_name)[source]

Whether a object with the supplied name exists.

filename_for_object_name(obj_name)[source]

Get the filename for the module we should load for a particular object. objects for a Repo live in $root/<object_name>/<object_type>.py

This will return a proper <object_type>.py path even if the object doesn’t exist yet, so callers will need to ensure the object exists before importing.

find_module(fullname, path=None)[source]

Python find_module import hook.

Returns this Repo if it can load the module; None if not.

get(spec)[source]

Returns the object associated with the supplied spec.

get_obj_class(obj_name)[source]

Get the class for the object out of its module.

First loads (or fetches from cache) a module for the object. Then extracts the object class from the module according to Ramble’s naming convention.

property index

Construct the index for this repo lazily.

is_prefix(fullname)[source]

True if fullname is a prefix of this Repo’s namespace.

last_mtime()[source]

Time a object file in this repo was last updated.

load_module(fullname)[source]

Python importer load hook.

Tries to load the module; raises an ImportError if it can’t.

object_path(spec)[source]
objects_with_tags(*tags)[source]
purge()[source]

Clear entire object instance cache.

real_name(import_name)[source]

Allow users to import Ramble objects using Python identifiers.

A python identifier might map to many different Ramble object names due to hyphen/underscore ambiguity.

Easy example:

num3proxy -> 3proxy

Ambiguous:

foo_bar -> foo_bar, foo-bar

More ambiguous:

foo_bar_baz -> foo_bar_baz, foo-bar-baz, foo_bar-baz, foo-bar_baz

property tag_index

Index of tags and which objects they’re defined on.

exception ramble.repository.RepoError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Superclass for repository-related errors.

class ramble.repository.RepoIndex(object_checker, namespace, object_type=ObjectTypes.applications)[source]

Bases: object

Container class that manages a set of Indexers for a Repo.

This class is responsible for checking objects in a repository for updates (using FastObjectChecker) and for regenerating indexes when they’re needed.

Indexers should be added to the RepoIndex using add_index(name, indexer), and they should support the interface defined by Indexer, so that the RepoIndex can read, generate, and update stored indices.

Generated indexes are accessed by name via __getitem__().

add_indexer(name, indexer)[source]

Add an indexer to the repo index.

Parameters:

  • name (str) – name of this indexer

  • indexer (object) – an object that supports create(), read(), write(), and get_index() operations

class ramble.repository.RepoLoader(fullname, repo, object_name)[source]

Bases: _PrependFileLoader

Loads a Python module associated with a object in specific repository

is_package(fullname)[source]

Concrete implementation of InspectLoader.is_package by checking if the path returned by get_filename has a filename of ‘__init__.py’.

class ramble.repository.RepoPath(*repos, object_type=ObjectTypes.applications)[source]

Bases: object

A RepoPath is a list of repos that function as one.

It functions exactly like a Repo, but it operates on the combined results of the Repos in its list instead of on a single object repository.

Parameters:

repos (list) – list Repo objects or paths to put in this RepoPath

all_object_classes()[source]
all_object_names()[source]

Return all unique object names in all repositories.

all_objects()[source]
dirname_for_object_name(obj_name)[source]
dump_provenance(spec, path)[source]

Dump provenance information for a spec to a particular path.

This dumps the object file and any associated patch files. Raises UnknownObjectError if not found.

exists(obj_name)[source]

Whether object with the give name exists in the path’s repos.

Note that virtual objects do not “exist”.

filename_for_object_name(obj_name)[source]
find_module(fullname, path=None)[source]

Implements precedence for overlaid namespaces.

Loop checks each namespace in self.repos for objects, and also handles loading empty containing namespaces.

property first_repo

Get the first repo in precedence order.

get(spec)[source]

Returns the object associated with the supplied spec.

get_full_namespace(namespace)[source]

Returns the full namespace of a repository, given its relative one.

get_obj_class(obj_name)[source]

Find a class for the spec’s object and return the class object.

get_repo(namespace, default=<object object>)[source]

Get a repository by namespace.

Parameters:

namespace – Look up this namespace in the RepoPath, and return it if found.

Optional Arguments:

default:

If default is provided, return it when the namespace isn’t found. If not, raise an UnknownNamespaceError.

in_path(maybe_obj_path)[source]

Whether the path belongs to any of the repos.

last_mtime()[source]

Time a object file in this repo was last updated.

load_module(fullname)[source]

Handles loading container namespaces when necessary.

See Repo for how actual object modules are loaded.

objects_with_tags(*tags)[source]
put_first(repo)[source]

Add repo first in the search path.

put_last(repo)[source]

Add repo last in the search path.

remove(repo)[source]

Remove a repo from the search path.

repo_for_obj(spec)[source]

Given a spec, get the repository for its object.

class ramble.repository.ReposFinder(object_type=ObjectTypes.applications)[source]

Bases: object

MetaPathFinder class that loads a Python module corresponding to an object

Return a loader based on the inspection of the current global repository list.

compute_loader(fullname)[source]
find_spec(fullname, python_path, target=None)[source]
class ramble.repository.RepositoryNamespace(namespace)[source]

Bases: ModuleType

Allow lazy loading of modules.

class ramble.repository.RepositoryNamespaceLoader[source]

Bases: object

create_module(spec)[source]
exec_module(module)[source]
class ramble.repository.TagIndex(object_type=ObjectTypes.applications)[source]

Bases: Mapping

Maps tags to list of applications.

static from_json(stream, object_type)[source]
to_json(stream)[source]
update_object(obj_name)[source]

Updates an object in the tag index.

Parameters:

  • obj_name (str) – name of the object to be removed from the

  • index

class ramble.repository.TagIndexer(object_type=ObjectTypes.applications)[source]

Bases: Indexer

Lifecycle methods for a TagIndex on a Repo.

read(stream)[source]

Read this index from a provided file object.

update(obj_fullname)[source]

Update the index in memory with information about an object.

write(stream)[source]

Write the index to a file object.

exception ramble.repository.UnknownEntityError(message: str, long_message: str | None = None)[source]

Bases: RepoError

Raised when we encounter a object ramble doesn’t have.

exception ramble.repository.UnknownNamespaceError(namespace)[source]

Bases: UnknownEntityError

Raised when we encounter an unknown namespace

exception ramble.repository.UnknownObjectError(name, repo=None, object_type='Object')[source]

Bases: UnknownEntityError

Raised when we encounter an object ramble doesn’t have.

ramble.repository.additional_repository(repository, object_type=ObjectTypes.applications)[source]

Adds temporarily a repository to the default one.

Parameters:

repository – repository to be added

ramble.repository.all_object_names(object_type=ObjectTypes.applications)[source]

Convenience wrapper around ramble.repository.all_object_names().

ramble.repository.autospec(function)[source]

Decorator that automatically converts the first argument of a function to a Spec.

ramble.repository.create(configuration, object_type=ObjectTypes.applications)[source]

Create a RepoPath from a configuration object.

Parameters:

configuration (ramble.config.Configuration) – configuration object

ramble.repository.create_or_construct(path, namespace=None)[source]

Create a repository, or just return a Repo if it already exists.

ramble.repository.create_repo(root, namespace=None, subdir='applications', object_type=ObjectTypes.applications, unified_repo=True)[source]

Create a new repository in root with the specified namespace.

If the namespace is not provided, use basename of root. Return the canonicalized path and namespace of the created repository.

ramble.repository.get(spec, object_type=ObjectTypes.applications)[source]

Convenience wrapper around ramble.repository.get().

ramble.repository.get_base_class(spec)[source]

Convenience wrapper around ramble.repository.get_obj_class() for base classes.

ramble.repository.get_obj_class(spec, object_type=ObjectTypes.applications)[source]

Convenience wrapper around ramble.repository.get_obj_class().

ramble.repository.list_object_files(obj_inst, object_type)[source]

List object file paths of the given object along the inheritance chain.

This is currently used by ramble deployment to copy relevant files to create a self-contained repo.

ramble.repository.set_path(repo, object_type=ObjectTypes.applications)[source]

Set the path singleton to a specific value.

Overwrite path and register it as an importer in sys.meta_path if it is a Repo or RepoPath.

ramble.repository.use_repositories(*paths_and_repos, object_type=ObjectTypes.applications)[source]

Use the repositories passed as arguments within the context manager.

Parameters:

*paths_and_repos – paths to the repositories to be used, or already constructed Repo objects

Returns:

Corresponding RepoPath object

ramble.results_table module

class ramble.results_table.ResultsAutoColumn(conf_dict)[source]

Bases: object

Class representing a template for auto-generated columns

class ramble.results_table.ResultsColumn(conf_dict)[source]

Bases: object

Class representing a single column in a results table

col_name(app_inst)[source]

Expand this columns name based on the current experiment

Parameters:

app_inst – Instance of an application class to expand name with

Returns:

Expanded column name

Return type:

(str)

extract_value(app_inst, extra_vars=None)[source]

Extract this column’s value based on an application instance and other column values

Parameters:

  • app_inst – Instance of an application class

  • extra_vars – Dictionary containing additional variables to expand with

  • iteration (Union[None, str]) – The iteration to search for.

  • instance (Union[None, str]) – The instance to search for.

class ramble.results_table.ResultsTable(conf_dict)[source]

Bases: object

A single results table based on the tables.py schema

add_where(expressions)[source]

Add a where expression to this table

Parameters:

expression (Union[List[str], str]) – The regular expression to search for.

extract_row(app_inst)[source]

Extract a row of data from an experiment

Parameters:

app_inst – Instance of an application class to extract data from

includes_experiment(app_inst)[source]

Determine if an experiment is included in this table.

Parameters:

app_inst – Instance of an application class

Returns:

True if the app_inst is included in table, False otherwise

Return type:

(bool)

render(app_inst)[source]
table_name(app_inst)[source]

Determine the name for this table, based on a given experiment

Parameters:

app_inst – Instance of an application class

Returns:

Name of table

Return type:

(str)

to_csv(directory, timestamp)[source]

CSV converter for results table

Parameters:

  • directory (str) – Directory to write tabular data into

  • timestamp (str) – Timestamp to apply to table output files

class ramble.results_table.ResultsTables[source]

Bases: object

Class representing a set of results tables

add_table_template(table_conf)[source]

Construct a new results table, and add to this set of tables

Parameters:

table_conf (dict) – Dictionary configuration of table, assuming table schema from tables.py

Returns:

New table instance

Return type:

(ResultsTable)

build_tables(experiment_set, filters)[source]

Extract data for each table in this set

Parameters:

  • experiment_set – Set of experiments to extract data from

  • filters – Filter object to downselect experiments

property num_tables
output_tables(directory, timestamp)[source]

Output tabular data for each of the tables in this set

Parameters:

  • directory (str) – Directory to write tables into

  • timestamp (str) – Timestamp to apply to table output files

ramble.software_environments module

class ramble.software_environments.ExternalEnvironment(name: str, name_or_path: str)[source]

Bases: SoftwareEnvironment

Class representing an externally defined software environment

exception ramble.software_environments.RambleSoftwareEnvironmentError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Super class for all software environment errors

class ramble.software_environments.RenderedEnvironment(name: str, package_manager)[source]

Bases: SoftwareEnvironment

Class representing an already rendered software environment

property package_manager_name
class ramble.software_environments.RenderedExternalEnvironment(name: str, name_or_path: str, package_manager)[source]

Bases: ExternalEnvironment

Class representing a rendered externally defined software environment

property package_manager_name
class ramble.software_environments.RenderedPackage(name: str, pkg_info: dict, package_manager, spec: str, compiler: str = '', compiler_spec: str = '')[source]

Bases: SoftwarePackage

Class representing an already rendered software package

info(indent: int = 0, verbosity: int = 0, color_level: int = 0, only_used: bool = True)[source]

String representation of package information

Parameters:

  • indent (int) – Number of spaces to indent lines with

  • verbosity (int) – Verbosity level

  • color_level (int) – Nested level for coloring

  • only_used (bool) – Whether to only track used info (True) or all info (False)

Returns:

String representation of this package

Return type:

(str)

spec_str(all_packages=None, compiler=False)[source]

Return a spec string for this software package

Parameters:

  • all_packages (dict) – Dictionary of all package definitions. Used to look up compiler packages.

  • compiler (bool) – True of this package is used as a compiler for another package. False if this is just a primary package. Toggles returning compiler_spec vs. spec in case they are different.

Returns:

String representation of the spec for this package definition

Return type:

(str)

class ramble.software_environments.SoftwareEnvironment(name: str)[source]

Bases: object

Class representing a single software environment

add_package(package: SoftwarePackage)[source]

Add a package definition to this environment

Parameters:

package (SoftwarePackage) – Package object

info(indent: int = 0, verbosity: int = 0, color_level: int = 0, only_used: bool = True)[source]

Software environment information

Parameters:

  • indent (int) – Number of spaces to inject as indentation

  • verbosity (int) – Verbosity level

  • color_level (int) – Nested level for coloring

  • only_used (bool) – Whether to only track used info (True) or all info (False)

Returns:

information of this environment

Return type:

(str)

property is_used

Determine if environment is used or not

Returns:

Whether environment is used or not

Return type:

(bool)

mark_used()[source]

Mark this environment (and all of its packages) as used

class ramble.software_environments.SoftwareEnvironments(workspace)[source]

Bases: object

Class representing a group of software environments

add_spec_to_environment(environment: SoftwareEnvironment, spec: SoftwareSpec, expander: Expander, package_manager)[source]

Add a spec to a given environment

Creates a new template / rendered package (if needed) from the input spec, and adds to the template and rendered environment as a package in the environment.

Parameters:
compiler_specs_for_environment(environment: RenderedEnvironment)[source]

Iterator over compiler specs for a given environment

Assumes all compilers have been defined via self.define_compiler_packages()

Parameters:

environment (SoftwareEnvironment) – Environment to extract compiler specs for

Yields:

(str) Package spec string for each compiler (str) Compiler spec string for each compiler

define_compiler_packages(environment: RenderedEnvironment, expander: Expander)[source]

Define packages for compilers in this environment

If compilers referenced by (environment) are not defined, create definitions for them to properly create compiler specs.

Parameters:
info(indent: int = 0, verbosity: int = 0, color_level: int = 0, only_used: bool = True)[source]

Information for all packages and environments

Parameters:

  • indent (int) – Number of spaces to indent lines with

  • verbosity (int) – Verbosity level

  • color_level (int) – Nested level for coloring

  • only_used (bool) – Whether to only track used info (True) or all info (False)

Returns:

Representation of all packages and environments

Return type:

(str)

package_specs_for_environment(environment: SoftwareEnvironment)[source]

Iterator over package specs for a given environment

Assumes all compilers have been defined via self.define_compiler_packages()

Parameters:

environment (SoftwareEnvironment) – Environment to extract package specs for

Yields:

(str) Spec string for each package

render_environment(env_name: str, expander: Expander, package_manager, require=True)[source]

Render an environment needed by an experiment

Parameters:

  • env_name (str) – Name of environment needed by the experiment

  • expander (ramble.expander.Expander) – Expander object from the experiment

  • package_manager – Package manager the environment is rendered with

Returns:

Reference to software environment for

the experiment

Return type:

(SoftwareEnvironment)

unused_environments()[source]

Iterator over environment templates that do not have any rendered environments

Yields:

(TemplateEnvironment) Each unused template environment in this group

unused_packages()[source]

Iterator over package templates that do not have any rendered packages

Yields:

(TemplatePackage) Each unused template package in this group

use_environment(package_manager, env_name)[source]

Mark an environment as used.

Given a package manager object and the name of a rendered environment, mark the environment as used. This allows the info method to only print information about used packages and environments.

Parameters:

  • package_manager – Reference to a package manager object

  • env_name (str) – Name of the rendered environment to mark as used

class ramble.software_environments.SoftwarePackage(name: str, pkg_info: dict)[source]

Bases: object

Class to represent a single software package

info(indent: int = 0, verbosity: int = 0, color_level: int = 0, only_used: bool = True)[source]

String representation of package information

Parameters:

  • indent (int) – Number of spaces to indent lines with

  • verbosity (int) – Verbosity level

  • color_level (int) – Nested level for coloring

  • only_used (bool) – Whether to only track used info (True) or all info (False)

Returns:

String representation of this package

Return type:

(str)

property is_used

Return if this package is used or not

Returns:

Whether package is used or not

Return type:

(bool)

mark_used()[source]

Mark this package a used

spec_str(all_packages=None, compiler=False)[source]

Return a spec string for this software package

Parameters:

  • all_packages (dict | None) – Dictionary of all package definitions. Used to look up compiler packages.

  • compiler (bool) – True of this package is used as a compiler for another package. False if this is just a primary package. Toggles returning compiler_spec vs. spec in case they are different.

Returns:

String representation of the spec for this package definition

Return type:

(str)

class ramble.software_environments.TemplateEnvironment(name: str)[source]

Bases: SoftwareEnvironment

Class representing a template software environment

add_package_name(package: str = '')[source]
add_rendered_environment(environment: RenderedEnvironment, all_environments: dict, all_packages: dict, pm_name: str)[source]

Add a rendered environment to this template

Parameters:

  • environment (RenderedEnvironment) – Reference to rendered environment

  • all_environments (dict) – Dictionary containing all environments

  • all_packages (dict) – Dictionary containing all packages

  • pm_name (str) – Name of the associated package manager

info(indent: int = 0, verbosity: int = 0, color_level: int = 0, only_used: bool = True)[source]

Software environment information

Parameters:

  • indent (int) – Number of spaces to inject as indentation

  • verbosity (int) – Verbosity level

  • color_level (int) – Nested level for coloring

  • only_used (bool) – Whether to only track used info (True) or all info (False)

Returns:

information of this environment

Return type:

(str)

property is_used

Determine if TemplateEnvironment is used or not

Returns:

Whether template environment is used or not

Return type:

(bool)

render_environment(expander: Expander, all_package_templates: dict, all_packages: dict, package_manager)[source]

Render a SoftwareEnvironment from this TemplateEnvironment

Parameters:

  • expander (ramble.expander.Expander) – Expander object to use when rendering

  • all_packages (dict) – All package definitions

  • package_manager – Package manager the environment is rendered with

Returns:

(RenderedEnvironment) Reference to the rendered SoftwareEnvironment

class ramble.software_environments.TemplatePackage(name: str, pkg_info: dict)[source]

Bases: SoftwarePackage

Class representing a template software package

add_rendered_package(new_package: RenderedPackage, all_packages: dict, pm_name: str)[source]

Add a rendered package to this template’s list of rendered packages

Parameters:

  • new_package (SoftwarePackage) – New package definition to add

  • all_packages (dict) – Dictionary of all package definitions

  • pm_name (str) – The name of the package manager used for the package

info(indent: int = 0, verbosity: int = 0, color_level: int = 0, only_used: bool = True)[source]

String representation of package information

Parameters:

  • indent (int) – Number of spaces to indent lines with

  • verbosity (int) – Verbosity level

  • color_level (int) – Nested level for coloring

  • only_used (bool) – Whether to only track used info (True) or all info (False)

Returns:

String representation of this package

Return type:

(str)

property is_used

Determine if template package is used

Iterate over all packages in this template, and determine if any are used.

Returns:

Whether this template contains any used packages or not

Return type:

(bool)

render_package(expander: Expander, package_manager)[source]

Render a SoftwarePackage from this TemplatePackage

Parameters:

expander (ramble.expander.Expander) – Expander to use to render a package from this template

Returns:

Rendered SoftwarePackage

Return type:

(SoftwarePackage)

ramble.software_info module

class ramble.software_info.SoftwareInfo(name='', version='unknown', compiler='', compiler_version='', target='', variants='')[source]

Bases: object

Represents information about a software build configuration (standard class).

to_dict()[source]
to_version_text()[source]

ramble.spec module

class ramble.spec.Spec(spec_like=None)[source]

Bases: object

cformat(*args, **kwargs)[source]

Same as format, but color defaults to auto instead of False.

copy()[source]
format(format_string='{name}', **kwargs)[source]

Prints out particular pieces of a spec, depending on what is in the format string.

Using the {attribute} syntax, any field of the spec can be selected. Those attributes can be recursive.

Commonly used attributes of the Spec for format strings include:

name
Parameters:

format_string (str) – string containing the format to be expanded

Keyword Arguments:

  • color (bool) – True if returned string is colored

  • transform (dict) – maps full-string formats to a callable that accepts a string and returns another one

property fullname
exception ramble.spec.SpecFormatStringError(message: str, long_message: str | None = None)[source]

Bases: SpecError

Called for errors in Spec format strings.

ramble.stage module

class ramble.stage.DIYStage(path)[source]

Bases: object

Simple class that allows any directory to be a ramble input stage. Consequently, it does not expect or require that the source path adhere to the standard directory naming convention.

cache_local()[source]
check()[source]
create()[source]
destroy()[source]
expand_archive()[source]
property expanded

Returns True since the source_path must exist.

fetch(*args, **kwargs)[source]
managed_by_ramble = False
restage()[source]
class ramble.stage.InputStage(url_or_fetch_strategy, name=None, path=None, mirror_paths=None, keep=True, lock=True, search_fn=None)[source]

Bases: object

Manages a stage directory for containing input files.

A Stage object is a context manager that handles a directory where some workload inputs are downloaded. It handles fetching inputs, either as an archive to be expanded or by checking it out of a repository. A stage’s lifecycle looks like this:

with InputStage() as stage:      # Context manager creates the stage
                            # directory
    stage.fetch()           # Fetch an input archive into the stage.
    stage.expand_archive()  # Expand the archive into input_path.

Because the input files are needed to execute various experiments, the stage is retained by default.

You can also use the stage’s create/destroy functions manually, like this:

stage = InputStage()
try:
    stage.create()          # Explicitly create the stage directory.
    stage.fetch()           # Fetch a source archive into the stage.
    stage.expand_archive()  # Expand the archive into source_path.
    <install>               # Build and install the archive.
                            # (handled by user of Stage)
finally:
    stage.destroy()         # Explicitly destroy the stage directory.

InputStages are required to be named by default. The name should match a workload namespace.

property archive_file

Path to the source archive within this stage directory.

cache_local()[source]
cache_mirror(mirror, stats)[source]

Perform a fetch if the resource is not already cached

Parameters:
check()[source]

Check the downloaded archive against a checksum digest. No-op if this stage checks code out of a repository.

create()[source]

Ensures the top-level (config:build_stage) directory exists.

destroy()[source]

Removes this stage directory.

expand_archive()[source]

Changes to the stage directory and attempt to expand the downloaded archive. Fail if the stage is not set up or if the archive is not yet downloaded.

property expanded

Returns True if source path expanded; else False.

property expected_archive_files

Possible archive file paths.

fetch(mirror_only=False, err_msg=None)[source]

Retrieves the code or archive

Parameters:

  • mirror_only (bool) – only fetch from a mirror

  • err_msg (str | None) – the error message to display if all fetchers fail or None for the default fetch failure message

managed_by_ramble = True
restage()[source]

Removes the expanded archive path if it exists, then re-expands the archive.

property save_filename
set_subdir(subdir_name)[source]
property source_path

Returns the well-known source directory path.

stage_locks: Dict[str, Lock] = {}

Most staging is managed by Ramble. DIYStage is one exception.

class ramble.stage.ResourceStage(url_or_fetch_strategy, root, resource, **kwargs)[source]

Bases: InputStage

expand_archive()[source]

Changes to the stage directory and attempt to expand the downloaded archive. Fail if the stage is not set up or if the archive is not yet downloaded.

restage()[source]

Removes the expanded archive path if it exists, then re-expands the archive.

exception ramble.stage.RestageError(message: str, long_message: str | None = None)[source]

Bases: StageError

“Error encountered during restaging.

class ramble.stage.StageComposite[source]

Bases: Composite

Composite for Stage type objects. The first item in this composite is considered to be the root workload, and operations that return a value are forwarded to it.

property archive_file
property expanded
property path
property source_path
exception ramble.stage.StageError(message: str, long_message: str | None = None)[source]

Bases: RambleError

“Superclass for all errors encountered during staging.

exception ramble.stage.StagePathError(message: str, long_message: str | None = None)[source]

Bases: StageError

“Error encountered with stage path.

ramble.stage.ensure_access(file)[source]

Ensure we can access a directory and die with an error if we can’t.

ramble.success_criteria module

class ramble.success_criteria.ScopedCriteriaList[source]

Bases: object

A scoped list of success criteria

This object represents a list of success criteria. The criteria are scoped based on which portion of a workspace they are defined in.

Possible scopes are:

  • object_definitions

  • application

  • workload

  • experiment

To see if success was met, all criteria will be checked and are AND-ed together.

add_criteria(scope, name, mode, *args, owning_object=None, **kwargs)[source]
all_criteria()[source]
find_criteria(name)[source]
flush_scope(scope)[source]

Remove criteria within a scope, and lower level scopes

Scope to be purged are defined in self._flush_scopes.

passed()[source]
reset()[source]
validate_scope(scope)[source]
class ramble.success_criteria.SuccessCriteria(name, mode, match=None, file='{log_file}', fom_name=None, fom_context='null', formula=None, anti_match=None, owning_object=None)[source]

Bases: object

A single success criteria object

This object represents a single criteria for success for a Ramble experiment.

anti_matched(test=None)[source]
mark_anti_found()[source]
mark_found()[source]
ok()[source]
passed(test=None, app_inst=None, fom_values=None)[source]
reset()[source]

ramble.syskit module

ramble.uploader module

class ramble.uploader.BigQueryUploader[source]

Bases: Uploader

Class to handle upload of FOMs to BigQuery

chunked_upload(table_id, data, uri=None)[source]

Abstract method for chunked uploads. Must be implemented by subclasses.

create_tables(uri)[source]
perform_upload(uri, results)[source]
upload_metadata(uri)[source]
class ramble.uploader.Experiment(name, workspace_hash, data, timestamp)[source]

Bases: object

Class representation of experiment data

generate_hash()[source]
get_hash()[source]
to_json()[source]
class ramble.uploader.ExperimentList(*args, **kwargs)[source]

Bases: list

class ramble.uploader.PrintOnlyUploader[source]

Bases: Uploader

An uploader that only prints out formatted data without actually uploading.

perform_upload(uri, results)[source]
class ramble.uploader.SQLiteUploader[source]

Bases: Uploader

Class to handle upload of FOMs to a local SQLite database

chunked_upload(table_id, data, uri=None)[source]

Abstract method for chunked uploads. Must be implemented by subclasses.

create_tables(uri)[source]
perform_upload(uri, results)[source]
upload_metadata(uri)[source]
class ramble.uploader.Uploader[source]

Bases: object

chunked_upload(table_id, data, uri=None)[source]

Abstract method for chunked uploads. Must be implemented by subclasses.

insert_data(uri: str, results) → None[source]
perform_upload(uri, data)[source]
schema = [{'metadata_key': 'experiment_schema_version', 'schema': {0.1: {'additionalProperties': False, 'properties': {'application_name': {'type': 'string'}, 'bulk_hash': {'type': 'string'}, 'data': {'type': 'string'}, 'id': {'type': 'integer'}, 'n_nodes': {'type': 'integer'}, 'n_ranks': {'type': 'integer'}, 'n_threads': {'type': 'integer'}, 'name': {'type': 'string'}, 'node_type': {'type': 'string'}, 'processes_per_node': {'type': 'integer'}, 'status': {'type': 'string'}, 'timestamp': {'format': 'date-time', 'type': 'string'}, 'user': {'type': 'string'}, 'workload_name': {'type': 'string'}, 'workspace_hash': {'type': 'string'}, 'workspace_name': {'type': 'string'}}, 'required': ['name', 'id', 'data', 'application_name', 'workspace_name', 'workspace_hash', 'workload_name', 'bulk_hash', 'n_nodes', 'processes_per_node', 'n_ranks', 'n_threads', 'node_type', 'status', 'user', 'timestamp'], 'type': 'object'}}, 'table': 'experiments', 'version': 0.1}, {'metadata_key': 'fom_schema_version', 'schema': {0.1: {'additionalProperties': False, 'properties': {'context': {'type': 'string'}, 'experiment_id': {'type': 'integer'}, 'experiment_name': {'type': 'string'}, 'name': {'type': 'string'}, 'origin': {'type': 'string'}, 'origin_type': {'type': 'string'}, 'unit': {'type': 'string'}, 'value': {'type': 'string'}}, 'required': ['name', 'value', 'unit', 'origin', 'origin_type', 'context', 'experiment_id', 'experiment_name'], 'type': 'object'}}, 'table': 'foms', 'version': 0.1}, {'metadata_key': 'metadata_schema_version', 'schema': {0.1: {'additionalProperties': False, 'properties': {'key': {'type': 'string'}, 'timestamp': {'format': 'date-time', 'type': 'string'}, 'value': {'type': 'string'}}, 'required': ['key', 'value', 'timestamp'], 'type': 'object'}}, 'table': 'metadata', 'version': 0.1}, {'metadata_key': 'experiments_metadata_schema_version', 'schema': {0.1: {'additionalProperties': False, 'properties': {'experiment_id': {'type': 'integer'}, 'key': {'type': 'string'}, 'timestamp': {'format': 'date-time', 'type': 'string'}, 'value': {'type': 'string'}}, 'required': ['experiment_id', 'key', 'value'], 'type': 'object'}}, 'table': 'experiments_metadata', 'version': 0.1}, {'metadata_key': 'software_db_schema_version', 'schema': {0.1: {'additionalProperties': False, 'properties': {'compiler': {'type': 'string'}, 'compiler_version': {'type': 'string'}, 'experiment_id': {'type': 'integer'}, 'experiment_name': {'type': 'string'}, 'name': {'type': 'string'}, 'target': {'type': 'string'}, 'variants': {'type': 'string'}, 'version': {'type': 'string'}}, 'required': ['experiment_id', 'experiment_name', 'name'], 'type': 'object'}}, 'table': 'software', 'version': 0.1}]
ramble.uploader.determine_node_type(experiment, contexts)[source]

Extract node type from available FOMS.

First prio is machine specific data, such as GCP meta data Second prio is more general data like CPU type

ramble.uploader.format_data(data_in)[source]

Goal: convert results to a more searchable and decomposed format for insertion into data store (etc)

Input:

{ experiment_name:
    { "CONTEXTS": {
        "context_name": "FOM_name { unit: "value", "value":value" }
    ...}
    }
}

Output: The general idea is the decompose the results into a more “database” like format, where the runs and FOMs are in a different table.

ramble.uploader.get_user()[source]
ramble.uploader.get_utc_timestamp() → str[source]

Returns the current UTC datetime formatted as an ISO 8601 string without timezone offset.

ramble.uploader.upload_results(results)[source]
class ramble.uploader.uploader_types(*values)

Bases: Enum

BigQuery = 1
PrintOnly = 2
SQLite = 3
ramble.uploader.validate_data(data, schema)[source]

Validate data against a JSON schema.

ramble.variants module

exception ramble.variants.RambleVariantError(message: str, long_message: str | None = None)[source]

Bases: RambleError

Class representing errors with variants

class ramble.variants.Variant(name: str, default: Any | None = None, description: str | None = '', values: Sequence | Callable[[Any], bool] | None = None)[source]

Bases: object

A custom set for housing multiple types of variants, and encapsulating the logic of merging them together.

as_definition() → str[source]

Build a definition for this variant

Format the variant as a string which can be used to test against when clauses.

Returns:

String definition for this variant

Return type:

str

as_definitions() → list[source]

Build a list of definitions for this variant

Format the variant as all possible strings which can be used to test against when clauses.

Returns:

String definitions for this variant

Return type:

list

as_str(n_indent: int = 0, verbose: bool = False)[source]

String documentation of this variant

Returns:

String for information of this variant

Return type:

str

copy()[source]
format_value(value: Any) → str[source]

Format a value for this variant into Spack-like syntax

class ramble.variants.VariantSet[source]

Bases: object

A custom set for housing multiple types of variants, and encapsulating the logic of merging them together.

as_set(expander: Expander | None = None) → set[source]

Construct a set of definitions for this variant set

The set of variant definitions will be used to determine if a when clause is valid or not.

Returns:

A set consisting of strings with the variant definitions expander (ramble.expander.Expander): Expander to use when expanding

variant definitions

Return type:

set

as_str(n_indent: int = 0, verbose: bool = False)[source]

String representation of this variant set

Parameters:

  • n_indent (int) – Number of spaces to indent string with

  • verbose – Print verbose

Returns:

Representation of this variant set

Return type:

(str)

copy()[source]
default_variant(name: str, default: Any | None = None, description: str = '', values: Sequence | Callable[[Any], bool] | None = None)[source]

Define a new default variant within this set.

Default variants are variants defined by directives in an object. These are used to define the defaults, and provide documentation for users.

Parameters:

  • name – Name of variant

  • default – Default value of the variant

  • description – Description of the variant, and what it’s used for

  • values – Set of valid values for the variant

experiment_variant(name: str, value: Any)[source]

Define a new experiment variant within this set.

Experiment variants are variants defined within a workspace’s configuration file. These are expected to be user defined values that will override the defaults of the object.

Experiment variants should always be defined after default variants (as defaults come from object directives, and experiment variants come from yaml). As a result, we only define experiment variants that have a corresponding default variant with the same name.

Parameters:

  • name – Name of variant

  • value – The value the variant should take.

merge_default_variants(in_set)[source]

Merge another variant set’s default variants into this variant set.

Parameters:

in_set – VariantSet to merge into self

merge_experiment_variants(in_set)[source]

Merge another variant set’s experiment variants into this variant set.

Parameters:

in_set – VariantSet to merge into self

merge_multi_value_variants(in_set)[source]

Merge another variant set’s multi value variants into this variant set.

Parameters:

in_set – VariantSet to merge into self

merge_variants(in_set)[source]
merge_version_variants(in_set)[source]

Merge another variant set’s version variants into this variant set.

Parameters:

in_set – VariantSet to merge into self

multi_value_variant(name: str, value: Any)[source]
value(name: str)[source]

Extract the value of a variant by name

Parameters:

name – Name of variant to determine value for

Returns:

Value of variant if found, otherwise None.

version(name: str)[source]

Extract the version of the named variant

Parameters:

name – Name of the variant to determine version of

Returns:

Version of the variant

Return type:

ramble.definitions.versions.ObjectVersion

version_variant(name: str, value: Any)[source]

Define a new version variant within this set.

Version variants are variants defined within the software section of a workspace’s configuration file.

Parameters:

  • name – Name of variant

  • default – Default value of the variant

  • description – Description of the variant, and what it’s used for

  • values – Set of valid values for the variant

ramble.variants.validate_variant(variant: str)[source]

Check if a variant name is valid or not

If the input variant name is not valid, this function will raise an exception. Otherwise this function will not perform any actions.

Parameters:

variant (str) – Variant name to test

class ramble.variants.variant_types(*values)

Bases: Enum

default = 1
experiment = 2
version = 3

ramble.wmkit module

wmkit is a set of useful modules to import when writing workflow managers

ramble.workload module

class ramble.workload.Workload(name: str, executables: List[str], inputs: List[str] | None = None, tags: List[str] | None = None, when: List[str] | None = None)[source]

Bases: object

Class representing a single workload

add_environment_variable(env_var: EnvironmentVariable)[source]

Add an environment variable to this workload

Parameters:

env_var (ramble.definitions.variables.EnvironmentVariable) – New environment variable to add to this workload

add_executable(executable: str)[source]

Add an executable to this workload

Parameters:

executable (str) – Name of executable to add to this workload

add_variable(variable: Variable)[source]

Add a variable to this workload

Parameters:

variable (ramble.definitions.variables.Variable) – New variable to add to this workload

as_str(n_indent: int = 0, verbose: bool = False)[source]

String representation of this workload

Parameters:

n_indent (int) – Number of spaces to indent string with

Returns:

Representation of this workload

Return type:

(str)

executables: List[str]
find_executable(exec_name: str)[source]

Find an executable in this workload

Parameters:

exec_name (str) – Name of executable to find

Returns:

Name of executable if it exists, None if it is not found

Return type:

(str | None)

find_input(input_name)[source]

Find an input in this workload

Parameters:

input_name (str) – Name of input to find

Returns:

Name of input if it exists, None if it is not found

Return type:

(str | None)

find_variable(name)[source]

Find a variable in this workload

Parameters:

var_name (str) – Name of variable to find

Returns:

Variable instance if it exists, None if

it is not found

Return type:

(ramble.definitions.variables.Variable | None)

inputs: List[str]
is_valid()[source]

Test if this workload is considered valid

Returns:

True if workload is valid, False otherwise

Return type:

(bool)

tags: List[str]
class ramble.workload.WorkloadGroup(name: str, workloads: List[str], when_list: List[str])[source]

Bases: object

Class representing a single workload group

add_workloads(workloads: List[str], when_list: List[str], mode: str)[source]

Add workloads to this workload group using a different set of ‘when’ condition

Parameters:

  • workloads – List of workloads

  • when_list – List of ‘when’ conditions for this list of workloads

  • mode – Append or overwrite workloads in this set of ‘when’ conditions

as_str(n_indent: int = 0, verbose: bool = False)[source]

String representation of this workload group

Parameters:

n_indent – Number of spaces to indent string with

Returns:

Representation of this workload

Return type:

(str)

name: str
workloads: Dict[FrozenSet[str], List[str]]