ramble.util package

Subpackages

• ramble.util.imp package
  • Submodules
  • ramble.util.imp.importlib_importer module
    • PrependFileLoader
      • PrependFileLoader.get_data()
      • PrependFileLoader.path_stats()
    • load_source()

Submodules

ramble.util.class_attributes module

ramble.util.class_attributes.convert_class_attributes(obj)

Convert class attributes defined from directives to instance attributes Class attributes that are valid for conversion are stored in the _directive_names attribute.

Parameters:

obj (object) – Input object instance to convert attributes in

ramble.util.colors module

exception ramble.util.colors.ColorParseError(message)

Bases: Exception

Raised when a color format fails to parse.

ramble.util.colors.auto_escape(s)

Escapes @ characters that are not part of valid color formats.

ramble.util.colors.cescape(string)

Escapes special characters needed for color codes.

Replaces the following symbols with their equivalent literal forms:

@	@@
}	}}

Parameters:

string (str) – the string to escape

Returns:

the string with color codes escaped

Return type:

(str)

ramble.util.colors.cextra(string)

Length of extra color characters in a string

ramble.util.colors.clen(string)

Return the length of a string, excluding ansi color sequences.

ramble.util.colors.colorize(string, **kwargs)

Wrapper for llnl.util.tty.color.colorize that automatically escapes @

ramble.util.colors.config_title(s)

ramble.util.colors.cprint(string, **kwargs)

Wrapper for llnl.util.tty.color.cprint that automatically escapes @ characters in the string if they do not match valid color codes.

ramble.util.colors.cwrite(string, **kwargs)

Wrapper for llnl.util.tty.color.cwrite that automatically escapes @ characters in the string if they do not match valid color codes.

ramble.util.colors.escape_str(s)

ramble.util.colors.get_color_when()

Return whether commands should print color or not.

ramble.util.colors.level_func(level)

ramble.util.colors.nested_1(s)

ramble.util.colors.nested_2(s)

ramble.util.colors.nested_3(s)

ramble.util.colors.nested_4(s)

ramble.util.colors.section_title(s)

ramble.util.colors.set_color_when(when)

Set when color should be applied. Options are:

• True or ‘always’: always print color

• False or ‘never’: never print color

• None or ‘auto’: only print color if sys.stdout is a tty.

ramble.util.colors.title_color(title: str, n_indent: int = 0)

Set the appropriate color for titles based on indentation

ramble.util.command_runner module

class ramble.util.command_runner.CommandRunner(name: str | None = None, command: str | None = None, shell: str = 'bash', dry_run: bool = False, path: str | None = None)

Bases: object

Runner for executing external commands

This class provides a generic wrapper on external commands, to provide a unified way to handle dry-run execution of external commands.

Can be inherited to construct custom command runners.

execute(executable: Executable, args: List[str], allow_failure: bool = False, return_output: bool = False) → str | None

Wrapper around execution of a command

Handles execution of a command when the execution path is dependent on whether dry run is enabled or disabled.

Parameters:

• executable (spack.util.executable.Executable) – Executable to run with arguments

• args (list(str)) – List of string arguments to pass into executable

• return_output (bool) – Whether the output of the command should be returned or not

get_version() → str

Hook to get the version of the executable

Should return a string representation of the executable’s version.

set_dry_run(dry_run: bool = False)

Set the dry_run state of this runner

exception ramble.util.command_runner.NoPathRunnerError

Bases: RunnerError

Raised when a runner is used that does not have a path set

exception ramble.util.command_runner.RunnerError

Bases: Exception

Raised when a problem occurs with a spack environment

exception ramble.util.command_runner.ValidationFailedError

Bases: RunnerError

Raised when a package manager requirement was not met

ramble.util.conflicts module

class ramble.util.conflicts.MODIFIER_CONFLICT(*values)

Bases: Enum

name_executables = 2

name_mode = 1

name_mode_executables = 3

name_only = 0

ramble.util.conversions module

ramble.util.conversions.canonical_none(maybe_none: Any) → Any | None

Convert a small set of “none-looking” inputs to None

ramble.util.conversions.convert_to_number(val: Any) → Any

Convert (in order of preference) to int, or float, or simply return itself

ramble.util.conversions.list_str_to_list(in_str: str) → str | list

Convert a comma delimited list as a string into a python list

Parameters:

in_str (str) – Input string, comma delimited list of values

Returns:

(list) Each value from input string is a separate entry in the list.

ramble.util.conversions.strip_quotes(in_str: str) → str

Remove quotes from a string if present.

ramble.util.directives module

ramble.util.directives.define_directive_methods_on_class(cls)

Create methods that execute directives on the class.

Wrap each directive, and inject it into this class as a method.

ramble.util.directives.wrap_named_directive_class_level(name)

Wrap a directive to simplify execution at the class level

Create a bound-like method that executes a directive on the instance

ramble.util.editor module

Module for finding the user’s preferred text editor.

Defines one function, editor(), which invokes the editor defined by the user’s VISUAL environment variable if set. We fall back to the editor defined by the EDITOR environment variable if VISUAL is not set or the specified editor fails (e.g. no DISPLAY for a graphical editor). If neither variable is set, we fall back to one of several common editors, raising an EnvironmentError if we are unable to find one.

ramble.util.editor.editor(*args, **kwargs)

Invoke the user’s editor.

This will try to execute the following, in order:

1. $VISUAL <args> # the “visual” editor (per POSIX)

2. $EDITOR <args> # the regular editor (per POSIX)

3. some default editor (see _default_editors) with <args>

If an environment variable isn’t defined, it is skipped. If it points to something that can’t be executed, we’ll print a warning. And if we can’t find anything that can be executed after searching the full list above, we’ll raise an error.

Parameters:

args (list of str) – args to pass to editor

Optional Arguments:

_exec_func (function): invoke this function instead of os.execv()

ramble.util.env module

class ramble.util.env.RambleEnvModifications(other=None, traced=None)

Bases: EnvironmentModifications

shell_modifications(shell='sh', explicit=False, env=None)

Wrapper around spack’s shell_modifications

ramble.util.executable module

class ramble.util.executable.CommandExecutable(name: str, template: str | List[str], use_mpi: bool = False, mpi: bool = False, variables: Dict[str, Any] | None = None, redirect: str = '{log_file}', output_capture: OUTPUT_CAPTURE = OUTPUT_CAPTURE.ALL, run_in_background: bool = False, allow_extension: bool = False, when: List[str] | None = None, **kwargs: Any)

Bases: object

CommandExecutable class This class is used to represent internal executables in Ramble.

These executables are portions of an experiment definition. They are generally used to group one or more commands together into an executable name.

copy() → CommandExecutable

Replicate a CommandExecutable instance

exception ramble.util.executable.CommandExecutableError(message: str, long_message: str | None = None)

Bases: RambleError

Class for errors when using command executable classes

class ramble.util.executable.PrefixedExecutable(name)

Bases: Executable

A version of spack.util.executable.Executable that allows command prefixes to be added

add_default_prefix(prefix: str | None)

Add a prefixed arg / cmd to the command

copy() → PrefixedExecutable

ramble.util.executable.which(*args: str, **kwargs: Any) → PrefixedExecutable | None

Finds an executable in the path like command-line which.

If given multiple executables, returns the first one that is found. If no executables are found, returns None.

Parameters:

*args (str) – One or more executables to search for

Keyword Arguments:

• path (list | str) – The path to search. Defaults to PATH

• required (bool) – If set to True, raise an error if executable not found

Returns:

The first executable that is found in the path

Return type:

(PrefixedExecutable)

ramble.util.file_cache module

exception ramble.util.file_cache.CacheError(message: str, long_message: str | None = None)

Bases: RambleError

class ramble.util.file_cache.FileCache(root, timeout=120)

Bases: object

This class manages cached data in the filesystem.

• Cache files are fetched and stored by unique keys. Keys can be relative paths, so that there can be some hierarchy in the cache.

• The FileCache handles locking cache files for reading and writing, so client code need not manage locks for cache entries.

cache_path(key)

Path to the file in the cache for a particular key.

destroy()

Remove all files under the cache root.

init_entry(key)

Ensure we can access a cache file. Create a lock for it if needed.

Return whether the cache file exists yet or not.

mtime(key)

Return modification time of cache file, or 0 if it does not exist.

Time is in units returned by os.stat in the mtime field, which is platform-dependent.

read_transaction(key)

Get a read transaction on a file cache item.

Returns a ReadTransaction context manager and opens the cache file for reading. You can use it like this:

with file_cache_object.read_transaction(key) as cache_file:

cache_file.read()

remove(key)

write_transaction(key)

Get a write transaction on a file cache item.

Returns a WriteTransaction context manager that opens a temporary file for writing. Once the context manager finishes, if nothing went wrong, moves the file into place on top of the old file atomically.

ramble.util.file_util module

ramble.util.file_util.create_symlink(base, link)

Create symlink of a file to give a known and predictable path

ramble.util.file_util.get_file_path(path: str, workspace) → str

A wrapper for file paths used in a Ramble application, to facilitate testing.

Parameters:

• path (str) – A file path

• workspace (ramble.workspace.Workspace) – A ramble workspace

Returns:

A file path

Return type:

(str)

ramble.util.file_util.get_newest_experiment_file(base_directory)

Given a base directory, determine the newest file in the directory (and it’s subdirectories)and return the file path and it’s timestamp in seconds.

Parameters:

base_directory (str) – Directory to search newest file for

Returns:

Path to newest file (or None if not found) (int): Timestamp of file in seconds (or None if no file is found)

Return type:

(str)

ramble.util.file_util.is_dry_run_path(path: str) → bool

Check if the path is already a dry_run path

ramble.util.foms module

class ramble.util.foms.BetterDirection(*values)

Bases: Enum

HIGHER = 1

INAPPLICABLE = 4

INDETERMINATE = 3

LOWER = 2

classmethod from_str(string)

class ramble.util.foms.FomType(*values)

Bases: Enum

CATEGORY = 4

INFO = 5

MEASURE = 3

THROUGHPUT = 2

TIME = 1

UNDEFINED = 6

better_direction()

copy()

classmethod from_str(string)

to_dict()

Converts the FomType enum member to a dictionary representation.

class ramble.util.foms.SummaryFoms(*values)

Bases: Enum

N_SUCCESS = 'n_success_repeats'

N_TOTAL = 'n_total_repeats'

SUMMARY = 'Experiment Summary'

ramble.util.foms.get_literal_from_regex(regex_str: str) → str

Extracts a first-encountered required literal string from a regex pattern.

This is used to create fast string pre-filters to avoid executing complex regex matching. This is not exhaustive, as it doesn’t recurse into sub-patterns. It is only intended as a heuristic to short-circuit the matching process.

ramble.util.format module

ramble.util.format.format_doc(doc_str, **kwargs)

Wrap doc string at 72 characters and format nicely

ramble.util.format.when_order(when: str)

Sort ‘when’ conditions consistently when unpacking from sets or lists

ramble.util.graph module

class ramble.util.graph.GraphNode(key, attribute=None, obj_inst=None)

Bases: object

Class representing a node of a graph, where the node can have an attribute attached to it.

This allows nodes to be added into a graph, have the topological order of the graph returned, and be able to refer to the attribute of the original node easily.

order_after(key)

Adds information that this node should come after another node

Parameters:

key (str) – Key of node that should come before this node.

order_before(key)

Adds information that this node should come before another node

Parameters:

key (str) – Key of node that should come after this node.

set_attribute(attr)

Sets the attribute of a graph node

Parameters:

attr – An arbitrary attribute to attach to this node.

ramble.util.hashing module

ramble.util.hashing.hash_file(file_path)

ramble.util.hashing.hash_json(in_json: Any) → str

ramble.util.hashing.hash_string(string)

ramble.util.install_cache module

Provide data structures to assist with caching operations

class ramble.util.install_cache.SetCache

Bases: object

add(tupl)

contains(tupl)

ramble.util.lock module

Wrapper for llnl.util.lock allows locking to be enabled/disabled.

class ramble.util.lock.Lock(*args, **kwargs)

Bases: Lock

Lock that can be disabled.

This overrides the _lock() and _unlock() methods from llnl.util.lock so that all the lock API calls will succeed, but the actual locking mechanism can be disabled via _enable_locks.

cleanup(*args)

exception ramble.util.lock.LockError

Bases: Exception

Raised for any errors related to locks.

exception ramble.util.lock.LockTimeoutError

Bases: LockError

Raised when an attempt to acquire a lock times out.

exception ramble.util.lock.LockUpgradeError(path)

Bases: LockError

Raised when unable to upgrade from a read to a write lock.

class ramble.util.lock.ReadTransaction(lock, acquire=None, release=None, timeout=None)

Bases: LockTransaction

LockTransaction context manager that does a read and releases it.

class ramble.util.lock.WriteTransaction(lock, acquire=None, release=None, timeout=None)

Bases: LockTransaction

LockTransaction context manager that does a write and releases it.

ramble.util.lock.check_lock_safety(path)

Do some extra checks to ensure disabling locks is safe.

This will raise an error if path can is group- or world-writable AND the current user can write to the directory (i.e., if this user AND others could write to the path).

This is intended to run on the Ramble prefix, but can be run on any path for testing.

ramble.util.logger module

class ramble.util.logger.Logger

Bases: object

Logger class

This class providers additional functionality on top of LLNL’s tty utility. Namely, this class provides a stack of log files, and allows errors to be printed to all log files instead of only one.

active_log()

Return the path for the active log

If any logs are in the log stack, return the filepath of the active log. Otherwise, return the string ‘stdout’

active_stream()

Return the stream for the active log

If any logs are in the log stack, return the stream object of the active log. Otherwise, return None to indicate the system is handling printing.

add_log(path)

Add a log to the current log stack

Opens (with ‘a+’ permissions) the file provided by the ‘path’ argument, and stores both the path, and the opened stream object in the current stack in the active position.

Parameters:

path – File path for the new log file

add_log_context(path)

Context manager to add and remove a log file

Ensures that the log is removed from the stack even if an exception occurs. If inner add_log/remove_log calls disrupt the stack order, it ensures it only removes the log it added.

aggregate_warnings(on=True)

Control whether warnings are aggregated or not

all_msg(*args, **kwargs)

Print a message to all logs

Pass all args and kwargs to tty.info (which will concatenate and print). Perform this action for all logs and the default log (to screen).

all_warnings()

Print all warnings that have been encountered

This is intended to be called once, and will print all warnings that were encountered over the course of the execution.

configure_colors(**kwargs)

debug(*args, **kwargs)

Print a debug message to the active log

Pass all args and kwargs to tty.debug (which will concatenate and print). Perform this action for the active log only.

die(*args, **kwargs)

Print an error message and terminate execution

Pass all args and kwargs to tty.error (which will concatenate and print). Perform this action all logs. After all logs are printed to, terminate execution (and error) using tty.die.

error(*args, **kwargs)

Print an error message

Pass all args and kwargs to tty.error (which will concatenate and print). Perform this action all logs, and the default stream (print to screen).

info(*args, **kwargs)

Print a message to the active log

Pass all args and kwargs to tty.info (which will concatenate and print). Perform this action for the active log only.

msg(*args, **kwargs)

Print a message to the active log

Pass all args and kwargs to tty.info (which will concatenate and print). Perform this action for the active log only.

remove_log()

Remove the active stack

Pop the active log from the log stack, and close the log stream.

verbose(*args, **kwargs)

Print a verbose message to the active log

Pass all args and kwargs to tty.verbose (which will concatenate and print). Perform this action for the active log only.

warn(*args, **kwargs)

Print a warning message to the active log

Pass all args and kwargs to tty.warn (which will concatenate and print). Perform this action for the active log only.

ramble.util.matrices module

ramble.util.matrices.extract_matrices(action, name, in_dict)

Extract matrix definitions from a dictionary

Parameters:

• action – The action of an object definition where matrices are being extracted from

• name – The name of the object

• in_dict – The dictionary containing definitions

Returns:

list of matrix definitions

ramble.util.module_utils module

A utility for dynamically importing modules.

ramble.util.module_utils.import_module(module_name)

A utility for dynamically importing a module.

ramble.util.module_utils.import_pandas()

A utility for dynamically importing pandas.

ramble.util.naming module

class ramble.util.naming.NamespaceTrie(separator='.')

Bases: object

class Element(value)

Bases: object

has_value(namespace)

True if there is a value set for the given namespace.

is_leaf(namespace)

True if this namespace has no children in the trie.

is_prefix(namespace)

True if the namespace has a value, or if it’s the prefix of one that does.

ramble.util.naming.match_pattern(pattern, string)

Match a string against a pattern (regex or glob)

Parameters:

• pattern (str) – pattern to match against

• string (str) – string to match

Returns:

(bool: matched, dict: captured groups)

Return type:

(tuple)

ramble.util.naming.mod_to_class(mod_name)

Convert a name from module style to class name style. Ramble mostly follows PEP-8:

• Module and package names use lowercase_with_underscores.

• Class names use the CapWords convention.

Regular source code follows these conventions. Ramble is a bit more liberal with its Application names:

• They can contain ‘-’ as well as ‘_’, but cannot start with ‘-‘.

• They can start with numbers, e.g. “3proxy”.

This function converts from the module convention to the class convention by removing _ and - and converting surrounding lowercase text to CapWords. If mod_name starts with a number, the class name returned will be prepended with ‘_’ to make a valid Python identifier.

ramble.util.naming.possible_ramble_module_names(python_mod_name)

Given a Python module name, return a list of all possible ramble module names that could correspond to it.

ramble.util.naming.ramble_module_to_python_module(mod_name)

Given a Ramble module name, returns the name by which it can be imported in Python.

ramble.util.naming.simplify_name(name)

Simplify package name to only lowercase, digits, and dashes.

Simplifies a name which may include uppercase letters, periods, underscores, and pluses. In general, we want our package names to only contain lowercase letters, digits, and dashes.

Parameters:

name (str) – The original name of the package

Returns:

The new name of the package

Return type:

str

ramble.util.naming.valid_fully_qualified_module_name(mod_name)

Return whether mod_name is a valid namespaced module name.

ramble.util.naming.valid_module_name(mod_name)

Return whether mod_name is valid for use in Ramble.

ramble.util.naming.validate_fully_qualified_module_name(mod_name)

Raise an exception if mod_name is not a valid namespaced module name.

ramble.util.naming.validate_module_name(mod_name)

Raise an exception if mod_name is not valid.

ramble.util.object_utils module

Module for providing utils related to Ramble objects.

ramble.util.object_utils.filter_by_name(glob_patterns: List[str], search_description: bool, obj_type: Any) → List[str]

Filters the sequence of object names by the given glob patterns.

Parameters:

• glob_patterns – patterns to match against object names

• search_description – whether to search inside object descriptions

• obj_type – type of the Ramble objects to search for

Returns:

filtered and sorted list of object names

ramble.util.output_capture module

class ramble.util.output_capture.OUTPUT_CAPTURE(*values)

Bases: Enum

ALL = 2

DEFAULT = 2

STDERR = 1

STDOUT = 0

class ramble.util.output_capture.output_mapper

Bases: object

generate_out_string(out_log, output_operator)

ramble.util.path module

Utilities for managing paths in ramble.

TODO: this is really part of ramble.config. Consolidate it.

ramble.util.path.canonicalize_path(path)

Same as substitute_path_variables, but also take absolute path.

ramble.util.path.substitute_config_variables(path, local_replacements)

Substitute placeholders into paths.

Ramble allows paths in configs to have some placeholders, as follows:

• $ramble The ramble instance’s prefix

• $user The current user’s username

• $tempdir Default temporary directory returned by tempfile.gettempdir()

These are substituted case-insensitively into the path, and users can use either $var or ${var} syntax for the variables.

ramble.util.path.substitute_path_variables(path, local_replacements=None)

Substitute config vars, expand environment vars, expand user home.

ramble.util.shell_utils module

Utils for handling shell specialization

exception ramble.util.shell_utils.UnsupportedError(message: str, long_message: str | None = None)

Bases: RambleError

Error when certain shell features are not supported.

ramble.util.shell_utils.cmd_sub_str(shell, cmd: str)

Get the command substitution string of the given shell

ramble.util.shell_utils.gen_dict_definition(var_name, dict, shell='bash')

A utility for generating commands for defining a dict in a shell. It currently supports bash only. It maps a python dict into an associative array in bash.

Parameters:

• var_name – the variable name for the generated dict in the shell.

• dict – the input dict.

• shell – the shell used. Currently this is bash-only.

Returns:

The string of the generated shell commands.

ramble.util.shell_utils.get_compatible_base_shell(shell)

Get the compatible base shell

For instance, given bash it returns sh. This is commonly used to translate shell name for calling spack utils, which only support such “base” shells.

Parameters:

shell (str) – Name of shell

Returns:

The base compatible shell.

Return type:

(str)

ramble.util.shell_utils.last_pid_var(shell: str) → str

ramble.util.shell_utils.source_str(shell)

Wrapper to return the source command for a given shell

Parameters:

shell (str) – Name of shell to get the source command for

Returns:

Source command for the requested shell.

Return type:

(str)

ramble.util.spec_utils module

class ramble.util.spec_utils.SoftwareSpec(name: str, pkg_spec: str, prefix: str = '', compiler: str | None = None, compiler_spec: str | None = None, inject_if_missing: bool = False, when: List[str] | None = None)

Bases: object

as_str(n_indent: int = 0, verbose: bool = False)

config_opts()

conflict_dict(test_dict: Dict)

conflict_spec(test, skip_conflicting_when: bool = True)

copy()

to_dict(apply_prefix: bool = False)

ramble.util.spec_utils.specs_conflict(new, existing, prefix='', skip_conflicting_when=False)

ramble.util.stats module

class ramble.util.stats.ConfidenceLevel(*values)

Bases: Enum

CL_50 = 0.5

CL_90 = 0.9

CL_95 = 0.95

CL_99 = 0.99

class ramble.util.stats.StatsBase

Bases: object

compute(values: List[float]) → float | str

get_unit(unit: str) → str

min_count: int = 1

name: str = ''

report(values: List[float], unit: str) → Tuple[float | str, str, str]

class ramble.util.stats.StatsCoefficientOfVariation

Bases: StatsBase

compute(values: List[float]) → float | str

get_unit(unit: str) → str

min_count: int = 2

name: str = 'cv'

class ramble.util.stats.StatsConfidenceIntervalBase

Bases: StatsBase

compute(values: List[float]) → float

confidence_level: ConfidenceLevel

is_upper: bool

min_count: int = 2

class ramble.util.stats.StatsConfidenceIntervalLower50

Bases: StatsConfidenceIntervalBase

confidence_level: ConfidenceLevel = 0.5

is_upper: bool = False

name: str = 'ci_50_lower'

class ramble.util.stats.StatsConfidenceIntervalLower90

Bases: StatsConfidenceIntervalBase

confidence_level: ConfidenceLevel = 0.9

is_upper: bool = False

name: str = 'ci_90_lower'

class ramble.util.stats.StatsConfidenceIntervalLower95

Bases: StatsConfidenceIntervalBase

confidence_level: ConfidenceLevel = 0.95

is_upper: bool = False

name: str = 'ci_95_lower'

class ramble.util.stats.StatsConfidenceIntervalLower99

Bases: StatsConfidenceIntervalBase

confidence_level: ConfidenceLevel = 0.99

is_upper: bool = False

name: str = 'ci_99_lower'

class ramble.util.stats.StatsConfidenceIntervalUpper50

Bases: StatsConfidenceIntervalBase

confidence_level: ConfidenceLevel = 0.5

is_upper: bool = True

name: str = 'ci_50_upper'

class ramble.util.stats.StatsConfidenceIntervalUpper90

Bases: StatsConfidenceIntervalBase

confidence_level: ConfidenceLevel = 0.9

is_upper: bool = True

name: str = 'ci_90_upper'

class ramble.util.stats.StatsConfidenceIntervalUpper95

Bases: StatsConfidenceIntervalBase

confidence_level: ConfidenceLevel = 0.95

is_upper: bool = True

name: str = 'ci_95_upper'

class ramble.util.stats.StatsConfidenceIntervalUpper99

Bases: StatsConfidenceIntervalBase

confidence_level: ConfidenceLevel = 0.99

is_upper: bool = True

name: str = 'ci_99_upper'

class ramble.util.stats.StatsHarmonicMean

Bases: StatsBase

compute(values: List[float]) → float | str

name: str = 'harmonic_mean'

class ramble.util.stats.StatsMax

Bases: StatsBase

compute(values: List[float]) → float

name: str = 'max'

class ramble.util.stats.StatsMean

Bases: StatsBase

compute(values: List[float]) → float

name: str = 'mean'

class ramble.util.stats.StatsMedian

Bases: StatsBase

compute(values: List[float]) → float

name: str = 'median'

class ramble.util.stats.StatsMin

Bases: StatsBase

compute(values: List[float]) → float

name: str = 'min'

class ramble.util.stats.StatsStdev

Bases: StatsBase

compute(values: List[float]) → float

min_count: int = 2

name: str = 'stdev'

class ramble.util.stats.StatsVar

Bases: StatsBase

compute(values: List[float]) → float

get_unit(unit: str) → str

min_count: int = 2

name: str = 'variance'

ramble.util.version module

ramble.util.version.get_git_hash(path: str = '/home/docs/checkouts/readthedocs.org/user_builds/ramble-dev-dapomeroy/checkouts/latest/lib/ramble/docs/_ramble_root') → str | None

Get get hash from a path

Outputs ‘<git commit sha>’.

ramble.util.version.get_version() → str

Get a descriptive version of this instance of Ramble.

Outputs ‘<PEP440 version> (<git commit sha>)’.

The commit sha is only added when available.

ramble.util.web module

exception ramble.util.web.HTMLParseError

Bases: Exception

class ramble.util.web.LinkParser

Bases: HTMLParser

This parser just takes an HTML page and strips out the hrefs on the links. Good enough for a really simple spider.

handle_starttag(tag, attrs)

ramble.util.web.RAMBLE_USER_AGENT = 'Ramblebot/0.6.0'

User-Agent used in Request objects

exception ramble.util.web.SpackWebError(message, long_message=None)

Bases: SpackError

Superclass for Spack web spidering errors.

ramble.util.web.get_header(headers, header_name)

Looks up a dict of headers for the given header value.

Looks up a dict of headers, [headers], for a header value given by [header_name]. Returns headers[header_name] if header_name is in headers. Otherwise, the first fuzzy match is returned, if any.

This fuzzy matching is performed by discarding word separators and capitalization, so that for example, “Content-length”, “content_length”, “conTENtLength”, etc., all match. In the case of multiple fuzzy-matches, the returned value is the “first” such match given the underlying mapping’s ordering, or unspecified if no such ordering is defined.

If header_name is not in headers, and no such fuzzy match exists, then a KeyError is raised.

ramble.util.web.list_url(url, recursive=False)

ramble.util.web.push_to_url(local_file_path, remote_path, keep_original=True, extra_args=None)

ramble.util.web.read_from_url(url, accept_content_type=None)

ramble.util.web.remove_url(url, recursive=False)

ramble.util.web.spider(root_urls, depth=0, concurrency=32)

Get web pages from root URLs.

If depth is specified (e.g., depth=2), then this will also follow up to <depth> levels of links from each root.

Parameters:

• root_urls (str | list) – root urls used as a starting point for spidering

• depth (int) – level of recursion into links

• concurrency (int) – number of simultaneous requests that can be sent

Returns:

A dict of pages visited (URL) mapped to their full text and the set of visited links.

ramble.util.web.url_exists(url)

ramble.util.web.uses_ssl(parsed_url)

ramble.util.yaml_generation module

Module representing utility functions for managing YAML configuration files within application definition files

These functions are intended to help read, write, and manipulate YAML based configuration files that an experiment might use as input.

Workload variables that represent configuration files should be defined using a ‘.’ delimiter between YAML object names. As an example:

foo:
    bar:
        baz: 1.0

Would translate to foo.bar.baz = 1.0 in Ramble syntax.

ramble.util.yaml_generation.all_config_options(config_data: Dict)

Extract all config options from config_data dictionary

Parameters:

config_data (dict) – A config dictionary representing data read from a YAML file.

Returns:

Set containing all detected fully qualified option names

Return type:

(set)

ramble.util.yaml_generation.apply_default_config_values(config_data, app_inst, default_config_string)

Apply default config values (from config_data) to an experiment

Process all workloads variables (for the current workload in app_inst). Any variable who’s expanded value is equal to default_config_string will have its value overwritten to the value in the config_dict dictionary.

Parameters:

• config_data (dict) – Dictionary of config data read from a YAML file

• app_inst – Application instance representing an experiment

• default_config_string (str) – String that conveys the default config_data should be used in place of the current value.

ramble.util.yaml_generation.get_config_value(config_data: Dict, option_name: str)

Get a config option based on dictionary attribute syntax

Given an option_name of the format: attr1.attr2.attr3 return its value from config_data.

Parameters:

• config_data (dict) – A config dictionary representing data read from a YAML file.

• option_name (str) – Name of config option to get

Returns:

Value of config option

Return type:

(Any)

ramble.util.yaml_generation.read_config_file(conf_path: str)

Read an existing YAML file and return its data as a dictionary

Parameters:

conf_path (str) – Path to input configuration file to read

Returns:

Dictionary representation of the data contained in conf_path

Return type:

(dict)

ramble.util.yaml_generation.remove_config_value(config_data: Dict, option_name: str)

Remove a config option based on dictionary attribute syntax

Given an option_name of the format: attr1.attr2.attr3 remote it from config_data. Also, remove any parent scopes that are empty.

Parameters:

• config_data (dict) – A config dictionary representing data read from a YAML file.

• option_name (str) – Name of config option to set

ramble.util.yaml_generation.set_config_value(config_data: Dict, option_name: str, option_value: Any, force: bool = False)

Set a config option based on dictionary attribute syntax

Given an option_name of the format: attr1.attr2.attr3 set its value to option_value in config_data.

Parameters:

• config_data (dict) – A config dictionary representing data read from a YAML file.

• option_name (str) – Name of config option to set

• option_value (Any) – Value to set config option to

• force (bool) – If true, all missing layers in the attribute list are created. If false, only sets existing attributes