.. Copyright 2022-2026 The Ramble Authors Licensed under the Apache License, Version 2.0 or the MIT license , at your option. This file may not be copied, modified, or distributed except according to those terms. .. _results: ============== Ramble Results ============== Ramble provides support and tooling around processing results data. See ``ramble workspace analyze`` (and associated :doc:`workspace documentation`) for generating such data. Most notably, the two supported functionalities are: * Uploading of stand alone JSON in ramble-output format * Generation of plots and analysis as part of a report ------ Upload ------ The ``ramble results upload`` command is used to import experiment results from a JSON file, and upload them as specified in the upload block of Ramble's config file (see :ref:`config documentation`). This can be used for both results from an arbitrary ramble workspace, or for data that is generated by tools which conform to rambles data schema. ------ Index ------ The ``ramble results index`` command is used to list attributes in results, including figures of merit and template variables. This is useful for discovering what metrics are available for plotting or analysis. .. code-block:: console ramble results index -f ~/ramble-results.json ------ Report ------ The ``ramble results report`` command is experimental functionality that provides the user a way to generate plots and reports from the output of a workspace. This output is in the form of individual `png` plots, and a combined `pdf` report Multiple plot types are supported, to aid in comparison of runs and common tasks, including: * Scaling (line) plots (both strong and weak) * Multi-line line plots * Comparison bar plots * General FOM bar plots For these plots, common operations are supported such as log axis, data grouping, and idealized line plotting. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Scaling and Multi-line Scaling Plots ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Activated by the ``strong-scaling``, ``weak-scaling``, or ``multi-line`` subcommands, these plots generate a line plot and attempt to annotate idealized scaling based on the data input. These commands take two sub-arguments: * The y-axis metric: the FOM you want to measure the performance of * The x-axis metric: the scaling variable to plot on the x axis And example call may look something like: .. code-block:: console ramble results report --strong-scaling 'simpleFoam Time' n_nodes -f ~/ramble-results.json (a very similar command should also work for ``--weak-scaling`` and ``--multi-line``) Another common use case might be to plot data by context (such as in OMB), which can be done by simply changing the y-axis metric: .. code-block:: console ramble results report --weak-scaling 'Bandwidth' context -f ~/ramble-results.json Other compelling options include ``-n`` to normalize the plot y values, or ``--logx``/``--logy`` for log axes. The ``--pandas-where`` option allows for down-selecting data using Pandas query format, and ``--split-by`` can be used to generate separate plots based on a Ramble variable. ^^^^^^^^^^^^^^^^^^^^^^^^ Comparison and FOM plots ^^^^^^^^^^^^^^^^^^^^^^^^ A comparison plot compares a specific FOM across all valid experiments, and generates a bar plot with one bar per experiment. It can be invoked by: .. code-block:: console ramble results report --compare "Total Elapsed Time" -f ~/ramble-results.json A FOM plot gives the user a quick way to visualize all FOMS in a workspace as a bar chart, for general data discovery. It can be invoked by: .. code-block:: console ramble results report --foms -f ~/ramble-results.json