Set-valued Output Sensitivity Analysis (SEVOSA)

Sensitivity Analysis tools for random-input set-valued-output systems with MPI/Slurm orchestration.

Copyright (C) 2026 Farbod Chamanian

This project is licensed under the GNU General Public License v2.0 or later.

See the LICENSE file for details.

Table of Contents

• About
• Declaration of AI use
• Figures
• Installation
• Functionality Overview
• Usage Guide - Reproducing Paper Results
• Usage Guide - Custom Use (under construction)

About

This software package accompanies the paper [Kernel-based Sensitivity Analysis - Current Status: Draft].

SEVOSA allows the user to perform sensitivity analysis on random-input set-valued-output systems, using both kernel-based and spatially-integrated Sobol' methods.

For non-Finite-Elements-based systems, one can use the statistical sampling functionality, or mass-matrix-based integration for the Finite-Elements-based systems.^*

This package also provides functionality to simulate a 2-dimensional Convection-Diffusion-Reaction combustion chamber with 5 random input parameters^2,3, 1-dimension diffusion problem, as well as analytical functions.^*,1

Note: This package currently only fully supports the reproducibility of the results of the accompanying paper. The custom-use is under construction and not yet fully supported or documented!

* 1D Diffusion and Analytical Function functionanlities are still under development. Refer to the Functionality Overview section below for a current status of supported functionalities of this package.
1 SEVOSA utilizes legacy FEniCS / DOLFIN 2019.1.0. to perform the FEM simulation. Martin S. Alnæs, Jan Blechta, Johan Hake, August Johansson, Benjamin Kehlet, Anders Logg, Chris N. Richardson, Johannes Ring, Marie E. Rognes, and Garth N. Wells. The FEniCS Project Version 1.5. Archive of Numerical Software, 3(100), 9–23, 2015. DOI: 10.11588/ans.2015.100.20553.
2 Marcelo Buffoni and Karen Willcox. Projection-Based Model Reduction for Reacting Flows. 40th Fluid Dynamics Conference and Exhibit, American Institute of Aeronautics and Astronautics, 2010. DOI: 10.2514/6.2010-5008.
3 E. Qian, B. Peherstorfer, D. O'Malley, V. V. Vesselinov, and K. Willcox. Multifidelity Monte Carlo Estimation of Variance and Sensitivity Indices. SIAM/ASA Journal on Uncertainty Quantification, 6(2), 683–706, 2018. DOI: 10.1137/17M1151006.

Declaration of AI use

This repository's author used OpenAI's ChatGPT (models GPT-5, GPT-5.1, GPT-5.2, and GPT-5.4) as an auxiliary software-development aid. The tool was used to provide explanations of submitted error messages and to assist in identifying potentially relevant external software packages and understanding the functionality and usage of package methods. In some cases, ChatGPT produced short illustrative code snippets showing how an external package function or method might be called or configured to achieve a specified behaviour. These snippets were used solely as explanatory examples for understanding package interfaces and functionality. They were not copied into, adapted for, or otherwise incorporated into this codebase. After consulting such examples, the author independently implemented the required functionality, and all resulting code was written, reviewed, tested, and verified by the author. This codebase, which was used to generate the data, analyses, and results reported in the accompanying paper contains no AI-generated code.

Figures

The following simulations are for the 2D Covection-Diffusion-Reaction (CDR) model with 5 random-input parameters as introduced by Qiani et. al.³ and Buffoni et. al.². The random parameters are:

• Reaction Coefficient (Pre-exponential Factor) $A$ with log-uniform distribution:

  A ~ logUnif(5.5e11, 1.5e12),

• Reaction Coefficient (Activation Energy) $E$ with log-uniform distribution:

  E ~ logUnif(1.5e3, 9.5e3),

• Fixed (Dirichlet) Temperature Boundary Condition at the chamber inlet on the left-most boundary, $T_i$ with uniform distribution:

  Tᵢ ~ Unif(850, 1000),

• Fixed (Dirichlet) Temperature Boundary Condition on the left-most boundary next to the inlet, $T_o$ with uniform distribution:

  Tₒ ~ Unif(200, 400),

• Fuel-to-oxidizer ratio of the premixed-inflow (indicating the amount of fuel and oxidizer Dirichlet Boundary Conditions at the inlet), $\phi$ with uniform distribution:

  ϕ ~ Unif(0.5, 1.5).

Note that these are default values as suggested by, e.g., Qiani et. al.³. SEVOSA allows the user to tinker with different distributions. Refer to the Functionality Overview and Usage sections below.

2D CDR Simulation: Temperature Field	Indicator Function Temperature ≤ 700 K

The indicator function plots indicate for which vertices on the FEM mesh (using order 1 elements) is the temperature below 700K. The 700 Kelvin value was chosen as an arbitrary critical temperature. In a realistic scenario, e.g., one might be interested in a temperature above which the chamber might pass a critical failure temperature. SEVOSA allows the user to tinker with different critical values, for all the four different variables that the 2D CDR problem is solved (namely, temperature, fuel, oxygen, and the combustion product (water) inside the chamber).

For the mathematically interested, the kernel-based sensitivity analysis method of SEVOSA (the HSIC sensitivity index), utilizes an $L^2$ norm on the symmetric difference of indicator functions defined via an exceedence sets (modeling the set-valued outputs), to calculate a "distance" between two set-valued outputs. Please refer to the accompanying paper for a more exhaustive and detailed explanation.

Symmetric Difference between Indicator Functions

SEVOSA allows the user to analyze sensitivity in different sections (observation windows) of the domain (the chamber). Below, we have the sensitivity analysis results of the 2D CDR problem within different sections of the chamber, where each specific section corresponding to the sensitivity results on the right, is indicated with a black rectangle on the left. The sensitivity analysis results on the right tell us "how important each of the 5 different random inputs are in causing the temperature to pass the critical value of 700K, relative to each other (importance ranking), and by how much (importance screening)".

An important takeaway from the results below, is that input-importance and model sensitivity for a complex-random system like the 2D CDR is highly domain-dependent. That is, one cannot understand the system only by looking at the whole domain. This insight also has implications for any other similar complex-random systems. Note that random systems do not necessarily have to be a partial-differential-equation as is the case here. Any system that (potentially nonlinearly) maps a number of random-inputs to a set of random outputs, can be analyzed in this way.

The sensitivity analysis estimation results are shown in each one of the plots below. Between the two figures, the more datapoints used, the lower the variability in the results. To create each of the boxplots, the sensitivity index estimation was re-ran 10 different times by using independent data of the Temperature field from the database of generated random 2D CDR model data. Note also that SEVOSA supports 2 different sensitivity indices, spatially-integrated Sobol' (spinSobol) and kernel-based (HSIC). spinSobol (in blue in the boxplots below) requires significantly more data to calculate than HSIC (in orange), as such, the spinSobol results presented below are estimated with less data, whilst HSIC results presented below are estimated using up to 10000 datapoints.

Sensitivity Analysis Results in Different Observation Windows (Temperature Critical Value of Interest: 700K)
Box plots of estimated spatially-integrated Sobol' sensitivity indices (blue) and HSIC indices (orange) for the temperature field of the 2D CDR problem, repeated 10 times for 1000 data points each within the respective sub-domain on the top row.
Box plots of estimated HSIC indices for the temperature field of the 2D CDR problem, repeated 10 times for 10000 data points each within the respective sub-domain on the top row.

The 2D CDR Simulation plots, Indicator Function plots, Symmetric Difference, and Sensitivity Analysis Results plots were generated using Matplotlib:

Hunter, J. D. Matplotlib: A 2D Graphics Environment, 2007. Computing in Science & Engineering, 9(3), 90–95. DOI: 10.1109/MCSE.2007.55

and seaborn:

Waskom, M. L. seaborn: statistical data visualization, 2021. Journal of Open Source Software, 6(60), 3021. DOI: 10.21105/joss.03021

Installation

1. Download Codebase

Either directly download and unzip the repository on your drive from the github repository, or run the following git clone command in your bash shell window:

Git clone https://github.com/Farbodch/sevosa.git

and then navigate to the code directory by typing:

cd sevosa

2. Conda Environment Manager

This package assumes a Python environment manager to manage dependencies, e.g., Miniforge3, Miniconda3. If you don’t already have Conda installed, you can install one of the following:

Miniforge3

Refer to the following link and scroll down to the "Install" section - Link to Miniforge.

Miniconda3

Refer to the following link for Miniconda3 or follow instructions below to install on your device - Link to Miniconda

Linux / macOS Copy-paste and run the following in your bash shell.

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

Windows

Download the Miniconda installer for Windows here: https://docs.conda.io/en/latest/miniconda.html#windows-installers

3. Setup Environment & Install Dependencies

1. Run:

conda env create -f environment.yml
conda activate sevosa

If you are on macOS you need to install the Homebrew package manager for the Libglu dependancy. You can do so by copy-pasting the following in a bash shell window (terminal):

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

2. Install libglu:

• macOS Brew install libglu
• non-macOS Conda install -c conda-forge libglu

4. Install sevosa:

Make sure you are in the root codebase directory (/sevosa) and using the correct conda environment by running

conda activate sevosa

in your bash shell window.

You can now install the sevosa package by running

python -m pip install -e .
bash install-conda-hooks.sh

---

Functionality Overview

SEVOSA supports two run modes

Mode	Use it to
--mode paper	Reproduce sensitivity index calculations and visualizations of the paper, and regenerating data. Refer to Usage Guide - Reproducing Paper Results.
--mode user	Custom configurations. Refer to Usage Guide - Custom Use (under construction).

Currently supported and tested functionalities include the ones directly relevant to the accompaying paper:

• Solving the Finite-Elements Method (FEM) formulation of a 2-dimensional Convection-Diffusion-Reaction (2D CDR) problem with 5 random input parameters using Legacy FEniCS 2019, where the user can choose from uniform and log-uniform distributions in their chosen range.
• Performing set-valued-output Sensitivity Analysis for FEM FEniCS function data, using a parallelized implementation of the variance-based Spatially-Integrated Sobol' Sensitivity Index and the kernel-based and spatially-integrated Hilbert-Schmidt-Independence-Criterion (HSIC) Sensitivity Index.
• Generating boxplots for ranking the importance-profile of a random-input system's inputs based on the calculated HSIC or Spatially-Integrated Sobol' indices. Note that the above functionalities use an FEM mass-matrix to perform and speed-up the spatial integrations.

In the absence of an FEM solution, one can statistically sample and estimate the set-valued-output instead of integrating. This functionality has been implemented in the codebase, but not fully tested and thus, needs further work to be released in a future version.

The following functionalities are currently under development and not fully stable to be used:

• Solving the FEM formulation of two types of random-input 1-dimensional Diffusion problems, one with the diffusion-coefficient defined as the expansion of a random-field, and one where the diffusion coefficient and a boundary-condition are defined using some user-defined distribution.
• Performing set-valued-output Sensitivity Analysis for any arbitrary analytical function with random-inputs defined according to some distribution.

---

Usage Guide - Reproducing Paper Results

A set of command-line-interface (CLI) commands are presented here for the interested ready to reproduce the results of the accompanying paper to this package. Note: this section assumes you have downloaded the accompanying dataset available at the Zenodo link provided above, and have unpacked the .zip file inside the ./data directory of this package. You can also generate a new set of data to re-run the index calculations with. However, we suggest you run these on an HPC cluster as the data generation can be very compute-intensive.

Which command should I run to reproduce what?

Before running any of the reproduction commands, you should activate the installed conda environment by typing the following in your shell environment and pressing enter:

conda activate sevosa

The reproduction scripts all begin with the following line

python -m sevosa.cli

and then require --mode, --task, and --type of task you want to run to be typed-in before pressing enter. For example

python -m sevosa.cli --mode paper --task viz --type f

As stated earlier, this package supports two modes: --mode paper and --mode user. We use --mode paper to reproduce results related to the accompanying paper.

In the following table, all of the different command-line options relevant to reproducing the paper results are detailed.

Task	Full Command	Description
Index calculation	--mode paper --task ic --type s_1000	--task ic indicates that you want to peform an index-calculation. --type s_1000 performs spinSobol sensitivity index estimation using 1000 data points, repeated 10 times, and saves the results to the directory data/results/paper_reproduction.
Index calculation	--mode paper --task ic --type h_1000	--task ic indicates that you want to peform an index-calculation. --type h_1000 performs HSIC sensitivity index estimation using 1000 data points, repeated 10 times, and saves the results to the directory data/results/paper_reproduction.
Index calculation	--mode paper --task ic --type h_10000	--task ic indicates that you want to peform an index-calculation. --type h_1000 performs HSIC sensitivity index estimation using 10000 data points, repeated 10 times, and saves the results to the directory data/results/paper_reproduction.
Visualization	--mode paper --task viz --type f	--task viz indicates that you want to generate a visulization/plot. --type f generates the three temperature field plots presented in the paper.
Visualization	--mode paper --task viz --type i	--task viz indicates that you want to generate a visulization/plot. --type i generates the three temperature indicator field plots with inequality constraint of 700 Kelvin, presented in the paper.
Visualization	--mode paper --task viz --type r1	--task viz indicates that you want to generate a visulization/plot. --type r1 generates the sensitivity index results box-plots together with the relevant observation window, for spinSobol and HSIC indices for N=1000 and repeated 10 times. Note: This assumes the sensitivity index calculation results are present inside the package's data directory data/results/paper_reproduction/. If you did not download the index calculation results from the zenodo directory and chose to re-run the calculations instead, then you first need to run --type s_1000 and --type h_1000 of the Index Calculation CLI commands presented earlier in this table first, before calling visualization function!.
Visualization	--mode paper --task viz --type r2	--task viz indicates that you want to generate a visulization/plot. --type r2 generates the sensitivity index results box-plots together with the relevant observation window, for HSIC indices for N=10000 and repeated 10 times. Note: This assumes the sensitivity index calculation results are present inside the package's data directory data/results/paper_reproduction/. If you did not download the index calculation results from the zenodo directory and chose to re-run the calculations instead, then you first need to run --type h_10000 of the Index Calculation CLI commands presented earlier in this table first, before calling visualization function!.
Data Generation	--mode paper --task dg --type s	--task dg indicates that you want to perform the FEM simulation of the random-input 2D Convection-Diffusion-Reaction system. --type s generates 1000 data points for the spinSobol index, and specifically only saves the temperature field to directory data/experiment_data/cdr/spinSobol to cut storage requirements by a quarter as the other 3 fields were not used in the sensitivity analyses of the paper. The Zenodo data directory provided with this paper and repository contains the 12000 data points used for estimating the spinSobol indices. As such, we recommend that if you want to generate your own data, to run this script multiple individual times. Note: the 2D CDR problem is computationally intensive to simulate. If you choose to generate your own data, then we recommend that you run this script on a High-Performance-Computing system. In order to run this script locally, first go to the configuration file ./configs/paper/manifest_data_generation.toml and set the parameter slurm_toggle to false (case-sensitive!), and then set the mpi_ranks parameter to as many cores as your computer supports to speed-up the simulations. If you chose to run this script on an HPC machine that supports slurm scheduling, then first set the parameter slurm_toggle to true (case-sensitive!), then head to the configuration file ./configs/paper/slurm_configs_data_generation.toml and edit this file according to your HPC provider's instructions. Every parameter-value pair you write inside the slurm_configs_data_generation.toml file will be written to a job.slurm script that will be submitted automatically using sbatch job.slurm when you run the data generation command --task dg.
Data Generation	--mode paper --task dg --type h	--task dg indicates that you want to perform the FEM simulation of the random-input 2D Convection-Diffusion-Reaction system. --type s generates 10000 data points for the HSIC index, and specifically only saves the temperature field to directory data/experiment_data/cdr/hsic to cut storage requirements by a quarter as the other 3 fields were not used in the sensitivity analyses of the paper. The Zenodo data directory provided with this paper and repository contains the 110000 data points used for estimating the HSIC indices, 100000 for the 10-repeats of 10000 datapoint-HSIC, and 10000 for 10-repeats of the 1000-HSIC estimations. As such, we recommend that if you want to generate your own data, to run this script multiple individual times. Note: the 2D CDR problem is computationally intensive to simulate. If you choose to generate your own data, then we recommend that you run this script on a High-Performance-Computing system. In order to run this script locally, first go to the configuration file ./configs/paper/manifest_data_generation.toml and set the parameter slurm_toggle to false (case-sensitive!), and then set the mpi_ranks parameter to as many cores as your computer supports to speed-up the simulations. If you chose to run this script on an HPC machine that supports slurm scheduling, then first set the parameter slurm_toggle to true (case-sensitive!), then head to the configuration file ./configs/paper/slurm_configs_data_generation.toml and edit this file according to your HPC provider's instructions. Every parameter-value pair you write inside the slurm_configs_data_generation.toml file will be written to a job.slurm script that will be submitted automatically using sbatch job.slurm when you run the data generation command --task dg.

The CLI decision tree is summarized in this chart:

flowchart TD 
    Start([Choose a mode])

    Start -->|--mode paper| Paper{Choose --task} 
    Start -->|--mode user| User[Customize run_manifest.toml<br/>and its sub-configuration files.] 
    
    Paper -->|--task dg| DG[Data Generation<br/>Choose --type] 
    Paper -->|--task ic| IC{Index Calculation<br/>Choose --type} 
    Paper -->|--task viz| Viz{Visualization<br/>Choose --type} 
    
    DG -->|--type s| S[generate 1000 data points <br/> for spinSobol]
    DG -->|--type h| H[generate 20000 data points <br/> for HSIC]

    IC -->|--type s_1000| S1000[Estimate spinSobol Index<br/>with N=1000<br/>repeated 10 times.] 
    IC -->|--type h_1000| H1000[Estimate HSIC Index<br/>with N=1000<br/>repeated 10 times.] 
    IC -->|--type h_10000| H10000[Estimate HSIC Index<br/>with N=10000<br/>repeated 10 times.] 
    
    Viz -->|--type f| F[Temperature Field] 
    Viz -->|--type i| I[Temperature Indicator Field] 
    Viz -->|--type r1| R1[Sensitivity results with<br/>observation windows<br/>HSIC and spinSobol<br/>N=1000 each<br/>repeated 10 times.] 
    Viz -->|--type r2| R2[Sensitivity results with<br/>observation windows<br/>HSIC<br/>N=10000<br/>repeated 10 times.] 

Loading

---

Usage Guide - Custom Use (under construction)

This package works by simple editing of experiment parameters in a .toml formatted config file, all living inside the configs/* directory. These files are easy-to-read text files that can be viewed and edited in any text editor. After editing the appropriate configuration files for your intended experiment, you can simply call sevosa.cli and pass in the directory to the run_manifest.toml file using --manifest configs/run_manifest.toml.

I. Define Intended Experiment - Configuration Files

To run any experiment (data generation, index calculation, or visualization), there are 2 main and 1 optional .toml files to configure:

1. configs/run_manifest.toml

   This config file contains the main orchastration of the experiment. In there, you need to edit the following parameters:

   Parameter	Description
   run_name	this parameter is functionally important, as it just serves to give the experiment a recognizable name during the experiment process and information logging.
   experiment_configs_path	this is the most important parameter, and in here you need to indicate the path to the .toml file which defines the purpose and parameters for your intended experiment.
   slurm_toggle	if you are using an High-Performance-Computing (HPC) cluster that is managed by slurm, then you can set this toggle to true to use the cluster resources to perform your experiment.
   slurm_configs_path	if you set the slurm_toggle to true, then you need to configure the configs/slurm_configs.toml file with the appropriate info relevant to your intended HPC cluster, and write the path to this file to the slurm_configs_path parameter. The path is defaulted to configs/slurm_configs.toml so you may not need to change this.
   mpi_rank	in case you are NOT using an HPC cluster and instead using your own machine, then you can pass-in the number of physical CPU cores you want to use to parallelize your experiment to the mpi_ranks parameter. The more physical CPU cores available and passed-in, the faster any of the experiments of this package will run (except visualization, as they only need 1 core).

2. configs/slurm_configs.toml (Optional)

   If you have access to a slurm-managed HPC cluster, then you have set the slurm_configs_path parameter inside configs/run_manifest.toml to true, and adjust the parameters inside this slurm_configs.toml path according to the cluster you intend to run your experiments on. Every arguement inside this .toml file is written to a job.sbatch file and automatically submitted to the indicated 'cluster' and 'partition'.

3. configs/<path-to-experiment-configs-file>.toml

   This is most important file you would need to edit, because it defines the type of experiment you want to run and its relevant parameters. The main supposed functionalities right now are Data Generation, *Index Calculation, and Visualization, where Data Generation currently only fully supports cdr.toml (even though a 1d_diffusion.toml file also exists, this is not yet fully tested), the Index Calculation supports spinSobol.toml (for spatially-integrated Sobol' indices) and hsic.toml (for the Hilbert-Schmidt-Indepedence-Criterion indices). Depeneding on which experiment you intend to run, you would need to adjust the experiment parameters. For the case of Index Calculation, you would also need to edit an extra file which contains the parameters of the model for which you have generated data and want to perform the sensitivitiy index calculation for. The experiment types, files, and relevant parameters are detailed as follows:

   1. Data Generation

      1. CDR: configs/data_generation/cdr.toml

         parameter	definition
         sensitivity_index_type	This parameter indicates for which sensitivity index type data should be generated. Options include: sensitivity_index_type="spinSobol" and sensitivity_index_type="hsic". Note that "spinSobol" will generate significantly more data than the "hsic" option, because the spatially-integrated Sobol' index requires more data than the HSIC index.
         N	The number of data points the program should generate.
         fem_iterations	This is a model parameter, and it defines the number of time-steps the FEM CDR model should be simulated for. By choosing a larger number of steps and keeping the fem_t_final parameter constant, the time-resolution of the simulation increases and will take longer to resolve.
         fem_t_final	This model parameter defines the final physical time (in seconds) you want to simulate the CDR model till.
         mesh_resolution	This is the mesh resolution parameter on which the FEM is solved, with the default value set to 0.025. If the mesh file for the passed-in mesh_resolution value exists, then you must set mesh_directory to the path of the mesh file. If the mesh file for the stated mesh_resolution does not exist, then the program will automatically generate a mesh for the stated resolution value and will save the mesh file at the directory stated in mesh_directory. Note that if you choose a different mesh option, you should leave the file name as rectangle.xdmf in mesh_directory parameter, and only edit the target save directory.
         data_directory	This is the directory at which the N different CDR FEM solutions will be saved.
         mesh_directory	This is the directory at which the mesh file of the FEM problem resides. If you intend to use a mesh with a mesh_resolution other than the default 0.025, then state your desired mesh_resolution value and then state the directory at which you want to save this new mesh inside mesh_directory, followed by /rectangle.xdmf. E.g., data/my_different_mesh_data/cdr/rectangle.xdmf.
         input_domain_specifications	Our CDR system is modeled using 5 different random input variables, including the reaction coefficients A and E, fixed Dirichlet Boundary Condition at the inlet T_i in the middle of the left boundary where the actual combustion occurs, fixed Dirichlet Boundary Condition T_i at the two sides of the left boundary sandwiching the inlet, and phi which indicates the fuel-to-oxygen ratio of the pre-mixed flow. Each of the 5 inputs have been given a specific distribution from which we sample and fix before starting a simulation. The default values are based on existing literature, but you can choose your prefered distribution_type and the min and max of the distribution. Currently, only distribution_type="log_uniform" and distribution_type="uniform" distributions are supported.

   2. Index Calculation

      1. HSIC: configs/index_calculation/hsic.toml

         Parameter	Description
         num_of_experiments	Number of experiments to run (sensitivity index re-calculations with different data points from the dataset). Note: if you choose a number larger than 1, then you have the option to set different seed values below in the shuffle_data_seeds for reproducibility.
         N	Number of data points to use to estimate the sensitivity index per experiment.
         shuffle_data_flag	If num_of_experiments is more than 1, you should set the shuffle_data_flag to true otherwise you would be using the exact same data for each experiment.
         shuffle_data_seeds	If you do choose num_of_experiments larger than 1, then you should either write in as many seeds in shuffle_data_seeds list as the num_of_experiments (e.g., for num_of_experiments = 3, we would have shuffle_data_seeds = [11, 132, 42]), or leave the shuffle_data_seeds list empty for random seeds for each experiment (e.g., shuffle_data_seeds = [])

      2. spinSobol: configs/index_calculation/spinSobol.toml

         Parameter	Description
         num_of_experiments	Number of experiments to run (sensitivity index re-calculations with different data points from the dataset). Note: if you choose a number larger than 1, then you have the option to set different seed values below in the shuffle_data_seeds for reproducibility.
         N	Number of data points to use to estimate the sensitivity index per experiment.
         shuffle_data_flag	If num_of_experiments is more than 1, you should set the shuffle_data_flag to true otherwise you would be using the exact same data for each experiment.
         shuffle_data_seeds	If you do choose num_of_experiments larger than 1, then you should either write in as many seeds in shuffle_data_seeds list as the num_of_experiments (e.g., for num_of_experiments = 3, we would have shuffle_data_seeds = [11, 132, 42]), or leave the shuffle_data_seeds list empty for random seeds for each experiment (e.g., shuffle_data_seeds = [])
         which_orders	Integers in the which_orders list determines which index orders to calculate. e.g., if we're only interested in main-effect (first-order) indices, then we set which_orders = [1], and if we want first-order and second-orders then we set which_orders = [1, 2]. The maximum number of orders is determined by the number of random-inputs of the model we're studying.
         get_total_sobols_flag	This flag determines whether we calculate the total spatially-integrated Sobol' indices or not. If the get_total_sobols_flag is set to true, then the algorithm expects Pick-Freeze-generated data with the index order of d-1, with d being the number of random-inputs of the model we're studying. The data_generation scripts included in this package generate these Pick-Freeze-based (d-1)-ordered data by default.

      3. index_calculation/model_data

         1. CDR: configs/index_calculation/model_data/cdr.toml

            Parameter	Description
            data_directory	Directory in which the CDR FEM solution data resides.
            field_of_interest	The data_generation for the 2D CDR model in this package, simulates four observables: fuel, oxygen, product (water), and temperature across the chamber. During index calculation, the user can indicate which of these fields they want to perform the sensitivity analysis for by passing in the relevant value into the field_of_interest parameter. The supported options are ["fuel_field", "oxygen_field", "product_field", "temp_field"].
            test_domains	In this parameter, the user can indicate which sub-sections inside the 2D CDR chamber they want to perform the sensitivity analysis for. The 2D CDR's default total domain lies within (inclusing) [0.0, 1.0] in the x-dimension and [0.0, 0.5] in the y-dimension. You can use indicate multiple intervals here to study them at the same time. E.g., test_domains = [[[0.0, 0.1], [0.0, 0.5]]] to study a single interval, or test_domains = [[[0.0, 0.1], [0.0, 0.5]], [[0.25, 0.5], [0.2, 0.3]]] for two intervals.
            g_inequality_constraints	The values here indicate the constant inequality contraint we are going to use to define the exceedence constraint (e.g., critical temperature), modeled as an excursion-set defining our set-valued-output. Effectively, here we define the "exceedence-level-set" we are interested to study. In this paramater, you can set your desired constraint value for any of the four fields (fuel, oxygen, product, temperature) that the 2D CDR model of this package calculates, but the contraint value that is actually used during index calculation, is for the specific field you passed into the field_of_interest parameter.
            model_params	Indicate the model parameters for which we're performing the sensitivity index calculations here. This data is used by the package to look for the generated data for the exact parameters passed-in here. Since there is a possibility that you might have generated 2D CDR data with different configurations (e.g., a 2D CDR that ran for 100 steps and a 2D CDR that ran for 500 steps) and saved them at the same location in drive, then the algorithm will use the information you pass-in here inside the model_params parameter to make sure that it performs the sensitivity analysis across different realizations of the same model consistently. If you are not using default values, you need to indicate the mesh_directory, fem_iterations, fem_t_final, and return_bool parameters of the target model. Refer to the "CDR: configs/data_generation/cdr.toml" Table above for a description of each of these parameters.
            input_domain_specifications	The exact same idea as model_params applies here to the input_domain_specifications. If you have generated 2D CDR simulation data for different types of distributions and stored them all in the same directory, then you need to indicate here which specific model you want to perform the sensitivity analysis for here. Refer to the "CDR: configs/data_generation/cdr.toml" Table above for a more detailed discription of this parameter.

      *Note that for index_calculation, run_manifest.toml is only concerned with paths to either hsic.toml or spinSobol.toml. You choose the intended model_data inside hsic.toml and spinSobol.toml simply by passing in, e.g., "cdr" to the 'model_name' parameter.

   3. Visualization

      Parameter	Description
      model_name	Indicate the model name for which you have calculated the sensitivity indices for, e.g., "cdr".
      root_data_path	The path at which you have stored the sensitivity index results, e.g., "data/results".
      save_directory	The path at which you would like to store the visualizations and plots, e.g., "data/figures".
      field_of_interest	You may have generated sensitivity indices for multiple different fields. For example, "temp_field" or "fuel_field" for the 2D CDR model, and stored them all at the same directory. In the field_of_interest parameter, you need to indicate which results you would like to visualize, e.g., field_of_interest = "temp_fueld".
      sensitivity_index_types	You may have calculated both the HSIC and the spatially-integrated Sobol' sensitivity indices for your desired field, or only one or the other. In the sensitivity_index_types parameter, you need to indicate the results of which indices you would like to visualize, e.g., sensitivity_index_types = ["spinSobol"] or = ["hsic"] for just one index type, or = ["spinSobol", "hsic"] for both when the results data for either is available. The package will generate boxplots of different colors (blue for Sobol' and orange for HSIC), and present them side-by-side when both are available for the same configuration profile (e.g., same n_datapoints, same field_of_interest, and same test_domains), or idividually when not.
      n_datapoints	You may have calculate the sensitivity indices using a set of different numbers of data points. In the n_datapoints parameter you need to indicate which results you want to visualize, e.g., n_datapoints = [1000] or n_datapoints = [1000, 10000]. The package will generate side-by-side boxplots with the different n_datapoints values on the x-axis and the sensitivity indices on the y-axis.
      test_domains	You may have calculated the sensitivity indices for different domains (intervals) of interest (as indicated in the "CDR: configs/index_calculation/model_data/cdr.toml" configuration file when you calculated the indices). In the test_domains parameter here, you should indicate the results of which domains you want to visualize, e.g., test_domains = [[[0.0, 0.1], [0.0, 0.5]]] to visualize the results of experiments on this individual interval, or test_domains = [[[0.0, 0.1], [0.0, 0.5]], [[0.25, 0.5], [0.2, 0.3]]] for these two intervals. A separate boxplot will be generated for each individual domain (interval) of interest, and stored in a folder that will be named according to the interval values (e.g., [[[0.0, 0.1], [0.0, 0.5]]] will store the visualization results inside save_directory/0.000_0.1000__0.000_0.5000, where save_directory is the parameter indicating in which directory the visualizations are to be saved, as described in earlier this table).

II. Run Experiment

After adjusting the relevant config files for your intended experiment, then you can copy-paste the following in your bash shell window to initiate and run your experiment:

conda activate sevosa
python -m sevosa.cli --mode user --manifest configs/run_manifest.toml

Note that you do not need to run conda activate sevosa every time. You just need to run it once to activate the conda environment that contains and manages the package's depedencies.