Developers - API

Internal configuration system

A Python module to maintain unique, run-wide QSIPrep settings.

This module implements the memory structures to keep a consistent, singleton config. Settings are passed across processes via filesystem, and a copy of the settings for each run and subject is left under <output_dir>/sub-<participant_id>/log/<run_unique_id>/qsiprep.toml. Settings are stored using ToML. The module has a to_filename() function to allow writing out the settings to hard disk in ToML format, which looks like:

Example file representation of QSIPrep settings.
[environment]
cpu_count = 1
exec_env = "docker"
free_mem = 52.2
overcommit_policy = "heuristic"
overcommit_limit = "50%"
nipype_version = "1.9.1"
templateflow_version = "23.1.0"
version = "1.0.0rc2.dev29+gdee1425.d20250113"

[execution]
bids_dir = "/tmp/src/qsiprep/.circleci/data/forrest_gump"
bids_database_dir = "/tmp/src/qsiprep/.circleci/work/forrest_gump/20250113-154517_1f7046f7-c5b1-4151-84ca-755a96dfbb55/bids_db"
bids_description_hash = "86891931159357fa937a6724ac27013f79ca56859100547ed9793245866312cd"
boilerplate_only = false
sloppy = true
debug = []
layout = "BIDS Layout: ...ep/.circleci/data/forrest_gump | Subjects: 1 | Sessions: 1 | Runs: 0"
log_dir = "/tmp/src/qsiprep/.circleci/out/forrest_gump/logs"
log_level = 15
low_mem = false
notrack = true
output_dir = "/tmp/src/qsiprep/.circleci/out/forrest_gump"
reports_only = false
run_uuid = "20250113-154517_1f7046f7-c5b1-4151-84ca-755a96dfbb55"
participant_label = [ "01",]
processing_list = [ "01:forrestgump",]
skip_anat_based_spatial_normalization = false
templateflow_home = "/home/qsiprep/.cache/templateflow"
work_dir = "/tmp/src/qsiprep/.circleci/work/forrest_gump"
write_graph = true

[workflow]
anat_modality = "T1w"
anat_only = false
anatomical_template = "MNI152NLin2009cAsym"
b0_threshold = 100
b0_motion_corr_to = "iterative"
b0_to_t1w_transform = "Rigid"
b1_biascorrect_stage = "none"
denoise_after_combining = false
denoise_method = "none"
distortion_group_merge = "none"
dwi_denoise_window = "auto"
dwi_no_biascorr = false
dwi_only = false
fmap_bspline = false
force_syn = false
hmc_model = "eddy"
hmc_transform = "Affine"
ignore = []
infant = false
intramodal_template_iters = 0
intramodal_template_transform = "BSplineSyN"
subject_anatomical_reference = "first-alphabetically"
longitudinal = false
no_b0_harmonization = false
output_resolution = 5.0
pepolar_method = "TOPUP"
separate_all_dwis = false
shoreline_iters = 2
use_syn_sdc = false
spaces = "MNI152NLin2009cAsym"

[nipype]
crashfile_format = "txt"
get_linked_libs = false
nprocs = 4
omp_nthreads = 4
plugin = "MultiProc"
remove_unnecessary_outputs = true
resource_monitor = false
stop_on_first_crash = true

[seeds]
master = 20506
ants = 2720
numpy = 32806

[execution.derivatives]

[execution.dataset_links]
raw = "/tmp/src/qsiprep/.circleci/data/forrest_gump"
templateflow = "/home/qsiprep/.cache/templateflow"

[nipype.plugin_args]
maxtasksperchild = 1
raise_insufficient = false

[execution.bids_filters.t1w]
reconstruction = "autobox"

[execution.bids_filters.t2w]
reconstruction = "autobox"

This config file is used to pass the settings across processes, using the load() function.

Configuration sections

class qsiprep.config.environment[source]

Read-only options regarding the platform and environment.

Crawls runtime descriptive settings (e.g., default FreeSurfer license, execution environment, nipype and QSIPrep versions, etc.). The environment section is not loaded in from file, only written out when settings are exported. This config section is useful when reporting issues, and these variables are tracked whenever the user does not opt-out using the --notrack argument.

cpu_count = 2

Number of available CPUs.

exec_docker_version = None

Version of Docker Engine.

exec_env = 'posix'

A string representing the execution platform.

free_mem = 6.0

Free memory at start.

nipype_version = '1.9.1'

Nipype’s current version.

overcommit_limit = '50%'

Linux’s kernel virtual memory overcommit limits.

overcommit_policy = 'heuristic'

Linux’s kernel virtual memory overcommit policy.

templateflow_version = '24.2.2'

The TemplateFlow client version installed.

version = '1.0.2.dev2+g00c73c4'

QSPrep’s version.

class qsiprep.config.execution[source]

Configure run-level settings.

bids_database_dir = None

Path to the directory containing SQLite database indices for the input BIDS dataset.

bids_description_hash = None

Checksum (SHA256) of the dataset_description.json of the BIDS dataset.

bids_dir = None

An existing path to the dataset, which must be BIDS-compliant.

bids_filters = None

A dictionary of BIDS selection filters.

boilerplate_only = False

Only generate a boilerplate.

A dictionary of dataset links to be used to track Sources in sidecars.

debug = []

Debug mode(s).

derivatives = {}

Path(s) to search for pre-computed derivatives

classmethod init()[source]

Create a new BIDS Layout accessible with layout.

layout = None

A BIDSLayout object, see init().

log_dir = None

The path to a directory that contains execution logs.

log_level = 25

Output verbosity.

low_mem = None

Utilize uncompressed NIfTIs and other tricks to minimize memory allocation.

notrack = False

Do not collect telemetry information for QSIPrep.

output_dir = None

Folder where derivatives will be stored.

output_layout = None

Layout of derivatives within output_dir.

participant_label = None

List of participant identifiers that are to be preprocessed.

processing_list = []

List of (subject_id, [session_id, …]) to be preprocessed together.

reports_only = False

Only build the reports, based on the reportlets found in a cached working directory.

run_uuid = '20250409-164410_ad156e64-e259-4ed4-87f9-a8168cc22e38'

Unique identifier of this particular run.

session_id = None

List of session identifiers that are to be preprocessed

skip_anat_based_spatial_normalization = False

Should we skip normalizing the anatomical data to a template?

sloppy = False

Run in sloppy mode (meaning, suboptimal parameters that minimize run-time).

templateflow_home = PosixPath('/home/docs/.cache/templateflow')[source]

The root folder of the TemplateFlow client.

work_dir = PosixPath('/home/docs/checkouts/readthedocs.org/user_builds/qsiprep/checkouts/latest/docs/work')[source]

Path to a working directory where intermediate results will be available.

write_graph = False

Write out the computational graph corresponding to the planned preprocessing.

class qsiprep.config.workflow[source]

Configure the particular execution graph of this workflow.

anat_modality = None

Modality to use as the anatomical reference. Images of this contrast will be skull stripped and segmented for use in the visual reports. If –infant, T2w is forced.

anat_only = False

Execute the anatomical preprocessing only.

anatomical_template = None

Anatomical template to use. This field doesn’t include the cohort.

b0_motion_corr_to = None

Perform SHORELine’s initial b=0-based registration to first volume? Or make a template? Either ‘iterative’ or ‘first’

b0_threshold = None

Any value in the .bval file less than this will be considered a b=0 image.

b0_to_t1w_transform = None

Transformation model for intramodal registration.

b1_biascorrect_stage = None

The stage of processing at which to apply B1 bias correction. Either “final” (after resampling), “none” (skipped entirely) or “legacy” (before concatenation).

denoise_after_combining = False

Run dwidenoise after combining dwis, but before motion correction.

denoise_method = None

Image-based denoising method. Either “dwidenoise” (MRtrix), “patch2self” (DIPY) or “none”.

distortion_group_merge = None

How to combine images across distortion groups (concatenate, average or none).

dwi_denoise_window = None

Window size in voxels for image-based denoising, integer or “auto”.

dwi_no_biascorr = None

see –b1-biascorrect-stage.

Type:

DEPRECATED

dwi_only = False

True if anat_modality is ‘none’.

Type:

DEPRECATED

eddy_config = None

Configuration for running Eddy.

fmap_bspline = None

Regularize fieldmaps with a field of B-Spline basis.

fmap_demean = None

Remove the mean from fieldmaps.

force_syn = None

Run fieldmap-less susceptibility-derived distortions estimation.

hmc_model = None

Model used to generate target images for hmc.

hmc_transform = None

Transformation to be used in SHORELine.

ignore = None

Ignore particular steps for QSIPrep.

infant = False

Configure pipelines specifically for infant brains

intramodal_template_iters = None

Number of iterations for intramodal template construction.

intramodal_template_transform = None

Transformation used for building the intramodal template.

longitudinal = False

Run FreeSurfer recon-all with the -longitudinal flag. [Deprecated]

no_b0_harmonization = False

Skip re-scaling dwi scans to have matching b=0 intensities.

output_resolution = None

Isotropic voxel size for outputs.

pepolar_method = None

SDC method to be used for PEPOLAR fieldmaps.

separate_all_dwis = False

Process all dwis separately - do not attempt concatenation.

shoreline_iters = None

How many iterations to run SHORELine.

subject_anatomical_reference = None

sessionwise, unbiased or first-alphabetically

Type:

How should the anatomical space be defined

unringing_method = None

Method for Gibbs-ringing removal. Either “none”, “mrdegibbs” or “rpg”.

use_syn_sdc = None

Run fieldmap-less susceptibility-derived distortions estimation in the absence of any alternatives.

class qsiprep.config.nipype[source]

Nipype settings.

crashfile_format = 'txt'

The file format for crashfiles, either text (txt) or pickle (pklz).

get_linked_libs = False

Run NiPype’s tool to enlist linked libraries for every interface.

classmethod get_plugin()[source]

Format a dictionary for Nipype consumption.

classmethod init()[source]

Set NiPype configurations.

memory_gb = None

Estimation in GB of the RAM this workflow can allocate at any given time.

nprocs = 2

Number of processes (compute tasks) that can be run in parallel (multiprocessing only).

omp_nthreads = None

Number of CPUs a single process can access for multithreaded execution.

plugin = 'MultiProc'

NiPype’s execution plugin.

plugin_args = {'maxtasksperchild': 1, 'raise_insufficient': False}

Settings for NiPype’s execution plugin.

remove_unnecessary_outputs = True

Clean up unused outputs after running

resource_monitor = False

Enable resource monitor.

stop_on_first_crash = True

Whether the workflow should stop or continue after the first error.

Usage

A config file is used to pass settings and collect information as the execution graph is built across processes.

from qsiprep import config
config_file = config.execution.work_dir / '.qsiprep.toml'
config.to_filename(config_file)
# Call build_workflow(config_file, retval) in a subprocess
with Manager() as mgr:
    from .workflow import build_workflow
    retval = mgr.dict()
    p = Process(target=build_workflow, args=(str(config_file), retval))
    p.start()
    p.join()
config.load(config_file)
# Access configs from any code section as:
value = config.section.setting

Logging

class qsiprep.config.loggers[source]

Keep loggers easily accessible (see init()).

cli = <Logger cli (WARNING)>[source]

Command-line interface logging.

default = <RootLogger root (WARNING)>[source]

The root logger.

classmethod init()[source]

Set the log level, initialize all loggers into loggers.

  • Add new logger levels (25: IMPORTANT, and 15: VERBOSE).

  • Add a new sub-logger (cli).

  • Logger configuration.

interface = <Logger nipype.interface (INFO)>[source]

NiPype’s interface logger.

utils = <Logger nipype.utils (INFO)>[source]

NiPype’s utils logger.

workflow = <Logger nipype.workflow (INFO)>[source]

NiPype’s workflow logger.

Other responsibilities

The config is responsible for other conveniency actions.

  • Switching Python’s multiprocessing to forkserver mode.

  • Set up a filter for warnings as early as possible.

  • Automated I/O magic operations. Some conversions need to happen in the store/load processes (e.g., from/to Path <-> str, BIDSLayout, etc.)

qsiprep.config.dumps()[source]

Format config into toml.

qsiprep.config.from_dict(settings, init=True, ignore=None)[source]

Read settings from a flat dictionary.

Parameters:

  • setting (dict) – Settings to apply to any configuration

  • init (bool or Container) – Initialize all, none, or a subset of configurations.

  • ignore (Container) – Collection of keys in setting to ignore

qsiprep.config.get(flat=False)[source]

Get config as a dict.

qsiprep.config.init_spaces(checkpoint=True)[source]

Initialize the spaces setting.

qsiprep.config.load(filename, skip=None, init=True)[source]

Load settings from file.

Parameters:

  • filename (os.PathLike) – TOML file containing QSIPrep configuration.

  • skip (dict or None) – Sets of values to ignore during load, keyed by section name

  • init (bool or Container) – Initialize all, none, or a subset of configurations.

qsiprep.config.to_filename(filename)[source]

Write settings to file.

Library API

Preprocessing Workflows

Other Utilities