{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Brain-behaviour association simulation example\n", "\n", "A multiverse is defined as a dictionary of decision/option pairs. For example, consider a hypothetical analysis pipeline which correlates global efficiency of static brain networks with general cognitive abiity. Such an analysis pipeline involves many decision points, for example the software package for preprocessing, parametrs for data cleaning, or different methods for the connectivity estimation.\n", "\n", "We here illustrate this with a hypothetical multiverse with the following forking paths:\n", "\n", "- 3 software packages (fMRIprep, CONN, or SPM) for preprocessing\n", "- 2 options for global signal regression (True/False)\n", "- 4 parcellation schemes (AAL, Schaefer 200, or Glasser MMP atlas)\n", "- 3 connectivity measures (pearson correlation, partial correlation, or mutual information)\n", "- 3 density values for thresholding the connectivity matrix (10%, 30%, and 50%)\n", "\n", "The total number of possible combinations is $3 \\times 2 \\times 4 \\times 3 \\times 3 = 216$, which means our multiverse will contain 216 universes." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from comet.multiverse import Multiverse\n", "\n", "forking_paths = {\n", " \"software\": [\"fMRIprep\", \"CONN\", \"SPM\"],\n", " \"gsr\": [True, False],\n", " \"parcellation\": [\"AAL\", \"Power 160\", \"Schaefer 200\", \"Glasser MMP\"],\n", " \"connectivity\": [\"pearson\", \"partial\", \"mutual info\"],\n", " \"threshold\": [0.1, 0.3, 0.5],\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We then simulate some results by adding systematic differences (depending on the analytical choices) and noise to a baseline value:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "def analysis_template():\n", " import comet\n", " import numpy as np\n", " from scipy.stats import ttest_1samp\n", "\n", " # Base correlation between global efficiency and cognitive ability\n", " base_correlation = 0.15\n", " \n", " # Global signal regression reduces the association by 0.1\n", " if {{gsr}} == True:\n", " base_correlation -= 0.1\n", " # More detailed parcellations increase the association by 0.1\n", " if {{parcellation}} in [\"Schaefer 200\", \"Glasser MMP\"]:\n", " base_correlation += 0.1\n", " # Pearson correlation increases the association by 0.03\n", " if {{connectivity}} == \"pearson\":\n", " base_correlation += 0.03\n", " # Higher threshold increase the association\n", " base_correlation += {{threshold}} * 0.1\n", "\n", " # Simulate 100 participants with some added variability\n", " brain_behaviour = [base_correlation + np.random.normal(0, 1) for _ in range(100)]\n", " _, p_val = ttest_1samp(brain_behaviour, popmean=0)\n", "\n", " result = {\n", " \"brain_behaviour\": [round(association, 3) for association in brain_behaviour],\n", " \"p_val\": round(p_val, 3),\n", " }\n", "\n", " comet.utils.save_universe_results(result)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With the forking paths and the analysis template defined, we can create and visualize the multiverse:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
| \n", " | Universe | \n", "Decision 1 | \n", "Value 1 | \n", "Decision 2 | \n", "Value 2 | \n", "Decision 3 | \n", "Value 3 | \n", "Decision 4 | \n", "Value 4 | \n", "Decision 5 | \n", "Value 5 | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "Universe_1 | \n", "software | \n", "fMRIprep | \n", "gsr | \n", "True | \n", "parcellation | \n", "AAL | \n", "connectivity | \n", "pearson | \n", "threshold | \n", "0.1 | \n", "
| 1 | \n", "Universe_2 | \n", "software | \n", "fMRIprep | \n", "gsr | \n", "True | \n", "parcellation | \n", "AAL | \n", "connectivity | \n", "pearson | \n", "threshold | \n", "0.3 | \n", "
| 2 | \n", "Universe_3 | \n", "software | \n", "fMRIprep | \n", "gsr | \n", "True | \n", "parcellation | \n", "AAL | \n", "connectivity | \n", "pearson | \n", "threshold | \n", "0.5 | \n", "
| 3 | \n", "Universe_4 | \n", "software | \n", "fMRIprep | \n", "gsr | \n", "True | \n", "parcellation | \n", "AAL | \n", "connectivity | \n", "partial | \n", "threshold | \n", "0.1 | \n", "
| 4 | \n", "Universe_5 | \n", "software | \n", "fMRIprep | \n", "gsr | \n", "True | \n", "parcellation | \n", "AAL | \n", "connectivity | \n", "partial | \n", "threshold | \n", "0.3 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 211 | \n", "Universe_212 | \n", "software | \n", "SPM | \n", "gsr | \n", "False | \n", "parcellation | \n", "Glasser MMP | \n", "connectivity | \n", "partial | \n", "threshold | \n", "0.3 | \n", "
| 212 | \n", "Universe_213 | \n", "software | \n", "SPM | \n", "gsr | \n", "False | \n", "parcellation | \n", "Glasser MMP | \n", "connectivity | \n", "partial | \n", "threshold | \n", "0.5 | \n", "
| 213 | \n", "Universe_214 | \n", "software | \n", "SPM | \n", "gsr | \n", "False | \n", "parcellation | \n", "Glasser MMP | \n", "connectivity | \n", "mutual info | \n", "threshold | \n", "0.1 | \n", "
| 214 | \n", "Universe_215 | \n", "software | \n", "SPM | \n", "gsr | \n", "False | \n", "parcellation | \n", "Glasser MMP | \n", "connectivity | \n", "mutual info | \n", "threshold | \n", "0.3 | \n", "
| 215 | \n", "Universe_216 | \n", "software | \n", "SPM | \n", "gsr | \n", "False | \n", "parcellation | \n", "Glasser MMP | \n", "connectivity | \n", "mutual info | \n", "threshold | \n", "0.5 | \n", "
216 rows × 11 columns
\n", "| \n", " | universe | \n", "brain_behaviour | \n", "p_val | \n", "decisions | \n", "
|---|---|---|---|---|
| 1 | \n", "universe_1 | \n", "[-0.856, 0.494, 0.3, 0.354, -1.421, 1.165, 0.7... | \n", "0.296 | \n", "{'Decision 1': 'software', 'Value 1': 'fMRIpre... | \n", "
| 2 | \n", "universe_2 | \n", "[0.387, -1.957, -0.59, -0.708, 0.863, -0.817, ... | \n", "0.827 | \n", "{'Decision 1': 'software', 'Value 1': 'fMRIpre... | \n", "
| 3 | \n", "universe_3 | \n", "[-0.332, -0.036, -0.766, 2.475, 0.499, 0.224, ... | \n", "0.645 | \n", "{'Decision 1': 'software', 'Value 1': 'fMRIpre... | \n", "
| 4 | \n", "universe_4 | \n", "[-0.13, -0.246, 0.65, -2.288, 0.201, -0.883, -... | \n", "0.968 | \n", "{'Decision 1': 'software', 'Value 1': 'fMRIpre... | \n", "
| 5 | \n", "universe_5 | \n", "[1.564, -0.337, 0.25, 0.642, 0.762, 0.625, 0.2... | \n", "0.768 | \n", "{'Decision 1': 'software', 'Value 1': 'fMRIpre... | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 212 | \n", "universe_212 | \n", "[0.575, 0.409, -0.434, 2.854, 0.657, 0.085, 0.... | \n", "0.003 | \n", "{'Decision 1': 'software', 'Value 1': 'SPM', '... | \n", "
| 213 | \n", "universe_213 | \n", "[1.41, 1.325, -0.57, 0.711, 0.266, 0.516, 1.54... | \n", "0.000 | \n", "{'Decision 1': 'software', 'Value 1': 'SPM', '... | \n", "
| 214 | \n", "universe_214 | \n", "[-1.01, 0.975, 0.556, 2.215, 1.199, 0.502, 0.2... | \n", "0.000 | \n", "{'Decision 1': 'software', 'Value 1': 'SPM', '... | \n", "
| 215 | \n", "universe_215 | \n", "[0.436, 0.755, -0.568, 2.722, -0.013, 0.679, 1... | \n", "0.000 | \n", "{'Decision 1': 'software', 'Value 1': 'SPM', '... | \n", "
| 216 | \n", "universe_216 | \n", "[-0.765, 1.346, 0.881, -1.109, 0.543, -1.718, ... | \n", "0.000 | \n", "{'Decision 1': 'software', 'Value 1': 'SPM', '... | \n", "
216 rows × 4 columns
\n", "