{ "cells": [ { "cell_type": "markdown", "id": "76ad5adb", "metadata": {}, "source": [ "# Multiverse recipes\n", "\n", "There are many ways of specifying and customizing complex multiverse analyses. The following code snippets aim to provide some examples/recipes for this." ] }, { "cell_type": "markdown", "id": "8f8786f7", "metadata": {}, "source": [ "## Specifying classes/functions\n", "\n", "You can define the method either as a direct function call or by passing the arguments separately in an `args` dictionary. \n", "Both approaches are equivalent:" ] }, { "cell_type": "code", "execution_count": 8, "id": "47e2ce99", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
UniverseDecision 1Value 1
0Universe_1dfcSW29
1Universe_2dfcSW49
\n", "
" ], "text/plain": [ " Universe Decision 1 Value 1\n", "0 Universe_1 dfc SW29\n", "1 Universe_2 dfc SW49" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from comet.multiverse import Multiverse\n", "\n", "forking_paths = {\n", " \"dfc\": [\n", " {\n", " \"name\": \"SW29\",\n", " \"func\": \"comet.connectivity.SlidingWindow(ts, windowsize=29).estimate()\"\n", " },\n", " {\n", " \"name\": \"SW49\",\n", " \"func\": \"comet.connectivity.SlidingWindow\",\n", " \"args\": {\n", " \"time_series\": \"ts\",\n", " \"windowsize\": 49}\n", " }\n", "]}\n", "\n", "def analysis_template():\n", " import comet\n", " \n", " ts = comet.utils.load_example(\"time_series\")\n", " dfc = {{dfc}}\n", "\n", "mverse = Multiverse(name=\"example_mv_recipes\")\n", "mverse.create(analysis_template, forking_paths)\n", "mverse.summary()" ] }, { "cell_type": "markdown", "id": "1d9353c5", "metadata": {}, "source": [ "From a functional perspective one could even pass the entire function call as a literal (as indicated by the `$` sign). \n", "However, as this does not allow users to specify a custom name, it will look messy in the outputs." ] }, { "cell_type": "code", "execution_count": 2, "id": "11e1d637", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
UniverseDecision 1Value 1
0Universe_1dfc$comet.connectivity.SlidingWindow(ts, windowsi...
\n", "
" ], "text/plain": [ " Universe Decision 1 Value 1\n", "0 Universe_1 dfc $comet.connectivity.SlidingWindow(ts, windowsi..." ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from comet.multiverse import Multiverse\n", "\n", "forking_paths = {\n", " \"dfc\": [\"$comet.connectivity.SlidingWindow(ts, windowsize=69).estimate()\"]\n", "}\n", "\n", "def analysis_template():\n", " import comet\n", " \n", " ts = comet.utils.load_example(\"time_series\")\n", " dfc = {{dfc}}\n", "\n", "mverse = Multiverse(name=\"example_mv_recipes\")\n", "mverse.create(analysis_template, forking_paths)\n", "mverse.summary()" ] }, { "cell_type": "markdown", "id": "c6da8772", "metadata": {}, "source": [ "This works the same for graph related functions, and also any other function/class not included in Comet:" ] }, { "cell_type": "code", "execution_count": 3, "id": "c14ddcb0", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
UniverseDecision 1Value 1
0Universe_1functionscomet clustering
1Universe_2functionsbct clustering
2Universe_3functionsscipy zscore
\n", "
" ], "text/plain": [ " Universe Decision 1 Value 1\n", "0 Universe_1 functions comet clustering\n", "1 Universe_2 functions bct clustering\n", "2 Universe_3 functions scipy zscore" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from comet.multiverse import Multiverse\n", "\n", "forking_paths = {\n", " \"functions\": [\n", " {\n", " \"name\": \"comet clustering\",\n", " \"func\": \"comet.graph.clustering_coef(G)\",\n", " },\n", " {\n", " \"name\": \"bct clustering\",\n", " \"func\": \"bct.clustering_coef_wu(G)\",\n", " },\n", " {\n", " \"name\": \"scipy zscore\",\n", " \"func\": \"scipy.stats.zscore(G)\",\n", " }\n", "]}\n", "\n", "def analysis_template():\n", " import comet\n", " import bct\n", " import scipy\n", " \n", " G = [[0, 1, 2], [1, 0, 3], [2, 3, 0]]\n", " result = {{functions}}\n", "\n", "mverse = Multiverse(name=\"example_mv_recipes\")\n", "mverse.create(analysis_template, forking_paths)\n", "mverse.summary()" ] }, { "cell_type": "markdown", "id": "fb461606", "metadata": {}, "source": [ "## Varying parameters for same connectivity measures\n", "\n", "One can either define the the complete measures in the forking paths, or only substitute the arguments in the analysis template:" ] }, { "cell_type": "code", "execution_count": 4, "id": "068434f7", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
UniverseDecision 1Value 1
0Universe_1swSW29
1Universe_2swSW39
2Universe_3swSW49
\n", "
" ], "text/plain": [ " Universe Decision 1 Value 1\n", "0 Universe_1 sw SW29\n", "1 Universe_2 sw SW39\n", "2 Universe_3 sw SW49" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from comet.multiverse import Multiverse\n", "\n", "forking_paths = {\n", " \"sw\": [\n", " {\n", " \"name\": \"SW29\",\n", " \"func\": \"comet.connectivity.SlidingWindow(ts, windowsize=29).estimate()\"\n", " },\n", " {\n", " \"name\": \"SW39\",\n", " \"func\": \"comet.connectivity.SlidingWindow(ts, windowsize=39).estimate()\"\n", " },\n", " {\n", " \"name\": \"SW49\",\n", " \"func\": \"comet.connectivity.SlidingWindow(ts, windowsize=49).estimate()\"\n", " },\n", "]}\n", "\n", "def analysis_template():\n", " import comet\n", " \n", " ts = comet.utils.load_example(\"time_series\")\n", " dfc = {{sw}}\n", "\n", "mverse = Multiverse(name=\"example_mv_recipes\")\n", "mverse.create(analysis_template, forking_paths)\n", "mverse.summary()" ] }, { "cell_type": "code", "execution_count": 5, "id": "f3b76f5e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
UniverseDecision 1Value 1Decision 2Value 2
0Universe_1size29shaperectangular
1Universe_2size29shapegaussian
2Universe_3size39shaperectangular
3Universe_4size39shapegaussian
4Universe_5size49shaperectangular
5Universe_6size49shapegaussian
\n", "
" ], "text/plain": [ " Universe Decision 1 Value 1 Decision 2 Value 2\n", "0 Universe_1 size 29 shape rectangular\n", "1 Universe_2 size 29 shape gaussian\n", "2 Universe_3 size 39 shape rectangular\n", "3 Universe_4 size 39 shape gaussian\n", "4 Universe_5 size 49 shape rectangular\n", "5 Universe_6 size 49 shape gaussian" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from comet.multiverse import Multiverse\n", "\n", "forking_paths = {\n", " \"size\": [29, 39, 49],\n", " \"shape\": [\"rectangular\", \"gaussian\"]\n", " }\n", "\n", "def analysis_template():\n", " import comet\n", " \n", " ts = comet.utils.load_example(\"time_series\")\n", " dfc = comet.connectivity.SlidingWindow(time_series=ts, windowsize={{size}}, shape={{shape}}).estimate()\n", "\n", "mverse = Multiverse(name=\"example_mv_recipes\")\n", "mverse.create(analysis_template, forking_paths)\n", "mverse.summary()" ] }, { "cell_type": "markdown", "id": "ce904035", "metadata": {}, "source": [ "A more complex design are different connectivity measures which have different parameters. Two potential solutions are:\n", "\n", "1) Give the full decision space in the forking paths:" ] }, { "cell_type": "code", "execution_count": 6, "id": "d604de0c", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
UniverseDecision 1Value 1
0Universe_1swSW29
1Universe_2swSW39
2Universe_3swFLS50
3Universe_4swFLS100
\n", "
" ], "text/plain": [ " Universe Decision 1 Value 1\n", "0 Universe_1 sw SW29\n", "1 Universe_2 sw SW39\n", "2 Universe_3 sw FLS50\n", "3 Universe_4 sw FLS100" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from comet.multiverse import Multiverse\n", "\n", "forking_paths = {\n", " \"sw\": [\n", " {\n", " \"name\": \"SW29\",\n", " \"func\": \"comet.connectivity.SlidingWindow(ts, windowsize=29).estimate()\"\n", " },\n", " {\n", " \"name\": \"SW39\",\n", " \"func\": \"comet.connectivity.SlidingWindow(ts, windowsize=39).estimate()\"\n", " },\n", " {\n", " \"name\": \"FLS50\",\n", " \"func\": \"comet.connectivity.FlexibleLeastSquares(ts, mu=50).estimate()\"\n", " },\n", " {\n", " \"name\": \"FLS100\",\n", " \"func\": \"comet.connectivity.FlexibleLeastSquares(ts, mu=100).estimate()\"\n", " }\n", "]}\n", "\n", "def analysis_template():\n", " import comet\n", " \n", " ts = comet.utils.load_example(\"time_series\")\n", " dfc = {{sw}}\n", "\n", "mverse = Multiverse(name=\"example_mv_recipes\")\n", "mverse.create(analysis_template, forking_paths)\n", "mverse.summary()" ] }, { "cell_type": "markdown", "id": "f1d2efe1", "metadata": {}, "source": [ "2) Use a conditional logic in the analysis script combined with a config rule. Here, combinations of `SlidingWindow` and `mu` as well as `FlexibleLeastSquares` and `windowsize` are invalid, so we exclude them from the multiverse:" ] }, { "cell_type": "code", "execution_count": 7, "id": "0c11e440", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Exclusion summary\n", "-----------------\n", "Total number of universes: 8\n", " - Removed 8 duplicate universes (set 'deduplicate' to False if you want to keep them)\n", " - Set 'mu' to NaN for universes matching {'method': 'SW'} (4 total).\n", " - Set 'windowsize' to NaN for universes matching {'method': 'FLS'} (4 total).\n", "\n", "8 universes remain for analysis.\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
UniverseDecision 1Value 1Decision 2Value 2Decision 3Value 3
0Universe_1methodSWwindowsize29.0muNaN
1Universe_2methodSWwindowsize29.0muNaN
2Universe_3methodSWwindowsize39.0muNaN
3Universe_4methodSWwindowsize39.0muNaN
4Universe_5methodFLSwindowsizeNaNmu50.0
5Universe_6methodFLSwindowsizeNaNmu100.0
6Universe_7methodFLSwindowsizeNaNmu50.0
7Universe_8methodFLSwindowsizeNaNmu100.0
\n", "
" ], "text/plain": [ " Universe Decision 1 Value 1 Decision 2 Value 2 Decision 3 Value 3\n", "0 Universe_1 method SW windowsize 29.0 mu NaN\n", "1 Universe_2 method SW windowsize 29.0 mu NaN\n", "2 Universe_3 method SW windowsize 39.0 mu NaN\n", "3 Universe_4 method SW windowsize 39.0 mu NaN\n", "4 Universe_5 method FLS windowsize NaN mu 50.0\n", "5 Universe_6 method FLS windowsize NaN mu 100.0\n", "6 Universe_7 method FLS windowsize NaN mu 50.0\n", "7 Universe_8 method FLS windowsize NaN mu 100.0" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from comet.multiverse import Multiverse\n", "\n", "forking_paths = {\n", " \"method\": [\"SW\", \"FLS\"],\n", " \"windowsize\": [29, 39],\n", " \"mu\": [50, 100] \n", " }\n", "\n", "config = {\n", " \"exclude\": [\n", " [{\"method\": \"SW\"}, \"mu\"],\n", " [{\"method\": \"FLS\"}, \"windowsize\"]\n", " ]\n", "}\n", "\n", "def analysis_template():\n", " import comet\n", "\n", " ts = comet.utils.load_example(\"time_series\")\n", " if {{method}} == \"SW\":\n", " dfc = comet.connectivity.SlidingWindow(ts, windowsize={{windowsize}}).estimate()\n", " elif {{method}} == \"FLS\":\n", " dfc = comet.connectivity.FlexibleLeastSquares(ts, mu={{mu}}).estimate()\n", "\n", "mverse = Multiverse(name=\"example_mv_recipes\")\n", "mverse.create(analysis_template, forking_paths, config)\n", "mverse.summary()" ] } ], "metadata": { "kernelspec": { "display_name": "comet-test", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.0" } }, "nbformat": 4, "nbformat_minor": 5 }