Date: March 07, 2021
This version corresponds to the fourth and final release candidate of Connectome Mapper 3 (CMP3). It incorporates the relatively large Pull Request #74 (~270 commits) which includes the following changes such that it marks the end of the release candidate phase.
CMP3 pipeline configuration files adopt
JSONas new format. (PR #76)
CMP3 is compatible with PyPI for installation. (PR #78)
BIDS convention naming of data derived from parcellation atlas adopt now the new BIDS entity
atlas-<atlas_label>to distinguish data derived from different parcellation atlases. The use of the entity
desc-<scale_label>to distinguish between parcellation scale has been replaced by the use of the entity
res-<scale_label>. (PR #79)
dataset_description.jsonfor each derivatives folder has been updated to conform to BIDS version 1.4.0. (PR #79)
Major refactoring of the
cmtklib.configmodule with the addition and replacement of a number of new methods to handle
JSONconfiguration files. (See full diff on GitHub) Configuration files in the old
INIformat can be converted automatically with the help of the two new methods
convert_config_ini_2_json()to detect if configuration files are in the
INIformat and to make the conversion. (PR #76)
Major changes to make
cmpbidsappmanagercompatible with the Python Package Index (
pip) for package distribution and installation. This includes the merge of
setup_gui.py, which have been merged into one
setup.pyand a major refactoring to make
piphappy, as well as the creation of a new
cmp.climodule, migration to
cmp.climodule and refactoring of the scripts
cmpbidsappmanagerwith correction of code style issues and addition of missing docstrings. (PR #78)
Clean parameters to be saved in configuration files with the new API. (PR #74)
Clean output printed by the cmpbidsappmanager Graphical User Interface. (PR #74)
cmtklib.configthe three new functions
print_warningto use different colors to differentiate general info (default color), error (red), command or action (blue), and highlight or warning (yellow). (PR #74)
Review usage and add a note regarding the adoption of the new
JSONformat for configuration files. (PR #76)
Update tutorial on using CMP3 and Datalad for collaboration. (PR #77)
Update installation instruction of
pip install .. (PR #78)
Update list of outputs following the new BIDS derivatives naming convention introduced. (PR #79)
Correct attributes related to the diffusion imaging model type
multishell. (PR #74)
Review code in
cmtklib/connectome.pyfor saving functional connectome files in GRAPHML format. (PR #74)
Update version of datalad and dependencies (PR #77):
Software development life cycle
Improve code coverage by calling the methods
fill_stages_outputs()on each pipeline when executed with coverage. (PR #75)
Improve code coverage by saving in test-01 structural connectome files in MAT and GRAPHML format. (PR #74)
Improve code coverage by saving in test-07 functional connectome files in GRAPHML format. (PR #74)
Update the list of outputs for all tests. (PR #74)
Please check the main pull request 74 page for more details.
Date: February 05, 2021
This version corresponds to the third release candidate of Connectome Mapper 3. In particular, it integrates Pull Request #62 which includes:
MRtrix3 has been updated from
Numpy has been updated from
Nipype has been updated to
Dipy has been updated from
CVXPY has been updated from
Correction of multiple typos.
Update code for Dipy tracking with DTI model following major changes in Dipy 1.0 (Fix reported issue #54).
Update to Dipy 1.3.0 has removed the deprecated warnings related to CVXPY when using MAP_MRI (#63)
Do not set anymore
OMP_NUM_THREADSat execution due to allocation errors raised when using numpy function dot in Dipy.
Software development life cycle
Test 08that runs anatomical and fMRI pipelines with: Lausanne2018 parcellation, FSL FLIRT co-registration, all nuisance regression, linear detrending and scrubbing
Test 09that runs anatomical and dMRI pipelines with: Lausanne2018 parcellation, FSL FLIRT, Dipy SHORE, MRtrix SD_Stream tracking, MRtrix SIFT tractogram filtering
deploy_singularity_latestfrom the workflow for the sake of space on Sylabs.io.
Please check the main pull request 62 page for more details.
Date: February 4, 2021
This version fixes bugs in the second release candidate of Connectome Mapper 3 (v3.0.0-RC2). In particular, it includes:
Software development life cycle
Remove publication of the Singularity image to sylabs.io when the master branch is updated for the sake of space (11GB limit)
CI: remove publication of latest tag image on sylabs.io for space (2 days ago) - commit c765f79
Merge pull request #66 from connectomicslab/v3.0.0-RC2-hotfix1 (3 days ago) - commit 0a2603e
FIX: update g2.node to g2.nodes when saving connectomes as graphml (fix #65) (6 days ago) - commit d629eef
FIX: enabled/disabled gray-out button “Run BIDS App” with Qt Style sheet [skip ci] (3 weeks ago) - commit 10e78d9
MAINT: removed commented lines in cmpbidsappmanager/gui.py [skip ci] (3 weeks ago) - commit 4cc11e7
FIX: check availability of modalities in the BIDS App manager window [skip ci] (3 weeks ago) - commit 80fbee2
MAINT: update copyright year [skip ci] (3 weeks ago) - commit f7d0ffb
CI: delete previous container with latest TAG on sylabs.io [skip ci] (4 weeks ago) - commit 15c9b18
DOC: update tag to latest in runonhpc.rst [skip ci] (4 weeks ago) - commit 3165bcc
CI: comment lines related to version for singularity push (4 weeks ago) - commit 3952d46
Date: December 24, 2020
This version corresponds to the second release candidate of Connectome Mapper 3. In particular, it integrates Pull Request #45 which includes:
Add a tracker to support us seeking for new funding. User is still free to opt-out and disable it with the new option flag
Add options suggested by Theaud G et al. (2020) to better control factors having impacts on reproducibility. It includes:
Set the number of ITK threads used by ANTs for registration (option flag
Set the seed of the random number generator used by ANTs for registration (option flag
Set the seed of the random number generator used by MRtrix for tractography seeding and track propagation (option flag
Full support of Singularity (see Software development life cycle).
A number of classes describing interfaces to
mrtrix3have been moved from
Capitalize the first letter of a number of class names.
Lowercase a number of variable names in
Graphical User Interface
Improve display of qpushbuttons with images in the GUI (PR #52).
Make the window to control BIDS App execution scrollable.
Allow to specify a custom output directory.
Tune new options in the window to control BIDS App multi-threading (OpenMP and ANTs) and random number generators (ANTs and MRtrix).
Full code documentation with numpydoc-style docstrings.
API documentation page at readthedocs.
Fix the error reported in #17 if it is still occuring.
Review statements for creating contents of BIDS App entrypoint scripts to fix issue with Singularity converted images reported in #47.
dcpackage inside the BIDS App to fix the issue with FSL BET reported in #50.
libopenblaspackage inside the BIDS App to fix the issue with FSL EDDY_OPENMP reported in #49.
Software development life cycle
Add a new job
test_docker_fmrithat test the fMRI pipeline.
deploy_singularity_releasejobs to build, test and deploy the Singularity image in CircleCI (PR #56).
Please check the main pull request 45 page for more details.
Date: August 03, 2020
This version corresponds to the first release candidate of Connectome Mapper 3. In particular, it integrates Pull Request #40 where the last major changes prior to its official release have been made, which includes in particular:
Migration to Python 3
Fixes automatically with
2to3and manually a number of Python 2 statements invalid in python 3 including the print() function
Correct automatically PEP8 code style issues with autopep8
Correct manually a number of code stly issues reported by Codacy (bandits/pylints/flake8)
Major dependency upgrades including:
dipy 0.15 -> 1.0and related code changes in
cmtklib/interfaces/dipy(Check here for more details about Dipy 1.0 changes)
Interface for tractography based on Dipy DTI model and EuDX tractography, which has been drastically changed in Dipy 1.0, has not been updated yet, It will be part of the next release candidate.
nipype 1.1.8 -> 1.5.0
pybids 0.9.5 -> 0.10.2
pydicom 1.4.2 -> 2.0.0
networkX 2.2 -> 2.4
statsmodels 0.9.0 -> 0.11.1
obspy 1.1.1 -> 1.2.1
traits 5.1 -> 6.0.0
traitsui 6.0.0 -> 6.1.3
numpy 1.15.4 -> 1.18.5
matplotlib 1.1.8 -> 1.5.0
fsleyes 0.27.3 -> 0.33.0
mne 0.17.1 -> 0.20.7
sphinx 1.8.5 -> 3.1.1
sphinx_rtd_theme 0.4.3 -> 0.5.0
recommonmark 0.5.0 -> 0.6.0
Option to run Freesurfer recon-all in parallel and to specify the number of threads used by not only Freesurfer but also all softwares relying on OpenMP for multi-threading. This can be achieved by running the BIDS App with the new option flag
Changes in BIDS derivatives
Renamed connectivity graph files to better conform to the BIDS extension proposal on connectivity data schema. They are now saved by default in a TSV file as a list of edges.
Functions to save and load pipeline configuration files have been moved to
Major changes in how inspection of stage/pipeline outputs with the graphical user interface (cmpbidsappmanager) which was not working anymore after migration to Python3
Fixes to compute the structural connectivity matrices following migration to python 3
Fixes to computes ROI volumetry for Lausanne2008 and NativeFreesurfer parcellation schemes
Add missing renaming of the ROI volumetry file for the NativeFreesurfer parcellation scheme following BIDS
Create the mask used for computing peaks from the Dipy CSD model when performing Particle Filtering Tractography (development still on-going)
Add missing renaming of Dipy tensor-related maps (AD, RD, MD) following BIDS
Remove all references to use Custom segmentation / parcellation / diffusion FOD image / tractogram, inherited from CMP2 but not anymore functional following the adoption of BIDS standard inside CMP3.
Software development life cycle
Use Codacy to support code reviews and monitor code quality over time.
- Add new regression tests in CircleCI to improve code coverage:
Test 01: Lausanne2018 (full) parcellation + Dipy SHORE + Mrtrix3 SD_STREAM tractography
Test 02: Lausanne2018 (full) parcellation + Dipy SHORE + Mrtrix3 ACT iFOV2 tractography
Test 03: Lausanne2018 (full) parcellation + Dipy SHORE + Dipy deterministic tractography
Test 04: Lausanne2018 (full) parcellation + Dipy SHORE + Dipy Particle Filtering tractography
Test 05: Native Freesurfer (Desikan-Killiany) parcellation
Test 06: Lausanne2008 parcellation (as implemented in CMP2)
Moved pipeline configurations for regression tests in CircleCI from
Moved lists of expected regression test outputs in CircleCI from
Please check the pull request 40 page for more details.
Date: June 02, 2020
This version integrates Pull Request #33 which corresponds to the last beta release that still relies on Python 2.7. It includes in particular:
fslview(now deprecated), which now included in the conda environment of the GUI (
Computes of ROI volumetry stored in
<output_dir>/sub-<label>(/ses<label>)/anatfolder, recognized by their
_stats.tsvfile name suffix.
MATRIX_RNG_SEEDenvironment variable (used by MRtrix) and seed for the numpy random number generator (
Fixes the output inspector window of the cmpbidsappmanager (GUI) that fails to find existing outputs, after adoption of /bids_dir and /output_dir in the bidsapp docker image.
Fixes the way to get the list of networkx edge attributes in
ConnectomeStagefor the output inspector window of the cmpbidsappmanager (GUI)
Added missing package dependencies (
vtk) that fixes dipy_CSD execution error when trying to import module actor from dipy.viz to save the results in a png
Fixes a number of unresolved references identified by pycharm code inspection tool
Interfaces for fMRI processing were moved to
Interface for fMRI connectome creation (
rsfmri_conmat) moved to
Please check the pull request 33 page for change details.
Date: March 26, 2020
This version integrates Pull Request #28 which includes in summary:
A major revision of continuous integration testing and deployment with CircleCI which closes Issue 14 integrates an in-house dataset published and available on Zenodo @ https://doi.org/10.5281/zenodo.3708962.
Multiple bug fixes and enhancements incl. close Issue 30 , update mrtrix3 to RC3 version, bids-app run command generated by the GUI, location of the configuration and log files to be more BIDS compliant.
Change in tagging beta version which otherwise might not be meaningfull in accordance with the release date (especially when the expected date is delayed due to unexpected errors that might take longer to be fixed than expected).
Please check the pull request 28 page for a full list of changes.
Date: February 27, 2020
This version addresses multiple issues to make successful conversion and run of the CMP3 BIDS App on HPC (Clusters) using Singularity.
Revised the build of the master and BIDS App images:
Install locales and set
$LANGto make freesurfer hippocampal subfields and brainstem segmentation (matlab-based) modules working when run in the converted SIngularity image
BIDS input and output directories inside the BIDS App container are no longer the
/output_dir. .. warning:: this might affect the use of Datalad container (To be confirmed.)
Fix the branch of mrtrix3 to check out
Fix the configuration of CircleCI to not use Docker layer cache feature anymore as this feature is not included anymore in the free plan for open source projects.
Improved documentation where the latest version should be dynamically generated everywhere it should appear.
Date: February 06, 2020
Implementation of an in-house Nipype interface to AFNI 3DBandPass which can handle to check output as ..++orig.BRIK or as ..tlrc.BRIK (The later can occur with HCP preprocessed fmri data)
Date: January 24, 2020
Updated multi-scale parcellation with a new symmetric version:
The right hemisphere labels were projected in the left hemisphere to create a symmetric version of the multiscale cortical parcellation proposed by Cammoun2012.
For scale 1, the boundaries of the projected regions over the left hemisphere were matched to the boundaries of the original parcellation for the left hemisphere.
This transformation was applied for the rest of the scales.
Updated documentation with list of changes