ReproNim

Describe

Enable reproducibility by design!

To provide the tools necessary for reproducible analyses. The analyses must also generate queryable, machine-readable data and results using interoperable and standardized data models. We aim to:

  • Design interoperable Data Models, Workflows and Linked Results
  • Create reusable workflows with provenance
  • Test computational reproducibility and robustness
  • Integrate tools in Brainverse

Neuroimaging data have become richer, with larger cohorts of participants, and a greater variety of acquisitions. A large number of useful analysis methods are now available, and many pipeline tools make such analysis efficient.

  • Interpreting and comparing scientific results and enabling reusable analysis require understanding provenance, i.e. how the data were generated and processed.
  • To be useful, the provenance must be understandable, easily communicated, and captured automatically in machine accessible form.
  • The NIDM specification captures data, analyses details across software and workflow tools, and results.

Currently we are focused on the following projects. You can help us by trying them, contributing to them, or by sharing your ideas on how to improve them.

NIDM

Describe experiment setup and data, analysis details, and results in a structured manner. This is a collaboration with the Neuroimaging Data Sharing Working group, to develop and maintain data models and software to support rich and structured description of research workflows.

  • To create and support a community using standards
  • Tools to help researchers describe their data, software, and results
  • Tools to help developers integrate structured description into software

These tools will include publishing, visualizing, and validating data described using NIDM. It will enable interoperability with existing efforts (e.g., BIDS, DataLad, Nipype). Example: Keator et al., 2017

  • Python-based API following the simple organizational structure of NIDM-Experiment documents. Functions to create, query, export, import, and transform NIDM documents.
  • NIDM Terminology / Ontology: NIDM ontology reuses existing terms from several other where appropriate definitions exist. Current work is focused on data descriptions, DICOM ontology, and a scientific workflow ontology.

BrainVerse

ReproNim is developing BrainVerse to help researchers manage, track and share information in a comprehensive format. Brainverse is an open-source, cross-platform desktop application to enable researchers add to reproducible practices into every project.

  • An electronic laboratory notebook built as a cross platform desktop application
  • Enables users to plan experiments, collect, analyze and reuse data, and collaborate
  • Adds semantic annotation to data with relevant metadata based on NeuroImaging data Model (NIDM) making experimental neuroimaging study more reproducible, and making data FAIR
  • Intercept the research workflow at planning stage, curating (raw, processed, results) data at the source and helping users to annotate every step of the process.

We are looking for beta users of the software and JavaScript developers to extend BrainVerse.

Testkraut

Create a comprehensive framework for robustness testing. This will enable continuous evaluation of algorithms under different operating system environments. Neuroimaging relies on numerical computations and results can depend on:

  • workflow used by scientist
  • data used as input
  • external software used in the analysis
  • computational environment

Testing analysis in various computational environments allows to:

  • learn about the analysis limitations
  • increase confidence in the analysis before publishing the results
  • check the results obtained using a newer versions of external software
  • or computational environment any time after publishing

ReproNim's Testkraut implementation will allow researchers to continuously evaluate the performance of algorithms developed in the community and to choose the most appropriate tools for a given analysis task.