What are the dimensions of a good testing dataset?

[satra]

• Motion and other artifacts
• Different types of fieldmaps or none
• Well known task paradigms

let's try to determine the different dimensions that we should have.

@oesteban @mgxd

[gkiar]

• is cross species data available?

For structural imaging/brain extraction:

• ground truth segmentations

[gkiar]

For determining the grade of a dataset/benchmark, which would help with interpretability, we could create an exhaustive list of dimensions and a rubric for how to determine the standard based on which subset of them are checked

[arueter1]

Hey Audrey, thought you might be a good person to be a part of this conversation. @hough129

[gkiar]

Here is a first pass at a list or properties we want to cover in the dataset:

Developmental Status:
Species:
  - State/Task
  - Contrast agents
Clinical Population:
Evaluation Standard:
Data quality:
  - Motion
  - Artifacts
Modality:
  - Acquisition protocol type:
    - Scanner
    - Magnet strength
  - Data quantity
Ancillary data availability:

It's possible this list could be enhanced with other BIDS-esque metadata, too...

[audreymhoughton]

https://docs.google.com/document/d/1QrJ0qyLTtYipAFn_4WhqqE-HmYHrMGG5ChLAeWDA5fY/edit?usp=sharing

I made a collaborative Google document for us to list this information on together.