Neuronex19

name:opening

#### Ailey in an Hour:<br>(A "Soup-to-Nuts" Pipeline for Analysis of Whole Cleared Brain Data)

.foot[[Joshua T. Vogelstein](https://neurodata.io) | [jovo@jhu.edu](mailto:jovo@jhu.edu) | [neurodata.io](http://neurodata.io) | [@neuro_data](https://twitter.com/neuro_data)]

---

---

### [Reggie](https://neurodata.io/reg/)

Pipeline for analysis of CLARITY/COLM/iDisco/etc. data

- Downloads 2D tiles from cloud
- Cleans artifacts
- Stitch 2D tiles into 3D volume using Terastitcher
- Downsample to atlas resolution
- Nonlinear registration  atlas to data
- Inverts registration to atlas
- Upsample registration to native resolution
- .gold[Detect all cells (works ok)]
- .r[Track all processes (nothing works yet out of the box for us)]
- Upload registered brain and atlas to cloud
- Visualize registered brain in cloud
- Democratize access

---

### [Reggie](https://neurodata.io/reg/)

- [Open source](https://github.com/neurodata/ardent) 
- [Documented with tutorials](https://ardent.neurodata.io/demo/demo.html)
- [pip installable](https://ardent.neurodata.io/install.html) 
- GPU and distributed support 
- Mostly automatic
- .r[Runs in a 1 hour] on LaVision data
- .r[Runs in a 9 hours] on 1 TB data (COLM)

---

### LaVision → Overlay ← Allen Atlas

.footnote[  (from Ailey Crow at Karl Deisseroth)]

---
### Rat COLM → Overlay ← Waxholm

.footnote[  (1 TB image from Ailey Crow at Karl Deisseroth)]

---
### LaVision iDisco Rat  ← Waxholm Atlas

.footnote[  .w[(from Audrey Branch at Michela Gallagher)]]

---
### Human MRI → Overlay ← Desikan

---

.headnote[Allen Atlas + CLARITY + AI Cell Detection (from Ailey Crow at Karl Deisseroth)]

---

### Data Visualization & Democratization

- From local to web viz with atlas overlays ([Neuroglancer](https://github.com/google/neuroglancer))
- From spatial database to flat files ([zarr](https://zarr.readthedocs.io/en/stable/)+[n5](https://github.com/saalfeldlab/n5) coming soon)
- From centralized to distributed storage (no more [BOSS](https://github.com/jhuapl-boss/boss))
- From blosc to brotli compression ([brotli](https://github.com/google/brotli))

---

### Discussion

- LDDMM previously implemented in [ITK](https://github.com/InsightSoftwareConsortium/ITKNDReg/), but lacked GPU support 
- Nonlinear registration works well across modalities and taxa
- Integrating 2D to 3D ([paper here](https://link.springer.com/chapter/10.1007/978-3-030-33226-6_18))
- Integrating missing data and artifacts [paper here](https://doi.org/10.1101/494005)
- Applying now to Janelia [MouseLight](https://www.janelia.org/project-team/mouselight)  project
- Working on BICCN with Brain Architecture  and Pavel Osten
- Working with Sam Wang to integrate atlas with full cerebellum
- Integrating into [scikit-image](https://scikit-image.org) (widely used image library)
- Active development, community participation encouraged!

#### Lessons learned

- New cloud technology enables more scalable data access
- From developer to integrator
- Open-source, community-developed, widely-used, general tools 
- Highly collaborative: Google, Janelia, CSHL, APL, Stanford, JHU...

---

### Papers

- A. Charles, et al. [Towards Community-Driven Big Open Brain Science](mailto:jovo@jhu.edu). Annual Reviews in Neuroscience, 2020.
- J. T. Vogelstein, et al. [A Community-Developed Open-Source Computational Ecosystem for Big Neuro Data](https://www.nature.com/articles/s41592-018-0181-1). Nature Methods, 2018.
- A. Branch, et al. [An optimized protocol for iDISCO+ rat brain clearing, imaging, and analysis](https://doi.org/10.1101/639674). bioRxiv, 2019.
- D. J. Tward, et al. [Diffeomorphic registration with intensity transformation and missing data: Application to 3D digital pathology of Alzheimer's disease](https://doi.org/10.1101/494005). bioRxiv, 2019.
- D. J. Tward, et al. [3D Mapping of Serial Histology Sections with Anomalies Using a Novel Robust Deformable Registration Algorithm](https://link.springer.com/chapter/10.1007/978-3-030-33226-6_18). International Workshop on Multimodal Brain Image Analysis, 2019
- V.  Chandrashekhar, Nonlinear Registration for all. In prep.

---

### Code

- [NeuroData Cloud](http://neurodata.io/nd_cloud)
- [Reggie](https://neurodata.io/reg/)
- [Neuroglancer visualization](https://github.com/google/neuroglancer) (not us)
- [Zarr file format](https://zarr.readthedocs.io/en/stable/) (not us)
- [n5](https://github.com/saalfeldlab/n5) (not us)
- [Brotli compression](https://github.com/google/brotli) (not us)
- [Scikit-Image](https://scikit-image.org) (not us)

---

### Website: [neurodata](https://neurodata.io)

- $\approx$ 100,000 unique visitors to date
- $>$130 countries & $>$3,000 cities
- $>$5,600 twitter followers
- Support data from $>$30 labs (including several NeuroNex PIs)
- Modalities/scales: EM/nano, LM/micro, MRI/milli 
- Taxa: worms, flies, fish, mice, rats, sea lions, coyotes, non-human primates, humans

---
### Acknowledgements

<!-- <div class="small-container">
  <img src="faces/ebridge.jpg"/>
  <div class="centered">Eric Bridgeford</div>
</div>

<div class="small-container">
  <img src="faces/pedigo.jpg"/>
  <div class="centered">Ben Pedigo</div>
</div>
-->

<div class="small-container">
  <img src="faces/yummy.jpg"/>
  <div class="centered">yummy</div>
</div>

<div class="small-container">
  <img src="faces/jklv.jpg"/>
  <div class="centered">family</div>
</div>

<div class="small-container">
  <img src="faces/earth.jpg"/>
  <div class="centered">earth</div>
</div>

<div class="small-container">
  <img src="faces/milkyway.jpg"/>
  <div class="centered">milkyway</div>
</div>

<br>

<div class="small-container">
  <img src="faces/randal.jpg"/>
  <div class="centered">Randal Burns</div>
</div>

<div class="small-container">
  <img src="faces/mim.jpg"/>
  <div class="centered">Michael Miller</div>
</div>

<div class="small-container">
  <img src="faces/dtward.jpg"/>
  <div class="centered">Daniel Tward</div>
</div>

<div class="small-container">
  <img src="faces/vikram.jpg"/>
  <div class="centered">Vikram Chandrashekhar</div>
</div>

<!-- <div class="small-container">
  <img src="faces/jaewon.jpg"/>
  <div class="centered">Jaewon Chung</div>
</div>

<div class="small-container">
  <img src="faces/drishti.jpg"/>
  <div class="centered">Drishti Mannan</div>
</div> -->

<div class="small-container">
  <img src="faces/jesse.jpg"/>
  <div class="centered">Jesse Patsolic</div>
</div>

<div class="small-container">
  <img src="faces/falk_ben.jpg"/>
  <div class="centered">Benjamin Falk</div>
</div>

<div class="small-container">
  <img src="faces/kwame.jpg"/>
  <div class="centered">Kwame Kutten</div>
</div>

<div class="small-container">
    <img src="faces/devin.jpg"/>
    <div class="centered">Devin Crowley</div>
  </div>

<!-- <div class="small-container">
  <img src="faces/tyler.jpg"/>
  <div class="centered">Tyler Tomita</div>
</div>

<div class="small-container">
  <img src="faces/james.jpg"/>
  <div class="centered">James Brown</div>
</div> -->

<br>

</div>
<img src="images/funding/nsf_fpo.png" STYLE="HEIGHT:95px;"/>

---
class:center

.headnote[Questions?]

---
class: middle, inverse

## .center[Extra Slides]

---

.footnote[Precomputed + Brotli + EM (from [Forrest Collman](https://alleninstitute.org/what-we-do/brain-science/about/team/staff-profiles/forrest-collman/) at Allen Institute)]

---

### Centralized vs. Distributed

---

### File Formats

| Property | [BOSS](https://github.com/jhuapl-boss/boss) | CloudVolume + [Precomputed](https://github.com/google/neuroglancer/tree/master/src/neuroglancer/datasource/precomputed)  | [Zarr](https://zarr.readthedocs.io/en/stable/) + [n5](https://github.com/saalfeldlab/n5)
|:--- |:--- |:--- | 
| Developer | Us → [Will Gray Roncal](https://bossdb.org/about) | [Seung Lab](http://seunglab.org/) + [Google](https://github.com/google/neuroglancer/graphs/contributors) | [Biologists](https://www.cggh.org/) + [Saalfeld Lab](https://www.janelia.org/lab/saalfeld-lab)
| Last commit | Aug 9 | Sep 26 | yesterday
| Used by (github) | 0 |  18 | 121 
| User Auth  | yes | no | no 
| Data Management | yes | no | no
| Our current | most | some | none 
| Our plan | no | maybe | probably

---

### Compression

---

### Visualization

| Property | [Fiji/ImageJ](https://imagej.net/Fiji) | [CATMAID](https://catmaid.readthedocs.io/en/stable/) |  [Neuroglancer](https://github.com/google/neuroglancer)
|:--- |:--- |:--- | 
| Developer | [scientists](https://github.com/imagej/imagej/graphs/contributors)| [Janelia](https://github.com/catmaid/CATMAID/graphs/contributors) | [Google](https://github.com/google/neuroglancer/graphs/contributors)+us
| Lat commit | Oct 8 |  yesterday | yesterday
| Used by (github) | ? |  ? | 18 
| Web | no | yes | yes
| skeletons | yes | yes | yes
| volume annotations |  yes | recently | yes 
| analysis | lots | skeletons | none 
| atlas overlay | yes | maybe | yes 
| Our current | local | none | all 
| Our plan | local | none | all

---

### Registration

| Property | [Reggie](https://neurodata.io/reg/) | [NDReg](https://github.com/InsightSoftwareConsortium/ITKNDReg) | [ANTS](https://github.com/ANTsX/ANTs) | [skimage](https://scikit-image.org)
|:--- |:--- |:--- | 
| Developer | [Us](https://github.com/neurodata/ardent/graphs/contributors)| [ITK](https://github.com/InsightSoftwareConsortium/ITKNDReg/graphs/contributors) | [Brian Avants](http://stnava.github.io/) | [300+](https://github.com/scikit-image/scikit-image/graphs/contributors)
| Lat commit | yesterday | Aug 21 | yesterday | yesterday 
| Used by  | just us | nobody? | MRI people | millions?
| Used by (github) | 0 | 0 | ? | 26,500
| Linear registration | yes | yes | yes | no
| LDDMM | yes | yes | approximate | no
| GPU Support | yes | no | no | no
| cleared brains | yes | yes | no | no
| Our current | yes | no | no | no 
| Our plan | yes | no | no | yes

---

### Access Rates

---

### [Benchmarks](https://www.opencpu.org/posts/brotli-benchmarks/)

<div style="display:inline-flex">
  <div>
      Size
       <img src="images/brotli_size.png" alt="Snow" style="height:380px">
  </div>
  <div>
      Decompress
      <img src="images/brotli_decompress.png" alt="nah" style="height:380px">
  </div>
    <div>
    Compress
       <img src="images/brotli_compress.png" alt="nah" style="height:380px">
  </div>
</div>

brotli = red, gzip = blue