L2M PI 21

### Lifelong Learning: <br>Theory and Context

PI: Joshua T. Vogelstein, [JHU](https://www.jhu.edu/) <br>
Co-PI: Vova Braverman, [JHU](https://www.jhu.edu/) <br>

Ali Geisa, Jayanta Dey, Will LeVine, Hayden Helm, Ronak Mehta,
Carey E. Priebe

![:scale 30%](images/neurodata_blue.png)

---

### Outline

- background
- theoretically motivate lifelong learning metrics
- properly situation lifelong learning within hierarchy of learning paradigms

---
class: middle 
# .center[Background]

---

### What is learning (Valiant)?

![:scale 100%](images/weak-learning.png)

basically, doing better than chance with enough data

![:scale 100%](images/strong-learning.png)

basically, doing arbitrarily well with enough data

.ye[weak learning theorem states that if a problem is weakly learnable, then it is also strongly learnable]

---

### Limitations of this formal definition

- there is only 1 task 
- requires large sample sizes for theory to be relevant 
- all data are from the same fixed distribution 
-  evaluation is with respect to the data distribution

---

### What is learning (Mitchell)?

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

-- Tom Mitchell,  1997

- Pro's
  - multiple tasks 
  - uncouples experience (data) with tasks
  - explicit mention of improving due to data 
  - implicitly requires transfer 
- Con's 
  - not formalized

---
class: middle 
# .center[Jovo Framework]

---

##### In-Distribution vs Out-of-Distribution Learning

![:scale 100%](images/learning-schematics.png)

- the key differences
  - evaluation distribution is uncoupled from data distributions
  - multiple datasets & distributions

---

### Formalizing OOD Learnability

![:scale 100%](images/weak-ood-learnability.png)

basically, using non-task data to improve performance at all

![:scale 100%](images/strong-ood-learnability.png)

basically, using non-task data to perform arbitrarily well

---

### Quantifying learning

The above two definitions enable one to assess .ye[whether] an agent $f$ has learned, but not .ye[how much] it learned.

![:scale 100%](images/learning-efficiency.png)

basically, using non-task data to improve performance over what it could achieve using only task data

---

### Weak OOD Learner Theorem

Theorem 1: With *only* out-of-distribution data, there exists some problems that are weakly, but not strongly, learnable.

This implies that OOD learning is different *in kind* (and .ye[harder]) from in-distribution learning.

---

### Transfer Learning Theorem

Theorem 2: Weak OOD learnability implies transfer learnability (i.e., learning efficiency > 1).  That is, if one can weakly learn, one can also transfer learn, but not necessarily vice versa.

- This implies that transfer learnability is a fundamental property of learning problems
- In other words, inability to transfer is equivalent to inability to learn at all.

If one cannot transfer, one cannot learn in any meaningful sense.

---

### Learning Efficiency Applications

Each of the previous definitions are all special cases of $LE^t_f(\mathbf{S}^A, \mathbf{S}^B)$, for specific choices of  $\mathbf{S}^A$ and $\mathbf{S}^B$

- Learning: $\mathbf{S}^A=\mathbf{S}\_0$ and $\mathbf{S}^B=\mathbf{S}\_n$.
- Transfer learning: $\mathbf{S}^A=\mathbf{S}\_n^t$ and $\mathbf{S}^B=\mathbf{S}\_n$.
- Multitask learning: for each $t$, $\mathbf{S}^A=\mathbf{S}\_n^t$ and $\mathbf{S}^B=\mathbf{S}\_n$.
- Forward learning: $\mathbf{S}^A=\mathbf{S}\_n^t$ and $\mathbf{S}^B=\mathbf{S}\_n^{< t}$.
- Backward learning: $\mathbf{S}^A=\mathbf{S}\_n^{< t}$ and $\mathbf{S}^B=\mathbf{S}\_n$.

---

### Lifelong Learning $\subsetneq$ OOD learning

![:scale 65%](images/nested-learning-schematic.png)

---

### Biological learning is on top

![:scale 100%](images/learning-table.png)

---

### Discussion

- unified definition and quantification of learning 
- presented hierarchy of learning paradigms 
- limitations of current framework: in biology, there are no tasks

---

### Transition Opportunities

### [http://proglearn.neurodata.io/](http://proglearn.neurodata.io/)

![:scale 80%](images/proglearn_webpage.png)

- code continues to improve (no time to discuss here)
- ensembling representations (rather than decision rules) continues to be a promising path to solving OOD (including lifelong) and eventually biological learning

---

### Acknowledgements

<!-- <div class="small-container">
  <img src="faces/ebridge.jpg"/>
  <div class="centered">Eric Bridgeford</div>
</div>

<div class="small-container">
  <img src="faces/pedigo.jpg"/>
  <div class="centered">Ben Pedigo</div>
</div>

<div class="small-container">
  <img src="faces/jaewon.jpg"/>
  <div class="centered">Jaewon Chung</div>
</div> -->

<div class="small-container">
  <img src="faces/yummy.jpg"/>
  <div class="centered">yummy</div>
</div>

<div class="small-container">
  <img src="faces/family.jpg"/>
  <div class="centered">family</div>
</div>

<div class="small-container">
  <img src="faces/earth.jpg"/>
  <div class="centered">earth</div>
</div>

<div class="small-container">
  <img src="faces/milkyway.jpg"/>
  <div class="centered">milkyway</div>
</div>

##### JHU

<div class="small-container">
  <img src="faces/cep.png"/>
  <div class="centered">Carey Priebe</div>
</div>

<!-- <div class="small-container">
  <img src="faces/bruce_rosen.jpg"/>
  <div class="centered">Bruce Rosen</div>
</div>

<div class="small-container">
  <img src="faces/kent.jpg"/>
  <div class="centered">Kent Kiehl</div>
</div> -->

<!-- <div class="small-container">
  <img src="faces/mim.jpg"/>
  <div class="centered">Michael Miller</div>
</div>

<div class="small-container">
  <img src="faces/dtward.jpg"/>
  <div class="centered">Daniel Tward</div>
</div> -->

<!-- <div class="small-container">
  <img src="faces/vikram.jpg"/>
  <div class="centered">Vikram Chandrashekhar</div>
</div>

<div class="small-container">
  <img src="faces/drishti.jpg"/>
  <div class="centered">Drishti Mannan</div>
</div> -->

<div class="small-container">
  <img src="faces/meghana.png"/>
  <div class="centered">Meghana Madhya</div>
</div>

<div class="small-container">
  <img src="faces/ronak.jpg"/>
  <div class="centered">Ronak Mehta</div>
</div>

<div class="small-container">
  <img src="faces/jayanta.jpg"/>
  <div class="centered">Jayanta Dey</div>
</div>

<div class="small-container">
  <img src="faces/will.jpg"/>
  <div class="centered">Will LeVine</div>
</div>

<div class="small-container">
  <img src="faces/hayden.png"/>
  <div class="centered">Hayden Helm</div>
</div>

<div class="small-container">
  <img src="faces/rguo.jpg"/>
  <div class="centered">Richard Gou</div>
</div>

<div class="small-container">
  <img src="faces/alig.jpg"/>
  <div class="centered">Ali Geisa</div>
</div>

##### Microsoft Research

<div class="small-container">
  <img src="faces/chwh-180x180.jpg"/>
  <div class="centered">Chris White</div>
</div>

<div class="small-container">
  <img src="faces/weiwei.jpg"/>
  <div class="centered">Weiwei Yang</div>
</div>

<div class="small-container">
  <img src="faces/jolarso150px.png"/>
  <div class="centered">Jonathan Larson</div>
</div>

<div class="small-container">
  <img src="faces/brtower-180x180.jpg"/>
  <div class="centered">Bryan Tower</div>
</div>

##### DARPA L2M: All code open source and reproducible from [proglearn.neurodata.io/](http://proglearn.neurodata.io/)

{[BME](https://www.bme.jhu.edu/),[CIS](http://cis.jhu.edu/), [ICM](https://icm.jhu.edu/), [KNDI](http://kavlijhu.org/)}@[JHU](https://www.jhu.edu/) | [neurodata](https://neurodata.io)
<br>
[jovo@jhu.edu](mailto:j1c@jhu.edu) | <http://neurodata.io/talks> | [@neuro_data](https://twitter.com/neuro_data)

</div>

---
background-image: url(images/l_and_v.jpeg)

.footnote[Questions?]