Proposal: design experiments to maximize inter-item discriminability, rather than simply checking reproducibility after conducting the experiment
If we measure a sample multiple times, then each measurement of that sample is closer to all the other measurements of that sample, as compared to any of the measurements of other samples.
Perfect discriminability
Imperfect discriminability
Discriminability is the probability of a measurement from the same item being closer than a measurement from a different item.
High discriminability: same-item measurements are more similar than across-item measurements
Process each measurement using 192 different pipelines
192=2×2×2×2×4×3
All options represent strategies experts consider useful
For each pre-processing strategy, for each dataset, compute:
Question: does a higher discriminability tend to yield larger effects for known biological signals?
Suppose (xik,yi) follow the Multivar. Additive Noise Setting, where i=1,...,n and k=1,...,s.
There exists an increasing function of D, f(D), which provides a lower bound on the predictive accuracy of a subsequent classification task
A strategy with a higher D provably provides a higher bound on predictive accuracy than a strategy with a lower D
Suppose D1<D2, then since f is increasing, f(D1)<f(D2)
We should use strategies with higher discriminability, as the worst-case for subsequent inference is better than a generic strategy with a lower discriminability
Discr. decreases proportionally with accuracy
Wilk's Λ=det(Σb)+det(Σe)det(Σb)
I2C2=tr(Σb)+tr(Σe)tr(Σb)
"ratio of total variability accounted for between groups"
total dispersion is the sum of between and within-sample dispersions (B and W)
DISCO=n⋅s−nWn−1B
"pseudo F" statistic
(xik,yi) i=1,...,n, k=1,...,s, yi∈ {1,...,Y}
Note that W=P(δ(xik,xik′)<δ(xik,xjk′′)∣yi=yj)= P(yi=yj)P(δ(xik,xik′)<δ(xik,xjk′′),yi=yj) by def conditional probability
Let w(y)=P(yi=yj=y) denote the within-group weights
With ω=∑yw(y), then:
W=ω1∑yw(y)D(y) is the within-group Discriminability
"How discriminable are samples from group y vs group y′, and vice versa?"
Note that B=P(δ(xik,xik′)<δ(xik,xjk′′)∣yi≠yj)= P(yi≠yj)P(δ(xik,xik′)<δ(xik,xjk′′),yi≠yj) by def conditional probability
"How discriminable are samples from group y vs group y′, and vice versa?"
Note that B=P(δ(xik,xik′)<δ(xik,xjk′′)∣yi≠yj)= P(yi≠yj)P(δ(xik,xik′)<δ(xik,xjk′′),yi≠yj) by def conditional probability
Let b(y,y′)=P(yi=y,yj=y′) denote the between group weights
B=β1∑y≠y′b(y,y′)D(y,y′) is the between-group Discriminability
Keyboard shortcuts
↑, ←, Pg Up, k | Go to previous slide |
↓, →, Pg Dn, Space, j | Go to next slide |
Home | Go to first slide |
End | Go to last slide |
Number + Return | Go to specific slide |
b / m / f | Toggle blackout / mirrored / fullscreen mode |
c | Clone slideshow |
p | Toggle presenter mode |
t | Restart the presentation timer |
?, h | Toggle this help |
Esc | Back to slideshow |