*June 2019*

tl;dr: Estimate the variance of segmentation uncertainty with dropout inference samples. Use the mean for prediction. The idea is quite similar to TTA (test time augmentation).

#### Overall impression

The paper provides a practical way to evaluate the uncertainty (this is the epistemic uncertainty), at a cost at inference time. Refer to Bayesian DL for integration with aleatoric uncertainty.

#### Key ideas

- Sampling with dropout performs better than weight averaging (normal dropout behavior during eval). Sampling dropout performs better than weight averaging after approximation with 6 samples. The performance saturates with 40 samples.
- This comes at inference time cost, but is naively parallelizable.

- The results also show that when the model predicts an incorrect label the model uncertainty is very high.
- Class boundaries usually display high level of uncertainty.
- Objects that are occluded or at a distance from the camera are are uncertain.
- The uncertainty score is inversely proportional to occurrence and accuracy. The model is more confident about classes which are easier and occur more often.
- The accuracy improves when we use a tighter threshold to filter out non-confident results. Uncertainty is an effective measure of accuracy.

#### Technical details

**No need to use dropout layer after every layer.** Get the optimal architecture first by test placing dropout in different places. Then keep using dropout during inference (variational inference).

#### Notes