Missing Modality
A missing modality occurs when a model expects multiple input signals but one or more are absent, corrupted, delayed, or unavailable at deployment. This is common in multimodal systems, scientific datasets, clinical-style tables, and agent workflows.
For modalities , let indicate whether modality is observed:
A robust model should define a prediction even when not all are one:
where is the observation mask.
The evaluation distribution must include the missingness pattern:
If training assumes but deployment often has missing modalities, aggregate test scores will overstate reliability.
Missingness Patterns
- MCAR: missing completely at random.
- MAR: missingness depends on observed variables.
- MNAR: missingness depends on unobserved variables or the missing value itself.
- Operational missingness: a sensor, tool, file, measurement, or modality is not available in production.
Common Strategies
- Modality dropout during training.
- Mask tokens or learned missing-modality embeddings.
- Late fusion models that can skip absent branches.
- Imputation, with uncertainty tracked separately.
- Fallback single-modality models.
Evaluation Matrix
| Case | Evaluate |
|---|---|
| all modalities present | full-system upper bound |
| one modality missing | graceful degradation |
| important modality corrupted | robustness and false confidence |
| delayed modality | online vs offline behavior |
| target-correlated missingness | bias and deployment risk |
| imputed modality | uncertainty and leakage from imputation |
Checks
- Which modalities are guaranteed at training, validation, and deployment?
- Is missingness random or correlated with the target?
- Does the model know which modality is missing?
- Are missing-modality cases evaluated separately?
- Does imputation leak future or target information?
- Does the model receive an explicit missingness mask?
- Is fallback behavior defined before deployment?
- Are metrics reported by missingness pattern, not only in aggregate?