Fusion architectures

Multimodal fusion is the mechanism by which information from different modalities is combined into a unified representation. The depth at which fusion occurs — early, intermediate, or late — determines the model capacity for cross modal reasoning and its computational cost.