Multimodal
16 concepts, 16 reviewed and merged.
Applications
Architecture
Evaluation
Foundations
- Audio-Visual LearningVideo comes with its own soundtrack, and the two are aligned for free — nobody had to label anything for the audio at…verified
- Modality FusionTwo streams of information have to become one prediction, and where they meet decides most of what the system can do.verified
- Modality ImbalanceTrain a model on two modalities and one of them usually wins.verified
Generation
- Classifier-Free GuidanceA conditioned diffusion model follows its prompt loosely.verified
- Diffusion ModelTake an image and add noise until nothing is left.verified
- Latent DiffusionRunning diffusion on raw pixels is enormously expensive, and most pixels carry no information anyone would miss.verified
- Video GenerationGenerating a video is not generating many images.verified