Multimodal Foundation Models: From Specialists to General-Purpose Assistants

By Chunyuan Li, Zhe Gan, Zhengyuan Yang

This monograph presents a comprehensive survey of the taxonomy and evolution of multimodal foundation models that demonstrate vision and vision-language capabilities, focusing on the transition from specialist models to general-purpose assistants.
The focus encompasses five core topics, categorized into two classes; (i) a survey of well-established research areas: multimodal foundation models pre-trained for specific purposes, including two topics - methods of learning vision backbones for visual understanding and text-to-image generation; (ii) recent advances in exploratory, open research areas: multimodal foundation models that aim to play the role of general-purpose assistants, including three topics - unified vision models inspired by large language models (LLMs), end-to-end training of multimodal LLMs, and chaining multimodal tools with LLMs.
The target audience of the monograph is researchers, graduate students, and professionals in computer vision and vision-language multimodal communities who are eager to learn the basics and recent advances in multimodal foundation models.

• Format: Paperback
• Language: English
• ISBN: 1638283362
• ISBN13: 9781638283362
• Release Date: May 2024
• Publisher: Now Publishers
• Length: 230 Pages
• Weight: 0.72 lbs.
• Dimensions: 0.5" x 6.1" x 9.2"