LLaVA: Large Language and Vision Assistant: Publications

Building Next-Gen Multimodal Foundation Models for General-Purpose Assistants

LLaVA is an open-source project, collaborating with research community to advance the state-of-the-art in AI. LLaVA represents the first end-to-end trained large multimodal model (LMM) that achieves impressive chat capabilities mimicking spirits of the multimodal GPT-4. The LLaVA family continues growing to support more modalities, capabilities, applications and beyond.

Publications

View by:

Multimodal Foundation Models: From Specialists to General-Purpose Assistants

Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao

September 2023

118-page book/survey on the literature review, evolution, trends and our position on multimodal foundation models. CVPR 2023 Tutorial: https://vlp-tutorial.github.io/2023/

Publication

LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao

June 2023

Publication

Visual Instruction Tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae Lee

NeurIPS 2023 | April 2023

Oral Presentation Project Page: https://llava-vl.github.io/

Preprint

Instruction Tuning with GPT-4

Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, Jianfeng Gao

MSR-TR-2023-35 | April 2023

Published by Microsoft

Project Page: https://instruction-tuning-with-gpt-4.github.io/

Publication

Multimodal Foundation Models: From Specialists to General-Purpose Assistants

Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao

September 2023

118-page book/survey on the literature review, evolution, trends and our position on multimodal foundation models. CVPR 2023 Tutorial: https://vlp-tutorial.github.io/2023/

Publication

LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao

June 2023

Publication

Visual Instruction Tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae Lee

NeurIPS 2023 | April 2023

Oral Presentation Project Page: https://llava-vl.github.io/

Preprint

Instruction Tuning with GPT-4

Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, Jianfeng Gao

MSR-TR-2023-35 | April 2023

Published by Microsoft

Project Page: https://instruction-tuning-with-gpt-4.github.io/

Publication
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao

June 2023

Publication

Visual Instruction Tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae Lee

NeurIPS 2023 | April 2023

Oral Presentation Project Page: https://llava-vl.github.io/

Preprint

Instruction Tuning with GPT-4

Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, Jianfeng Gao

MSR-TR-2023-35 | April 2023

Published by Microsoft

Project Page: https://instruction-tuning-with-gpt-4.github.io/

Publication
Visual Instruction Tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae Lee

NeurIPS 2023 | April 2023

Oral Presentation Project Page: https://llava-vl.github.io/

Preprint
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao

June 2023

Publication

Instruction Tuning with GPT-4

Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, Jianfeng Gao

MSR-TR-2023-35 | April 2023

Published by Microsoft

Project Page: https://instruction-tuning-with-gpt-4.github.io/

Publication
Multimodal Foundation Models: From Specialists to General-Purpose Assistants

Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao

September 2023

118-page book/survey on the literature review, evolution, trends and our position on multimodal foundation models. CVPR 2023 Tutorial: https://vlp-tutorial.github.io/2023/

Publication
Visual Instruction Tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae Lee

NeurIPS 2023 | April 2023

Oral Presentation Project Page: https://llava-vl.github.io/

Preprint
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao

June 2023

Publication

Building Next-Gen Multimodal Foundation Models for General-Purpose Assistants

Publications

Publications by Year

2023

Publications by Research Area

Artificial intelligence

Computer vision

Human language technologies

Medical, health and genomics

Publications by Type

Tech Report

Book

Inproceedings (Conference)

Miscellaneous