Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
Vibhav Vineet, Xin Wang, Neel Joshi
October 2024
Vibhav Vineet
Principal Researcher
Publications
Publications by Year
-
EUREKA: Evaluating and Understanding Large Foundation Models
Vidhisha Balachandran, Jingya Chen, Neel Joshi, Besmira Nushi, Hamid Palangi, Eduardo Salinas, Vibhav Vineet, James Woffinden-Luey, Safoora Yousefi
MSR-TR-2024-33 | September 2024
Published by Microsoft
Exposing the Achilles’ Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning
Joykirat Singh, Akshay Nambi, Vibhav Vineet
June 2024
Peekaboo: Interactive video generation via masked-diffusion
Yash Jain, Anshul Nasery, Vibhav Vineet, Harkirat Behl
2024 Computer Vision and Pattern Recognition | June 2024
Invited Talk at Holistic Video Understanding Workshop, CVPR 2024
-
PEEKABOO: Interactive Video Generation via Masked-Diffusion
Yash Jain, Anshul Nasery, Vibhav Vineet, Harkirat Behl
CVPR 2024 | December 2023
Invited Talk at Holistic Video Understanding Workshop, CVPR 2024
DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
Yash Jain, Harkirat Behl, Zsolt Kira, Vibhav Vineet
2023 Neural Information Processing Systems | December 2023
DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
Yash Jain, Harkirat Behl, Zsolt Kira, Vibhav Vineet
NeurIPS 2023 | October 2023
PLEX: Making the Most of the Available Data for Robotic Manipulation Pretraining
Garrett Thomas, Ching-An Cheng, Ricky Loynd, Felipe Vieira Frujeri, Vibhav Vineet, Mihai Jalobeanu, Andrey Kolobov
CoRL 2023 | June 2023
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang
MSR-TR-2023-44 | May 2023
Published by Microsoft
A Large-scale Robustness Analysis of Video Action Recognition Models
Madeline Chantry Schiappa, Naman Biyani, Prudvi Kamtam, Shruti Vyas, Hamid Palangi, Vibhav Vineet, Yogesh Rawat
April 2023
-
Robustness Analysis of Video-Language Models Against Visual and Language Perturbations
Madeline C. Schiappa, Shruti Vyas, Hamid Palangi, Yogesh S. Rawat, Vibhav Vineet
2022 Neural Information Processing Systems | October 2022
Image Retrieval from Contextual Descriptions
Benno Krojer, Vaibhav Adlakha, Vibhav Vineet, Yash Goyal, Edoardo Ponti, Siva Reddy
ACL 2022 | May 2022
Missingness Bias in Model Debugging
Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vemprala, Aleksander Madry
ICLR 2022 | April 2022
-
CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
Daniel McDuff, Yale Song, Jiyoung Lee, Vibhav Vineet, Sai Vemprala, Nicholas Gyde, Hadi Salman, Shuang Ma, Kwanghoon Sohn, Ashish Kapoor
June 2021
Preprint under review
-
Depth Completion Using a View-constrained Deep Prior
Pallabi Ghosh, Vibhav Vineet, Larry S. Davis, Abhinav Shrivastava, Sudipta Sinha, Neel Joshi
2020 International Conference on 3D Vision | October 2020
RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs
Zhiwei Xu, Thalaiyasingam Ajanthan, Vibhav Vineet, Richard I. Hartley
3DV | October 2020
FAST Depth Completion using a view Constrained Deep Prior
3DV | October 2020
AutoSimulate: (Quickly) Learning Synthetic Data Generation
Harkirat Singh Behl, Atılım Güneş Baydin, Ran Gal, Philip H.S. Torr, Vibhav Vineet
16th European Conference Computer Vision (ECCV 2020) | August 2020
Fast acoustic scattering using convolutional neural networks
Ziqi Fan, Vibhav Vineet, Hannes Gamper, Nikunj Raghuvanshi
ICASSP 2020 - IEEE International Conference on Acoustics, Speech and Signal Processing | May 2020
Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations
Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian Scherer, Ashish Kapoor
In submission | March 2020
-
Photorealistic Image Synthesis For Object Instance Detection
Tomas Hodan, Vibhav Vineet, Ran Gal, Emanuel Shalev, Jon Hanzelka, Treb Connell, Pedro Urbina, Sudipta Sinha, Brian Guenter
IEEE International Conference on Image Processing | September 2019
Privacy Preserving Action Recognition using Coded Aperture Videos
Zihao W. Wang, Vibhav Vineet, Francesco Pittaluga, Sudipta Sinha, Oliver Cossairt, Sing Bing Kang
The Bright and Dark Sides of Computer Vision: Challenges and Opportunities for Privacy and Security (CV-COPS 2019) | June 2019
-
Feature Space Optimization for Semantic Video Segmentation
Abhijit Kundu, Vibhav Vineet, Vladlen Koltun
July 2016
Playing for Data: Ground Truth from Computer Games
Stephan R. Richter, Vibhav Vineet, Stefan Roth, Vladlen Koltun
July 2016
Dense Monocular Depth Estimation in Complex Dynamic Scenes
Vladlen Koltun, Qifeng Chen, Vibhav Vineet, Rene Ranftl
April 2016
-
Incremental Dense Multi-modal 3D Scene Reconstruction
Philip H. S. Torr, Vibhav Vineet, Yousef Amar, Ondrej Miksik
July 2015
Publications by Research Area
-
Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
Vibhav Vineet, Xin Wang, Neel Joshi
October 2024
EUREKA: Evaluating and Understanding Large Foundation Models
Vidhisha Balachandran, Jingya Chen, Neel Joshi, Besmira Nushi, Hamid Palangi, Eduardo Salinas, Vibhav Vineet, James Woffinden-Luey, Safoora Yousefi
MSR-TR-2024-33 | September 2024
Published by Microsoft
Exposing the Achilles’ Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning
Joykirat Singh, Akshay Nambi, Vibhav Vineet
June 2024
DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
Yash Jain, Harkirat Behl, Zsolt Kira, Vibhav Vineet
2023 Neural Information Processing Systems | December 2023
DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
Yash Jain, Harkirat Behl, Zsolt Kira, Vibhav Vineet
NeurIPS 2023 | October 2023
PLEX: Making the Most of the Available Data for Robotic Manipulation Pretraining
Garrett Thomas, Ching-An Cheng, Ricky Loynd, Felipe Vieira Frujeri, Vibhav Vineet, Mihai Jalobeanu, Andrey Kolobov
CoRL 2023 | June 2023
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang
MSR-TR-2023-44 | May 2023
Published by Microsoft
Robustness Analysis of Video-Language Models Against Visual and Language Perturbations
Madeline C. Schiappa, Shruti Vyas, Hamid Palangi, Yogesh S. Rawat, Vibhav Vineet
2022 Neural Information Processing Systems | October 2022
Missingness Bias in Model Debugging
Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vemprala, Aleksander Madry
ICLR 2022 | April 2022
CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
Daniel McDuff, Yale Song, Jiyoung Lee, Vibhav Vineet, Sai Vemprala, Nicholas Gyde, Hadi Salman, Shuang Ma, Kwanghoon Sohn, Ashish Kapoor
June 2021
Preprint under review
Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations
Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian Scherer, Ashish Kapoor
In submission | March 2020
-
EUREKA: Evaluating and Understanding Large Foundation Models
Vidhisha Balachandran, Jingya Chen, Neel Joshi, Besmira Nushi, Hamid Palangi, Eduardo Salinas, Vibhav Vineet, James Woffinden-Luey, Safoora Yousefi
MSR-TR-2024-33 | September 2024
Published by Microsoft
Peekaboo: Interactive video generation via masked-diffusion
Yash Jain, Anshul Nasery, Vibhav Vineet, Harkirat Behl
2024 Computer Vision and Pattern Recognition | June 2024
Invited Talk at Holistic Video Understanding Workshop, CVPR 2024
PEEKABOO: Interactive Video Generation via Masked-Diffusion
Yash Jain, Anshul Nasery, Vibhav Vineet, Harkirat Behl
CVPR 2024 | December 2023
Invited Talk at Holistic Video Understanding Workshop, CVPR 2024
DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
Yash Jain, Harkirat Behl, Zsolt Kira, Vibhav Vineet
2023 Neural Information Processing Systems | December 2023
A Large-scale Robustness Analysis of Video Action Recognition Models
Madeline Chantry Schiappa, Naman Biyani, Prudvi Kamtam, Shruti Vyas, Hamid Palangi, Vibhav Vineet, Yogesh Rawat
April 2023
Robustness Analysis of Video-Language Models Against Visual and Language Perturbations
Madeline C. Schiappa, Shruti Vyas, Hamid Palangi, Yogesh S. Rawat, Vibhav Vineet
2022 Neural Information Processing Systems | October 2022
CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
Daniel McDuff, Yale Song, Jiyoung Lee, Vibhav Vineet, Sai Vemprala, Nicholas Gyde, Hadi Salman, Shuang Ma, Kwanghoon Sohn, Ashish Kapoor
June 2021
Preprint under review
Depth Completion Using a View-constrained Deep Prior
Pallabi Ghosh, Vibhav Vineet, Larry S. Davis, Abhinav Shrivastava, Sudipta Sinha, Neel Joshi
2020 International Conference on 3D Vision | October 2020
RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs
Zhiwei Xu, Thalaiyasingam Ajanthan, Vibhav Vineet, Richard I. Hartley
3DV | October 2020
FAST Depth Completion using a view Constrained Deep Prior
3DV | October 2020
AutoSimulate: (Quickly) Learning Synthetic Data Generation
Harkirat Singh Behl, Atılım Güneş Baydin, Ran Gal, Philip H.S. Torr, Vibhav Vineet
16th European Conference Computer Vision (ECCV 2020) | August 2020
Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations
Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian Scherer, Ashish Kapoor
In submission | March 2020
Photorealistic Image Synthesis For Object Instance Detection
Tomas Hodan, Vibhav Vineet, Ran Gal, Emanuel Shalev, Jon Hanzelka, Treb Connell, Pedro Urbina, Sudipta Sinha, Brian Guenter
IEEE International Conference on Image Processing | September 2019
Privacy Preserving Action Recognition using Coded Aperture Videos
Zihao W. Wang, Vibhav Vineet, Francesco Pittaluga, Sudipta Sinha, Oliver Cossairt, Sing Bing Kang
The Bright and Dark Sides of Computer Vision: Challenges and Opportunities for Privacy and Security (CV-COPS 2019) | June 2019
-
EUREKA: Evaluating and Understanding Large Foundation Models
Vidhisha Balachandran, Jingya Chen, Neel Joshi, Besmira Nushi, Hamid Palangi, Eduardo Salinas, Vibhav Vineet, James Woffinden-Luey, Safoora Yousefi
MSR-TR-2024-33 | September 2024
Published by Microsoft
Image Retrieval from Contextual Descriptions
Benno Krojer, Vaibhav Adlakha, Vibhav Vineet, Yash Goyal, Edoardo Ponti, Siva Reddy
ACL 2022 | May 2022
-
Robustness Analysis of Video-Language Models Against Visual and Language Perturbations
Madeline C. Schiappa, Shruti Vyas, Hamid Palangi, Yogesh S. Rawat, Vibhav Vineet
2022 Neural Information Processing Systems | October 2022
Feature Space Optimization for Semantic Video Segmentation
Abhijit Kundu, Vibhav Vineet, Vladlen Koltun
July 2016
Dense Monocular Depth Estimation in Complex Dynamic Scenes
Vladlen Koltun, Qifeng Chen, Vibhav Vineet, Rene Ranftl
April 2016
Incremental Dense Multi-modal 3D Scene Reconstruction
Philip H. S. Torr, Vibhav Vineet, Yousef Amar, Ondrej Miksik
July 2015
-
Fast acoustic scattering using convolutional neural networks
Ziqi Fan, Vibhav Vineet, Hannes Gamper, Nikunj Raghuvanshi
ICASSP 2020 - IEEE International Conference on Acoustics, Speech and Signal Processing | May 2020
-
Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations
Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian Scherer, Ashish Kapoor
In submission | March 2020
Playing for Data: Ground Truth from Computer Games
Stephan R. Richter, Vibhav Vineet, Stefan Roth, Vladlen Koltun
July 2016
-
Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations
Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian Scherer, Ashish Kapoor
In submission | March 2020
-
Dense Monocular Depth Estimation in Complex Dynamic Scenes
Vladlen Koltun, Qifeng Chen, Vibhav Vineet, Rene Ranftl
April 2016
-
Playing for Data: Ground Truth from Computer Games
Stephan R. Richter, Vibhav Vineet, Stefan Roth, Vladlen Koltun
July 2016
Publications by Type
-
Peekaboo: Interactive video generation via masked-diffusion
Yash Jain, Anshul Nasery, Vibhav Vineet, Harkirat Behl
2024 Computer Vision and Pattern Recognition | June 2024
Invited Talk at Holistic Video Understanding Workshop, CVPR 2024
PEEKABOO: Interactive Video Generation via Masked-Diffusion
Yash Jain, Anshul Nasery, Vibhav Vineet, Harkirat Behl
CVPR 2024 | December 2023
Invited Talk at Holistic Video Understanding Workshop, CVPR 2024
DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
Yash Jain, Harkirat Behl, Zsolt Kira, Vibhav Vineet
2023 Neural Information Processing Systems | December 2023
DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets
Yash Jain, Harkirat Behl, Zsolt Kira, Vibhav Vineet
NeurIPS 2023 | October 2023
PLEX: Making the Most of the Available Data for Robotic Manipulation Pretraining
Garrett Thomas, Ching-An Cheng, Ricky Loynd, Felipe Vieira Frujeri, Vibhav Vineet, Mihai Jalobeanu, Andrey Kolobov
CoRL 2023 | June 2023
Robustness Analysis of Video-Language Models Against Visual and Language Perturbations
Madeline C. Schiappa, Shruti Vyas, Hamid Palangi, Yogesh S. Rawat, Vibhav Vineet
2022 Neural Information Processing Systems | October 2022
Image Retrieval from Contextual Descriptions
Benno Krojer, Vaibhav Adlakha, Vibhav Vineet, Yash Goyal, Edoardo Ponti, Siva Reddy
ACL 2022 | May 2022
Missingness Bias in Model Debugging
Saachi Jain, Hadi Salman, Eric Wong, Pengchuan Zhang, Vibhav Vineet, Sai Vemprala, Aleksander Madry
ICLR 2022 | April 2022
Depth Completion Using a View-constrained Deep Prior
Pallabi Ghosh, Vibhav Vineet, Larry S. Davis, Abhinav Shrivastava, Sudipta Sinha, Neel Joshi
2020 International Conference on 3D Vision | October 2020
RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs
Zhiwei Xu, Thalaiyasingam Ajanthan, Vibhav Vineet, Richard I. Hartley
3DV | October 2020
FAST Depth Completion using a view Constrained Deep Prior
3DV | October 2020
AutoSimulate: (Quickly) Learning Synthetic Data Generation
Harkirat Singh Behl, Atılım Güneş Baydin, Ran Gal, Philip H.S. Torr, Vibhav Vineet
16th European Conference Computer Vision (ECCV 2020) | August 2020
Fast acoustic scattering using convolutional neural networks
Ziqi Fan, Vibhav Vineet, Hannes Gamper, Nikunj Raghuvanshi
ICASSP 2020 - IEEE International Conference on Acoustics, Speech and Signal Processing | May 2020
Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations
Rogerio Bonatti, Ratnesh Madaan, Vibhav Vineet, Sebastian Scherer, Ashish Kapoor
In submission | March 2020
Photorealistic Image Synthesis For Object Instance Detection
Tomas Hodan, Vibhav Vineet, Ran Gal, Emanuel Shalev, Jon Hanzelka, Treb Connell, Pedro Urbina, Sudipta Sinha, Brian Guenter
IEEE International Conference on Image Processing | September 2019
Privacy Preserving Action Recognition using Coded Aperture Videos
Zihao W. Wang, Vibhav Vineet, Francesco Pittaluga, Sudipta Sinha, Oliver Cossairt, Sing Bing Kang
The Bright and Dark Sides of Computer Vision: Challenges and Opportunities for Privacy and Security (CV-COPS 2019) | June 2019
Feature Space Optimization for Semantic Video Segmentation
Abhijit Kundu, Vibhav Vineet, Vladlen Koltun
July 2016
Playing for Data: Ground Truth from Computer Games
Stephan R. Richter, Vibhav Vineet, Stefan Roth, Vladlen Koltun
July 2016
Dense Monocular Depth Estimation in Complex Dynamic Scenes
Vladlen Koltun, Qifeng Chen, Vibhav Vineet, Rene Ranftl
April 2016
Incremental Dense Multi-modal 3D Scene Reconstruction
Philip H. S. Torr, Vibhav Vineet, Yousef Amar, Ondrej Miksik
July 2015
-
Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
Vibhav Vineet, Xin Wang, Neel Joshi
October 2024
Exposing the Achilles’ Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning
Joykirat Singh, Akshay Nambi, Vibhav Vineet
June 2024
A Large-scale Robustness Analysis of Video Action Recognition Models
Madeline Chantry Schiappa, Naman Biyani, Prudvi Kamtam, Shruti Vyas, Hamid Palangi, Vibhav Vineet, Yogesh Rawat
April 2023
CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
Daniel McDuff, Yale Song, Jiyoung Lee, Vibhav Vineet, Sai Vemprala, Nicholas Gyde, Hadi Salman, Shuang Ma, Kwanghoon Sohn, Ashish Kapoor
June 2021
Preprint under review
-
EUREKA: Evaluating and Understanding Large Foundation Models
Vidhisha Balachandran, Jingya Chen, Neel Joshi, Besmira Nushi, Hamid Palangi, Eduardo Salinas, Vibhav Vineet, James Woffinden-Luey, Safoora Yousefi
MSR-TR-2024-33 | September 2024
Published by Microsoft
Benchmarking Spatial Relationships in Text-to-Image Generation
Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, Yezhou Yang
MSR-TR-2023-44 | May 2023
Published by Microsoft