Mark Ibrahim

We open-source OpenApps, a Python research environment that generates endless versions of six real apps — with ground-truth state and rewards, on a single CPU — to train and evaluate computer-use agents across the variations they break on in deployment.

📣 code release, 📃 paper, and 🎬 video tutorial

Karen Ullrich, Jingtong Su, Claudia Shi, Arjun Subramonian, Amir Bar, Ivan Evtimov, Nikolaos Tsilivis, Randall Balestriero, Julia Kempe, Mark Ibrahim

Multimodal models can perceive objects but hallucinate when reasoning across scenes. Common-O is a decontaminated multi-scene reasoning benchmark on which today's best models score under 25%.

> paper + data

Candace Ross, Florian Bordes, Adina Williams, Polina Kirichenko, Mark Ibrahim

A state-of-the-art open-weight vision encoder (ViT-G) with optimized visual cross-attention. Scaled to 5B samples, LLIP outperforms MetaCLIP by an average of 2.9% across 22 zero-shot benchmarks and 6% R@1 on COCO retrieval.

> paper + weights

Samuel Lavoie, Polina Kirichenko, Mark Ibrahim, Mahmoud Assran, Andrew Gordon Wilson, Aaron Courville, Nicolas Ballas

A 50+ benchmark suite of vision-language capabilities revealing that scaling alone doesn't improve visual reasoning — evaluate a model across 7 capability types on 1 GPU in minutes.

> paper

Haider Al-Tahan, Quentin Garrido, Randall Balestriero, Diane Bouchacourt, Caner Hazirbas, Mark Ibrahim

A graph-based contrastive loss that explicitly encodes relationships across samples during training, improving efficiency and robustness.

> paper

Vlad Sobal, Mark Ibrahim, Randall Balestriero, Vivien Cabannes, Diane Bouchacourt, Pietro Astolfi, Kyunghyun Cho, Yann LeCun

We show GRPO needs to update as few as 13 parameters (26 bytes) to bring Qwen2.5-8B within 5% of full-finetuning GSM8K performance — recovering 90% of reasoning gains while training 1000× fewer parameters.

> paper

John X. Morris, Niloofar Mireshghallah, Mark Ibrahim, Saeed Mahloujifar

We evaluate LLMs' capacity for abstention — the skill of knowing when NOT to answer. We find reasoning LLMs struggle with unanswerable questions and hallucinate.

> paper + code + data

Polina Kirichenko*, Samuel J. Bell*, Kamalika Chaudhuri, Mark Ibrahim*

We show that training transformers to predict multiple tokens ahead and back (instead of just the single next token) improves models' ability to retrieve knowledge.

> paper

Ouail Kitouni, Niklas Nolte, Diane Bouchacourt, Adina Williams, Mike Rabbat, Mark Ibrahim

In a follow-up, the same objective improves a transformer's ability to plan in maze navigation (MLM-U), converging 2× faster in GPU hours.

A method to automatically discover the spurious environments that break out-of-distribution generalization — without human annotations.

> paper

Mohammad Pezeshki, Diane Bouchacourt, Mark Ibrahim, Nicolas Ballas, Pascal Vincent, David Lopez-Paz

We find surprisingly similar strengths and vulnerabilities across more than 2,200 deep learning models.

> paper + website

Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestriero, Ivan Evtimov, Caner Hazirbas, Nicolas Ballas, Pascal Vincent, Michal Drozdzal, David Lopez-Paz, Mark Ibrahim

A method for and study of how deep learning techniques cope with multiple shortcuts (Whac-A-Mole).

> paper + code

Zhiheng Li, Ivan Evtimov*, Albert Gordo, Caner Hazirbas, Tal Hassner, Cristian Canton Ferrer, Chenliang Xu, Mark Ibrahim*

Progress on standard benchmarks fails to improve — and can worsen — geographic disparities in today's best models.

> paper

Megan Richards, Polina Kirichenko, Diane Bouchacourt, Mark Ibrahim

Recipes for training and evaluating self-supervised learning systems — co-authored with Randall Balestriero, Yann LeCun, and many others.

> paper + open source library + blog post

Mark Ibrahim, Melissa Louie, Ceena Modarres, John Paisley (Columbia University)