Research Themes

Check out the full list of my papers for an up-to-date overview of my research. The following are the core themes of my work and some associated selected publications.

• Open Language Modeling Recipes
  • Tulu 3: Pushing Frotiers in Open Language Model Post-Training
  • How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources
• Efficient Adaptation of Language Models
  • Large-Scale Data Selection for Instruction Tuning
  • Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback
  • Scalable Data Ablation Approximations for Language Models through Modular Training and Merging
  • Merge to Learn: Efficiently Adding Skills to Language Models with Model Merging
  • Data-Efficient Finetuning Using Cross-Task Nearest Neighbors
• Robustness to Distribution Shifts
  • TRAM: Bridging Trust Regions and Sharpness Aware Minimization
  • AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization
  • Generating Data to Mitigate Spurious Correlations in Natural Language Inference Datasets
• Evaluation Benchmarks and Guidelines
• LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization
• A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers
• Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning
• DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs