Ashudeep Singh

Founding Member of Technical Staff, Sycamore.
Ph.D., Computer Science, Cornell University.
Résumé · CV

Areas of Interest: AI Agent Safety & Governance, AI Agent Orchestration, Responsible AI, Large Language Models, Machine Learning.

I am a Founding Member of Technical Staff at Sycamore, where we are building the trust infrastructure for AI agents in production — safety, security, governance, and orchestration to enable enterprise adoption at scale.

Previously, I was a Principal Applied Scientist at Microsoft AI, focusing on agentic AI for search systems, developing safe, human-aligned multimodal LLMs. Before that, I worked at Pinterest Labs, pioneering bias evaluation frameworks and red-teaming methodologies for LLMs and image generation models in production, and establishing fairness evaluation and mitigation frameworks across recommendation systems. I completed my PhD at Cornell University under Thorsten Joachims, establishing foundational work in fairness for ranking and recommender systems. I also completed research internships at Google Brain, Facebook Research, and Microsoft Research.

My research bridges AI safety theory with practical deployment, publishing at top venues (NeurIPS, ICML, SIGIR, KDD, FAccT) while shipping state-of-the-art responsible AI systems serving hundreds of millions of users. I also serve as Area Chair for NeurIPS and ICML, and co-taught the NeurIPS 2022 tutorial on responsible ML.

News

Mar 30, 2026 Sycamore is coming out of stealth! We are building the trust infrastructure for AI agents in production — safety, security, governance, and orchestration. We raised a $65M seed round led by Coatue and Lightspeed Venture Partners. We're hiring across engineering and research — reach out or apply at sycamore.so.

Jan 2, 2026 I left Microsoft to join Sycamore as a Founding Member of Technical Staff, working on trust infrastructure for AI agents in production.

Aug 12, 2024 I joined Microsoft AI as a Principal Applied Scientist, working on agentic AI for search systems.

May 17, 2024 Paper on measuring user impact of diversification in recommendations titled "Inclusive Recommendations and User Engagement: Experimental Evidence from Pinterest" accepted at ACM EC 2024.

Oct 17, 2023 Presented a guest lecture for the Operations Management class (BUAD 311) at USC Marshall School of Business on "Responsible ML for Real-World Search and Recommender Systems: A Multistakeholder Perspective" (slides).

Sep 9, 2023 Ongoing research with Madhav Kumar (MIT) on the impact of diversification in recommender systems on its users was recently accepted to Workshop on Information Systems and Economics (WISE) 2023.

Jun 15, 2023 Our work on interpretability for recommender systems called "RecRec: Algorithmic Recourse for Recommender Systems" was accepted to ACM CIKM 2023. This is joint work with co-authors from University of Washington and University of Maryland. Here is the arXiv link for the paper.

Jun 11, 2023 Our recent work on diversification in search and recommender systems across the Pinterest platform was accepted to ACM FAccT 2023. See you in Chicago! It was recently also published on the Pinterest Engineering blog.

Publications [google scholar]

Preprint

VisTIRA: Closing the Image-Text Modality Gap in Visual Math Reasoning via Structured Tool Integration

Saeed Khaki, Ashudeep Singh , Nima Safaei, Kamal Ginotra

arXiv preprint arXiv:2601.14440 , 2026

arXiv

Vision-language models underperform on mathematical reasoning tasks when problems appear as images rather than text. We identify this modality gap as stemming from difficulties reading dense formulas, layout, and mixed symbolic-diagrammatic content. We present VisTIRA, which decomposes math problems iteratively into natural language explanations and executable Python code. We create a LaTeX-based pipeline converting math corpora into image variants and synthetic tool-use trajectories. Our findings indicate that tool-integrated supervision and OCR grounding improve performance, though benefits diminish with larger models.

FnTIR® Journal

Fairness in Search Systems

Yi Fang , Ashudeep Singh , Zhiqiang Tao

Foundations and Trends® in Information Retrieval , vol. 18 , pp. 262-416 , 2024

Search engines play a crucial role in organizing and delivering information to billions of users worldwide. However, these systems often reflect and amplify existing societal biases and stereotypes through their search results and rankings. This concern has prompted researchers to investigate methods for measuring and reducing algorithmic bias, with the goal of developing more equitable search systems. This monograph presents a comprehensive taxonomy of fairness in search systems and surveys the current research landscape. We systematically examine how bias manifests across key search components, including query interpretation and processing, document representation and indexing, result ranking algorithms, and system evaluation metrics. By critically analyzing the existing literature, we identify persistent challenges and promising research directions in the pursuit of fairer search systems. Our aim is to provide a foundation for future work in this rapidly evolving field while highlighting opportunities to create more inclusive and equitable information retrieval technologies.

ACM EC

Inclusive Recommendations and User Engagement: Experimental Evidence from Pinterest

Madhav Kumar , Pedro Silva, Ashudeep Singh , Abhay Varmaraja

ACM International Conference on Economics and Computation (EC'24) , 2024

Also appeared at Workshop on Information Systems and Economics (WISE) 2023 and Conference on Information Systems and Technology (CIST) 2024. Won the Best Paper Award at CIST 2024.

PDF

We study the impact of diversifying recommendations for inclusivity on one of the largest visual content discovery platforms in the world, Pinterest. Pinterest re-designed its recommendation systems to improve the representation of all skin tones in recommended content and foster a more inclusive user experience. We describe the design of the new recommendation system and present results from a field experiment in which users across six countries were randomly assigned to receive a more diverse set of recommendations based on content skin tone. We find that the overall engagement rates remain stable and engagement with previously underrepresented content increases significantly. More broadly, users diversify their consumption by engaging with content from all skin tone ranges. We shed light on the mechanism driving these results using heterogeneous treatment effect analysis. We find that engagement for users with preference for deeper skin tone content increases significantly and engagement for users with preference for lighter skin tone content remains relatively stable. Finally, we analyze post-launch data to better understand the long-term implications of diversifying recommendations. Our research provides practical insights for platform managers and policymakers to create inclusive digital environments that promote engagement while catering to diverse user preferences.

FAccT

Representation Online Matters: Practical End-to-End Diversification in Search and Recommender Systems

Pedro Silva, Bhawna Juneja, Shloka Desai, Ashudeep Singh , Nadia Fawaz

Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pp. 1735–1746 , 2023

arXiv

As the use of online platforms continues to grow across all demographics, users often express a desire to feel represented in the content. To improve representation in search results and recommendations, we introduce end-to-end diversification, ensuring that diverse content flows throughout the various stages of these systems, from retrieval to ranking. We develop, experiment, and deploy scalable diversification mechanisms in multiple production surfaces on the Pinterest platform, including Search, Related Products, and New User Homefeed, to improve the representation of different skin tones in beauty and fashion content. Diversification in production systems includes three components: identifying requests that will trigger diversification, ensuring diverse content is retrieved from the large content corpus during the retrieval stage, and finally, balancing the diversity-utility trade-off in a self-adjusting manner in the ranking stage. Our approaches, which evolved from using Strong-OR logical operator to bucketized retrieval at the retrieval stage and from greedy re-rankers to multi-objective optimization using determinantal point processes for the ranking stage, balances diversity and utility while enabling fast iterations and scalable expansion to diversification over multiple dimensions. Our experiments indicate that these approaches significantly improve diversity metrics, with a neutral to a positive impact on utility metrics and improved user satisfaction, both qualitatively and quantitatively, in production.

CIKM

RecRec: Algorithmic Recourse for Recommender Systems

Sahil Verma , Ashudeep Singh , Varich Boonsanong, John P. Dickerson , Chirag Shah

Proceedings of the 32nd ACM International Conference on Information and Knowledge Management , 2023

arXiv

Recommender systems play an essential role in the choices people make in domains such as entertainment, shopping, food, news, employment, and education. The machine learning models underlying these recommender systems are often enormously large and black-box in nature for users, content providers, and system developers alike. It is often crucial for all the stakeholders to understand the model's rationale behind making certain predictions and recommendations. This is especially true for the content providers whose livelihoods depend on the recommender system. Drawing motivation from the practitioners' need, in this work, we propose a recourse framework for recommender systems, targeted towards the content providers. Algorithmic recourse in the recommendation setting is a set of actions, which, if executed, would modify the recommendations (or ranking) of an item in the desired manner. A recourse suggests actions of the form: "if a feature changes X to Y, then the ranking of that item for a set of users will change to Z." Furthermore, we demonstrate that RecRec is highly effective in generating valid, sparse, and actionable recourses through an empirical evaluation of recommender systems trained on three real world datasets. To the best of our knowledge, this work is the first to conceptualize and empirically test a generalized framework for generating recourses for recommender systems.

NeurIPS

Fairness in Ranking under Uncertainty

Ashudeep Singh , David Kempe , Thorsten Joachims

Neural Information Processing Systems (NeurIPS) , 2021

arXiv Code Website

Fairness has emerged as an important consideration in algorithmic decision-making. Unfairness occurs when an agent with higher merit obtains a worse outcome than an agent with lower merit. Our central point is that a primary cause of unfairness is uncertainty. A principal or algorithm making decisions never has access to the agents' true merit, and instead uses proxy features that only imperfectly predict merit (e.g., GPA, star ratings, recommendation letters). None of these ever fully capture an agent's merit; yet existing approaches have mostly been defining fairness notions directly based on observed features and outcomes. Our primary point is that it is more principled to acknowledge and model the uncertainty explicitly. The role of observed features is to give rise to a posterior distribution of the agents' merits. We use this viewpoint to define a notion of approximate fairness in ranking. We call an algorithm ϕ-fair (for ϕ∈[0,1]) if it has the following property for all agents x and all k: if agent x is among the top k agents with respect to merit with probability at least ρ (according to the posterior merit distribution), then the algorithm places the agent among the top k agents in its ranking with probability at least ϕρ. We show how to compute rankings that optimally trade off approximate fairness against utility to the principal. In addition to the theoretical characterization, we present an empirical analysis of the potential impact of the approach in simulation studies. For real-world validation, we applied the approach in the context of a paper recommendation system that we built and fielded at the KDD 2020 conference.

SIGIR Best Paper

Controlling Fairness and Bias in Dynamic Learning-to-Rank

Marco Morik, Ashudeep Singh , Jessica Hong, Thorsten Joachims

Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval , 2020

arXiv Code Website Slides

Rankings are the primary interface through which many online platforms match users to items (e.g. news, products, music, video). In these two-sided markets, not only the users draw utility from the rankings, but the rankings also determine the utility (e.g. exposure, revenue) for the item providers (e.g. publishers, sellers, artists, studios). It has already been noted that myopically optimizing utility to the users – as done by virtually all learning-to-rank algorithms – can be unfair to the item providers. We, therefore, present a learning-to-rank approach for explicitly enforcing merit-based fairness guarantees to groups of items (e.g. articles by the same publisher, tracks by the same artist). In particular, we propose a learning algorithm that ensures notions of amortized group fairness, while simultaneously learning the ranking function from implicit feedback data. The algorithm takes the form of a controller that integrates unbiased estimators for both fairness and utility, dynamically adapting both as more data becomes available. In addition to its rigorous theoretical foundation and convergence guarantees, we find empirically that the algorithm is highly practical and robust.

FAccTRec

Building Healthy Recommendation Sequences for Everyone: A Safe Reinforcement Learning Approach

Ashudeep Singh , Yoni Halpern, Nithum Thain, Konstantina Christakopoulou, Ed H. Chi , Jilin Chen, Alex Beutel

FAccTRec Workshop at ACM RecSyS , 2020

PDF Code Website Slides

A key consideration in the design of recommender systems is the long term well-being of users. In this work, we formalize this challenge as a multi-objective, safe reinforcement learning problem, balancing positive user feedback and the “healthiness” of user trajectories. We note that in some cases, naively balancing these objectives can lead to unhealthy experiences, even if unlikely, still occurring in a small subset of users leading us to examine a distributional notion of recommendation safety. Thus, we propose a reinforcement learning approach that optimizes for positive feedback from users while simultaneously optimizing for the health of worst-case users to remain high. To empirically validate our method, we develop a novel research simulation environment motivated by a movie recommendation setting that considers exposure to violence as a proxy for unhealthy recommendations. We demonstrate how our method reduces unhealthy recommendations to the most vulnerable users, without sacrificing much user satisfaction.

NeurIPS

Policy Learning for Fairness in Ranking

Ashudeep Singh , Thorsten Joachims

Neural Information Processing Systems (NeurIPS) , pp. 5426--5436 , 2019

arXiv Code Website

Conventional Learning-to-Rank (LTR) methods optimize the utility of the rankings to the users, but they are oblivious to their impact on the ranked items. However, there has been a growing understanding that the latter is important to consider for a wide range of ranking applications (e.g. online marketplaces, job placement, admissions). To address this need, we propose a general LTR framework that can optimize a wide range of utility metrics (e.g. NDCG) while satisfying fairness of exposure constraints with respect to the items. This framework expands the class of learnable ranking functions to stochastic ranking policies, which provides a language for rigorously expressing fairness specifications. Furthermore, we provide a new LTR algorithm called Fair-PG-Rank for directly searching the space of fair ranking policies via a policy-gradient approach. Beyond the theoretical evidence in deriving the framework and the algorithm, we provide empirical results on simulated and real-world datasets verifying the effectiveness of the approach in individual and group-fairness settings.

KDD

Fairness of Exposure in Rankings

Ashudeep Singh , Thorsten Joachims

In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, London, UK , pp. 2219--2228 , 2018

arXiv

Rankings are ubiquitous in the online world today. As we have transitioned from finding books in libraries to ranking products, jobs, job applicants, opinions and potential romantic partners, there is a substantial precedent that ranking systems have a responsibility not only to their users but also to the items being ranked. To address these often conflicting responsibilities, we propose a conceptual and computational framework that allows the formulation of fairness constraints on rankings in terms of exposure allocation. As part of this framework, we develop efficient algorithms for finding rankings that maximize the utility for the user while provably satisfying a specifiable notion of fairness. Since fairness goals can be application specific, we show how a broad range of fairness constraints can be implemented using our framework, including forms of demographic parity, disparate treatment, and disparate impact constraints. We illustrate the effect of these constraints by providing empirical results on two ranking problems.

NeurIPS Workshop

Equality of Opportunity in Rankings

Ashudeep Singh , Thorsten Joachims

Workshop On Prioritizing Online Content (WPOC) at NeurIPS , 2017

PDF

With the ubiquity of algorithmic methods for ranking in systems like search engines, recommendations, and advertisements, it becomes essential to ensure that these systems do not impact different groups at different rates. In this work, we focus on defining the notions of fairness in such ranking scenarios in terms of the amount of exposure (opportunity) each item gets in its position while keeping in mind the protected groups. We formulate these definitions for protected groups of documents, queries as well as users.

NeurIPS Workshop

Learning item embeddings using biased feedback

Ashudeep Singh , Thorsten Joachims

NeurIPS Workshop on Causal Inference and Machine Learning for Intelligent Decision Making , 2017

PDF

Learning item embeddings from browsing logs of recommender systems provides intriguing opportunities for understanding user preferences. However, such log data can be severely biased because recommendations imply a selection bias on the number of clicks an item receives. This selection bias can lead to learned embeddings that are distorted by past recommendations and that do not reflect the true semantic similarity one would like to capture. To overcome this problem, we formulate the task of learning embeddings as a counterfactual learning problem: how would the user have clicked, if the recommendation algorithm had not interfered? To demonstrate effectiveness and promise of this approach, we present synthetic experiments that illustrate how the counterfactual learning approach can recover the true embeddings despite biased data.

ICML

Recommendations as treatments: Debiasing learning and evaluation

Tobias Schnabel , Adith Swaminathan , Ashudeep Singh , Navin Chandak, Thorsten Joachims

International Conference on Machine Learning (ICML) , vol. 48 , pp. 1670--1679 , 2016

arXiv Website

Most data for evaluating and training recommender systems is subject to selection biases, either through self-selection by the users or through the actions of the recommendation system itself. In this paper, we provide a principled approach to handle selection biases by adapting models and estimation techniques from causal inference. The approach leads to unbiased performance estimators despite biased data, and to a matrix factorization method that provides substantially improved prediction performance on real-world data. We theoretically and empirically characterize the robustness of the approach, and find that it is highly practical and scalable.

ITS

Predicting Student Learning from Conversational Cues

David Adamson, Akash Bharadwaj, Ashudeep Singh , Colin Ashe, David Yaron, Carolyn P Rosé

International Conference on Intelligent Tutoring Systems (ITS) , pp. 220--229 , 2014

PDF

In the work here presented, we apply textual and sequential methods to assess the outcomes of an unconstrained multiparty dialogue. In the context of chat transcripts from a collaborative learning scenario, we demonstrate that while low-level textual features can indeed predict student success, models derived from sequential discourse act labels are also predictive, both on their own and as a supplement to textual feature sets. Further, we find that evidence from the initial stages of a collaborative activity is just as effective as using the whole.

Preprint

A Semantic Approach to Summarization

Divyanshu Bhartiya, Ashudeep Singh

arXiv preprint arXiv:1406.1203 , 2014

PDF arXiv

Sentence extraction based summarization has some limitations as it doesn't go into the semantics of the document. Also, it lacks the capability of sentence generation which is intuitive to humans. Here we present a novel method to summarize text documents taking the process to semantic levels with the use of WordNet and other resources, and using a technique for sentence generation. We involve semantic role labeling to get the semantic representation of text and use of segmentation to form clusters of the related pieces of text. Picking out the centroids and sentence generation completes the task. We evaluate our system against human composed summaries and also present an evaluation done by humans to measure the quality attributes of our summaries.

AIED

Automatically Generating Discussion Questions

David Adamson, Divyanshu Bhartiya, Biman Gujral, Radhika Kedia, Ashudeep Singh , Carolyn P. Rosé

International Conference on Artificial Intelligence in Education (AIED) , pp. 81--90 , 2013

PDF

Automatic question generation can support instruction and learning. However, work to date has produced mostly “shallow” questions that fall short of supporting deep learning and discussion. We propose an extension to a state-of-the-art question generation system that allows it to produce deep, subjective questions suitable for group discussion. We evaluate the questions generated by this system against a panel of experienced judges, and find that our approach fares significantly better than the baseline system.