Who Does AI Consider an Expert? New Benchmark Shows Bias Across 22 LLMs

AI chatbots powered by large language models (LLMs) are widely used as recommender systems. While accelerating access to information, AI can also give factually wrong responses or propagate societal biases.

This matters in many contexts, including when AI is used to recommend expertise. Biased expert recommendations can deny important career opportunities for those who are left out. For example, in academia, explains computer scientist Lisette Espín-Noboa, who is a Research Fellow at the Complexity Science Hub: “A conference organizer may use LLMs to find potential keynote speakers. If your name never comes up, you are invisible and will miss out.”

Espín-Noboa and her colleagues wanted to better understand the quality of AI-generated expert recommendations. “We tested state-of-the-art LLMs, including open weight models like LLaMA and DeepSeek as well as proprietary models like Gemini and GPT,” explains Espín-Noboa.

First, the researchers looked for bias in how AI tools recommend experts in physics. “When we initially asked six LLMs to find scholars in specific physics subfields, they mostly cited real scientists but mismatched their field 40% of the time, on average. Moreover, the models tended to favor male senior experts from the US and those who are highly cited”, Espín-Noboa summarizes the main findings.

Next, they introduced LLMScholarBench, a benchmark with nine metrics to assess technical quality and social representation, and tested four user interventions across 22 LLMs. The interventions improved either factual accuracy or social representation, but not both at the same time.

Finally, they examined whether roles like “you are a student looking for a supervisor” or “a recruiter looking for a professor”, language, and geographic location affect recommendations across six academic disciplines. Geographic location influenced who was recommended, while language and role did not.

Together with CSH data visualization expert Yi Zhe Ang, the team published the interactive visualization “Whose name comes up” where users can explore how often each physicist was recommended across runs and view individual chatbot performance per evaluation metric.

In the first study, the team assessed the performance of six open-weight LLMs (models whose parameters are publicly available) in terms of consistency, accuracy, and social bias. “We set the models to five different recommendation tasks, such as to compile a list of the top five and top 100 most influential experts in physics,” explains Espín-Noboa.

A database of more than 450,000 scientists who published in journals of the American Physical Society (APS) between 1893 and 2020 was used as the ground truth.

Women have historically been underrepresented in physics, but still account for 14–32% of researchers in the scientific record, depending on the subfield and era.  “Yet most LLMs recommended even fewer women than the already low baseline, and in some cases none at all,” says Espín-Noboa. The pattern extends to ethnicity: Asian scholars are the largest demographic group in the APS database, but in most cases LLMs overrepresented White scholars, while Black and Latino researchers were often absent entirely.

Meanwhile, recommended scholars were overwhelmingly among the most highly published and cited, far above the typical physicist in the field.

Smaller models also tended to recommend scholars clustered within the same countries, narrowing geographic diversity further. Espín-Noboa summarizes: “Taken together, these patterns show that LLMs do not just mirror existing inequalities – they can amplify them, reinforcing a rich-get-richer dynamic where the already visible become more visible, and the already marginalized remain unseen.”

In terms of factuality, the models named scientists from the database in roughly 80% of recommendations across tasks. “This means models occasionally struggle to verify their responses with the ground truth”, explains Espín-Noboa. “For more complex tasks, such as correctly assessing a scholar’s level of seniority or their subfield, accuracy was even lower.”

After this foundational work, the team developed LLMScholarBench, a benchmark that captures LLM performance across various aspects of technical quality and social representation. This second study was recently presented by Espín-Noboa at the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2026).

The researchers defined five metrics in the technical dimension, such as factual accuracy and consistency. In the social dimension, they defined four metrics, such as similarity (how similar recommended scholars are in terms of career development), diversity (how evenly recommendations are distributed across categories of a given attribute), and parity (whether the distribution of recommended scholars across a given category reflects their share in their field).

Using this benchmark, they evaluated how different user-controlled interventions affect the quality of responses generated by 22 LLMs (20 open-weight, 2 proprietary).

The interventions tested included retrieval-augmented generation (RAG) – essentially telling the model to gather information from the web instead of merely sourcing responses from the training data – as well as prompt engineering to explicitly steer the chatbots toward generating more socially balanced responses.

“It turns out that RAG improves technical quality, in particular factual accuracy, and prompt engineering steers social representation as requested. But even when combining both, the trade-off persists – improving all metrics at once remains challenging,” highlights Espín-Noboa.

In the third study, the researchers examined whether the role, language or geographic location of a person specified in a prompt influence which scholars an LLM recommends, across six academic disciplines, including physics. 

They found that geographic location affected the recommendations, while language did not. Prompts in different languages produced similar results. “This matters because geographic framing can influence the accuracy and quality of recommendations, even though it should not be a factor in assessing scientific expertise,” says Espín-Noboa.

“While our studies include some LLMs that have since been updated, the core contribution is not a snapshot of any single model – it is LLMScholarBench, an open evaluation benchmark that equips practitioners and researchers with a systematic tool to audit any LLM when used as a people recommender system,” Espín-Noboa emphasizes. “Our results consistently show that the biases we identify are structural, not model-specific.”

When the team tested newer proprietary models like Gemini 2.5 Pro and Flash with retrieval-augmented generation (web search), factual accuracy improved – but at the cost of reduced diversity and parity. “In other words, grounding LLMs in the web does not fix representation gaps; it imports them, because the web itself underrepresents entire communities of scholars,” says the researcher.

According to the researchers, training on more or newer data will not resolve this unless the underlying data becomes more representative – which is why they aim to build a comprehensive scholarly knowledge base that captures the full breadth of contributors to science.

Espín-Noboa continues: “And the stakes extend well beyond academia: people already use LLMs to find doctors, lawyers, and professionals across every field. If these systems have a skewed representation, real people are the ones who miss out. We hope that our findings will inform the development of better interventions to make LLMs more trustworthy.”

The visualization brings together findings from three related papers:

Whose Name Comes Up? I: Auditing LLM-Based Scholar Recommendations (preprint)

Whose Name Comes Up? II: Benchmarking and Intervention-Based Auditing of LLM-Based Scholar Recommendation (KDD 2026; presented at the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining in Jeju, South Korea; extended version available on arXiv) – doi: 10.1145/3770855.3817543

Whose Name Comes Up? III: Persona Prompting Effects in LLM-Based Scholar Recommendation (preprint)

Data collection: The analyses were conducted using data collected between December 2024 and January 2026, with the individual studies covering different periods: December 9, 2024–January 8, 2025 (Paper I); November–December 2025 (Paper III); and December 19, 2025–January 18, 2026 (Paper II). The data and the code are openly available.

The Complexity Science Hub (CSH) is Europe's research center for the study of complex systems. Drawing on large-scale data across economics, medicine, ecology, and the social sciences, CSH develops quantitative methods to understand the interconnected networks that underlie society – from financial markets and supply chains to public health and urban development. The goal is to provide a rigorous basis for navigating the challenges of an increasingly complex world. 

Members of the Complexity Science Hub are: AIT Austrian Institute of Technology, BOKU University, Central European University (CEU), IT:U Interdisciplinary Transformation University Austria, Medical University Vienna, TU Wien, TU Graz, University for Continuing Education Krems, Vetmeduni, WU Vienna and WKO.

csh.ac.at