PhD Candidate · Nuremberg, Germany
Gagan Bhatia
I study LLM interpretability, social biases, and chain-of-thought monitorability.
Doctoral Researcher, CSAI, University of Technology Nuremberg
Open to research internships in AI and NLP.
At EMNLP 2026
- EMNLP 2026
Who Annotates in NLP? A Large-scale Assessment of Human Annotation Reporting between 2018 and 2025
Audits how NLP papers report human annotation, revealing widespread gaps in annotator backgrounds, training, compensation, and quality checks.
Read the paper - EMNLP 2026 (Findings)
What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?
Introduces MultiTempBench to examine how date tokenization and internal representations affect temporal reasoning across languages and calendars.
Read the paper
Selected work
ChronoLens: Measuring language change
ChronoLens uses interpretable multilingual representations to trace changes in grammar, meaning, and language use across two centuries of parliamentary text.
Read the paperTemporal reasoning in LLMs
I investigate the roles of tokenisation and temporal representations in LLM reasoning, using attention-based interpretability methods to analyse model behavior.
Read the paperProduction machine learning
At Xtract One, I trained and deployed a Vision Transformer for image manipulation detection via a REST API on AWS, and developed a document intelligence pipeline processing over 10,000 documents daily.
View experience
About
I am a first-year PhD candidate at the Chair of Computer Science and Artificial Intelligence (CSAI), University of Technology Nuremberg, advised by Steffen Eger. My current research focuses on social biases in large language models, model interpretability, and chain-of-thought monitorability: understanding model behavior and what we can learn by monitoring the reasoning models produce.
Before starting my PhD, I completed an MSc in Artificial Intelligence at the University of Aberdeen (2024) and a BASc at the University of British Columbia (2022). During and after my undergraduate studies, I worked with Muhammad Abdul-Mageed’s Deep Learning & Natural Language Processing Group at UBC, contributing to Peacock, Dallah, Qalam, and FinTral — multimodal Arabic and financial large language models published at ACL and ArabicNLP.
My broader work spans multilingual generative-text evaluation, temporal reasoning and tokenisation in LLMs, and low-resource and Arabic NLP. My papers have appeared at ACL, EMNLP, NAACL, and ArabicNLP, and I collaborate with research groups across evaluation, temporal reasoning, and multilingual NLP.
- Social biases in LLMs
- Model interpretability
- Chain-of-thought monitorability
- Multilingual NLP & evaluation
Publications
- 2026
- arXivChronoLens: Measuring Language Change Across Time, Languages, and Linguistic Levels
Uses interpretable multilingual representations to compare changes in grammar, meaning, and language use across two centuries of parliamentary text.
Paper@misc{bhatia2026chronolensmeasuringlanguagechange, title={ChronoLens: Measuring Language Change Across Time, Languages, and Linguistic Levels}, author={Gagan Bhatia and Julian Schlenker and Simone Paolo Ponzetto and Steffen Eger}, year={2026}, eprint={2608.03507}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2608.03507}, } - ACL 2026 (Findings)From RAG to Agentic RAG for Faithful Islamic Question Answering
Introduces IslamicFaithQA and an agentic retrieval framework that improves evidence-grounded Islamic question answering in Arabic and English.
Paper@inproceedings{bhatia-etal-2026-rag, title = "From {RAG} to Agentic {RAG} for Faithful Islamic Question Answering", author = "Bhatia, Gagan and Mubarak, Hamdy and Jarrar, Mustafa and Mikros, George and Zaraket, Fadi and Alhirthani, Mahmoud and al-Khatib, Mutaz and Cochrane, Logan and Darwish, Kareem Mohamed and Yahiaoui, Rashid and Alam, Firoj", editor = "Liakata, Maria and Moreira, Viviane P. and Zhang, Jiajun and Jurgens, David", booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026", month = jul, year = "2026", address = "San Diego, California, United States", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2026.findings-acl.1317/", doi = "10.18653/v1/2026.findings-acl.1317", pages = "26469--26488", ISBN = "979-8-89176-395-1" } - ACL 2026 (Industry Track)Fanar-Sadiq: A Multi-Agent Architecture for Grounded Islamic QA
Combines specialized agents, verified citations, and rule-based calculators to answer Islamic questions in Arabic and English.
Paper@inproceedings{abbas-etal-2026-fanar, title = "Fanar-Sadiq: A Multi-Agent Architecture for Grounded Islamic {QA}", author = "Abbas, Ummar and Ouzzani, Mourad and Eltabakh, Mohamed Y. and Sinan, Omar and Bhatia, Gagan and Mubarak, Hamdy and Hawasly, Majd and Hashim, Mohammed Qusay and Darwish, Kareem Mohamed and Alam, Firoj", editor = "Li, Yunyao and Rehm, Georg and Tu, Mei", booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 6: Industry Track)", month = jul, year = "2026", address = "San Diego, California, USA", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2026.acl-industry.109/", doi = "10.18653/v1/2026.acl-industry.109", pages = "1567--1588", ISBN = "979-8-89176-394-4" } - EMNLP 2026Who Annotates in NLP? A Large-scale Assessment of Human Annotation Reporting between 2018 and 2025
Audits how NLP papers report human annotation, revealing widespread gaps in annotator backgrounds, training, compensation, and quality checks.
Paper@inproceedings{kunilovskaya-etal-2026-annotates, title = "Who Annotates in NLP? A Large-scale Assessment of Human Annotation Reporting between 2018 and 2025", author = "Maria Kunilovskaya and Gagan Bhatia and Lisa Sophie Albertelli and Yanran Chen and Christian Greisinger and Lotta Kiefer and Christoph Leiter and Subhadeep Roy and Tewodros Achamaleh and Muhammad Arslan Manzoor and Sebastian Pohl and Yufang Hou and Steffen Eger", booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing", year = "2026", publisher = "Association for Computational Linguistics", note = "To appear", url = "https://arxiv.org/abs/2606.02255" } - arXivBeyond LLM-as-a-judge: Deterministic Metrics for Multilingual Generative Text Evaluation
Introduces OmniScore, lightweight deterministic metrics for reproducible multilingual evaluation of generated text.
Paper@misc{alam2026llmasajudgedeterministicmetricsmultilingual, title={Beyond LLM-as-a-Judge: Deterministic Metrics for Multilingual Generative Text Evaluation}, author={Firoj Alam and Gagan Bhatia and Sahinur Rahman Laskar and Shammur Absar Chowdhury}, year={2026}, eprint={2604.05083}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2604.05083}, } - EMNLP 2026 (Findings)What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?
Introduces MultiTempBench to examine how date tokenization and internal representations affect temporal reasoning across languages and calendars.
Paper@inproceedings{bhatia-etal-2026-really, title = "What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?", author = "Gagan Bhatia and Ahmad Muhammad Isa and Maxime Peyrard and Wei Zhao", booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2026", year = "2026", publisher = "Association for Computational Linguistics", note = "To appear", url = "https://arxiv.org/abs/2603.19017" } - TechRxivAdvances in AI Systems on Islamic Knowledge Capabilities: A Critical Survey
Surveys 160 studies on AI for Islamic knowledge, organizing methods and identifying gaps in grounding, evaluation, and trustworthiness.
Paper@misc{bhatia2026advances, title={Advances in AI Systems on Islamic Knowledge Capabilities: A Critical Survey}, author={Gagan Bhatia and Hamdy Mubarak and Majd Hawasly and Mustafa Jarrar and George Mikros and Fadi Zaraket and Mahmoud Alhirthani and Mutaz al-Khatib and Logan Cochrane and Kareem Darwish and Rashid Yahiaoui and Firoj Alam}, year={2026}, publisher={TechRxiv}, note={Preprint}, url={https://huggingface.co/datasets/QCRI/islamic-knowledge-ai-survey} } - arXivPrototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics
Shows that image evaluation metrics can favor stereotypical but incorrect images, and introduces a benchmark and mitigation method.
Paper@misc{roy2026prototypicalitybiasrevealsblindspots, title={Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics}, author={Subhadeep Roy and Gagan Bhatia and Steffen Eger}, year={2026}, eprint={2601.04946}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2601.04946}, } - 2025
- EMNLP 2025Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoning
Shows how fragmented date tokens hinder temporal reasoning and traces how language models reconstruct date components internally.
Paper@inproceedings{bhatia-etal-2025-date, title = "Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoning", author = "Bhatia, Gagan and Peyrard, Maxime and Zhao, Wei", editor = "Christodoulopoulos, Christos and Chakraborty, Tanmoy and Rose, Carolyn and Peng, Violet", booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing", month = nov, year = "2025", address = "Suzhou, China", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2025.emnlp-main.159/", doi = "10.18653/v1/2025.emnlp-main.159", pages = "3201--3219", ISBN = "979-8-89176-332-6" } - arXivDistributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Models
Traces concepts through model layers to explain how semantic drift produces fluent but contextually unsupported answers.
Paper@misc{bhatia2026distributionalsemanticstracingframework, title={Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Models}, author={Gagan Bhatia and Somayajulu G Sripada and Kevin Allan and Jacobo Azcona}, year={2026}, eprint={2510.06107}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2510.06107}, } - arXivLeveraging Vision-Language Pre-training for Human Activity Recognition in Still Images
Shows that finetuning CLIP improves recognition of walking, running, sitting, and standing from individual photographs.
Paper@misc{mahanta2025leveragingvisionlanguagepretraininghuman, title={Leveraging Vision-Language Pre-training for Human Activity Recognition in Still Images}, author={Cristina Mahanta and Gagan Bhatia}, year={2025}, eprint={2506.13458}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2506.13458}, } - NAACL 2025 (Findings)Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks
Introduces Arabic embedding models and a 94-dataset benchmark covering dialects, cross-language tasks, and cultural contexts.
Paper@inproceedings{bhatia-etal-2025-swan, title = "Swan and {A}rabic{MTEB}: Dialect-Aware, {A}rabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks", author = "Bhatia, Gagan and Nagoudi, El Moatez Billah and El Mekki, Abdellah and Alwajih, Fakhraddin and Abdul-Mageed, Muhammad", editor = "Chiruzzo, Luis and Ritter, Alan and Wang, Lu", booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025", month = apr, year = "2025", address = "Albuquerque, New Mexico", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2025.findings-naacl.263/", doi = "10.18653/v1/2025.findings-naacl.263", pages = "4669--4685", ISBN = "979-8-89176-195-7" } - NAACL 2025 (SRW)DateLogicQA: Benchmarking Temporal Biases in Large Language Models
Introduces a benchmark probing temporal biases in language models across date formats, time periods, and reasoning tasks.
Paper@inproceedings{bhatia-etal-2025-datelogicqa, title = "{D}ate{L}ogic{QA}: Benchmarking Temporal Biases in Large Language Models", author = "Bhatia, Gagan and Tang, MingZe and Mahanta, Cristina and Kazi, Madiha and Peyrard, Maxime and Zhao, Wei", editor = "Ebrahimi, Abteen and Haider, Samar and Liu, Emmy and Haider, Sammar and Pacheco, Maria Leonor and Wein, Shira", booktitle = "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop)", month = apr, year = "2025", address = "Albuquerque, USA", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2025.naacl-srw.32/", doi = "10.18653/v1/2025.naacl-srw.32", pages = "321--332", ISBN = "979-8-89176-192-6" } - 2024
- ACL 2024 (Findings)FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models
Develops multimodal financial models that combine text, tables, images, retrieval, and tools for financial analysis.
Paper@inproceedings{bhatia-etal-2024-fintral, title = "{F}in{T}ral: A Family of {GPT}-4 Level Multimodal Financial Large Language Models", author = "Bhatia, Gagan and Nagoudi, El Moatez Billah and Cavusoglu, Hasan and Abdul-Mageed, Muhammad", editor = "Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek", booktitle = "Findings of the Association for Computational Linguistics: ACL 2024", month = aug, year = "2024", address = "Bangkok, Thailand", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2024.findings-acl.774/", doi = "10.18653/v1/2024.findings-acl.774", pages = "13064--13087" } - ACL 2024Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks
Introduces Arabic vision-language models and the Henna benchmark for evaluating visual reasoning and cultural understanding.
Paper@inproceedings{alwajih-etal-2024-peacock, title = "Peacock: A Family of {A}rabic Multimodal Large Language Models and Benchmarks", author = "Alwajih, Fakhraddin and Nagoudi, El Moatez Billah and Bhatia, Gagan and Mohamed, Abdelrahman and Abdul-Mageed, Muhammad", editor = "Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek", booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)", month = aug, year = "2024", address = "Bangkok, Thailand", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2024.acl-long.689/", doi = "10.18653/v1/2024.acl-long.689", pages = "12753--12776" } - ArabicNLP 2024Qalam: A Multimodal LLM for Arabic Optical Character and Handwriting Recognition
Introduces a model for recognizing printed and handwritten Arabic, including diacritics and high-resolution manuscript images.
Paper@inproceedings{bhatia-etal-2024-qalam, title = "Qalam: A Multimodal {LLM} for {A}rabic Optical Character and Handwriting Recognition", author = "Bhatia, Gagan and Nagoudi, El Moatez Billah and Alwajih, Fakhraddin and Abdul-Mageed, Muhammad", editor = "Habash, Nizar and Bouamor, Houda and Eskander, Ramy and Tomeh, Nadi and Abu Farha, Ibrahim and Abdelali, Ahmed and Touileb, Samia and Hamed, Injy and Onaizan, Yaser and Alhafni, Bashar and Antoun, Wissam and Khalifa, Salam and Haddad, Hatem and Zitouni, Imed and AlKhamissi, Badr and Almatham, Rawan and Mrini, Khalil", booktitle = "Proceedings of the Second Arabic Natural Language Processing Conference", month = aug, year = "2024", address = "Bangkok, Thailand", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2024.arabicnlp-1.19/", doi = "10.18653/v1/2024.arabicnlp-1.19", pages = "210--224" } - ArabicNLP 2024Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic
Builds an Arabic multimodal assistant that understands images and responds in six Arabic dialects.
Paper@inproceedings{alwajih-etal-2024-dallah, title = "Dallah: A Dialect-Aware Multimodal Large Language Model for {A}rabic", author = "Alwajih, Fakhraddin and Bhatia, Gagan and Abdul-Mageed, Muhammad", editor = "Habash, Nizar and Bouamor, Houda and Eskander, Ramy and Tomeh, Nadi and Abu Farha, Ibrahim and Abdelali, Ahmed and Touileb, Samia and Hamed, Injy and Onaizan, Yaser and Alhafni, Bashar and Antoun, Wissam and Khalifa, Salam and Haddad, Hatem and Zitouni, Imed and AlKhamissi, Badr and Almatham, Rawan and Mrini, Khalil", booktitle = "Proceedings of the Second Arabic Natural Language Processing Conference", month = aug, year = "2024", address = "Bangkok, Thailand", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2024.arabicnlp-1.27/", doi = "10.18653/v1/2024.arabicnlp-1.27", pages = "320--336" } - 2023
- ArabicNLP 2023Beyond English: Evaluating LLMs for Arabic Grammatical Error Correction
Evaluates prompting for Arabic grammar correction, showing that smaller task-finetuned models still outperform instruction-tuned LLMs.
Paper@inproceedings{kwon-etal-2023-beyond, title = "Beyond {E}nglish: Evaluating {LLM}s for {A}rabic Grammatical Error Correction", author = "Kwon, Sang and Bhatia, Gagan and Nagoudi, El Moatez Billah and Abdul-Mageed, Muhammad", editor = "Sawaf, Hassan and El-Beltagy, Samhaa and Zaghouani, Wajdi and Magdy, Walid and Abdelali, Ahmed and Tomeh, Nadi and Abu Farha, Ibrahim and Habash, Nizar and Khalifa, Salam and Keleg, Amr and Haddad, Hatem and Zitouni, Imed and Mrini, Khalil and Almatham, Rawan", booktitle = "Proceedings of ArabicNLP 2023", month = dec, year = "2023", address = "Singapore (Hybrid)", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2023.arabicnlp-1.9/", doi = "10.18653/v1/2023.arabicnlp-1.9", pages = "101--119" } - arXivChatGPT for Arabic Grammatical Error Correction
Studies ChatGPT for Arabic grammar correction and uses synthetic training data to improve specialized correction models.
Paper@misc{kwon2023chatgptarabicgrammaticalerror, title={ChatGPT for Arabic Grammatical Error Correction}, author={Sang Yun Kwon and Gagan Bhatia and El Moatez Billah Nagoud and Muhammad Abdul-Mageed}, year={2023}, eprint={2308.04492}, archivePrefix={arXiv}, primaryClass={cs.AI}, url={https://arxiv.org/abs/2308.04492}, } - SemEval 2023UBC-DLNLP at SemEval-2023 Task 12: Impact of Transfer Learning on African Sentiment Analysis
Applies transfer learning and multilingual finetuning to sentiment analysis across 14 African languages.
Paper@inproceedings{bhatia-etal-2023-ubc, title = "{UBC}-{DLNLP} at {S}em{E}val-2023 Task 12: Impact of Transfer Learning on {A}frican Sentiment Analysis", author = "Bhatia, Gagan and Adebara, Ife and Elmadany, Abdelrahim and Abdul-mageed, Muhammad", editor = {Ojha, Atul Kr. and Do{\u{g}}ru{\"o}z, A. Seza and Da San Martino, Giovanni and Tayyar Madabushi, Harish and Kumar, Ritesh and Sartori, Elisa}, booktitle = "Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)", month = jul, year = "2023", address = "Toronto, Canada", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2023.semeval-1.33/", doi = "10.18653/v1/2023.semeval-1.33", pages = "246--255" } - VarDial 2023SIDLR: Slot and Intent Detection Models for Low-Resource Language Varieties
Investigates multilingual models for intent detection and slot filling in language varieties with limited training data.
Paper@inproceedings{kwon-etal-2023-sidlr, title = "{SIDLR}: Slot and Intent Detection Models for Low-Resource Language Varieties", author = "Kwon, Sang Yun and Bhatia, Gagan and Nagoudi, Elmoatez Billah and Alcoba Inciarte, Alcides and Abdul-mageed, Muhammad", editor = {Scherrer, Yves and Jauhiainen, Tommi and Ljube{\v{s}}i{\'c}, Nikola and Nakov, Preslav and Tiedemann, J{\"o}rg and Zampieri, Marcos}, booktitle = "Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2023)", month = may, year = "2023", address = "Dubrovnik, Croatia", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2023.vardial-1.24/", doi = "10.18653/v1/2023.vardial-1.24", pages = "241--250" }
Experience
- Sep 2025 — Mar 2026Research Scientist
- Designed agentic RAG architectures with structured tool-calling for iterative evidence retrieval and answer revision, achieving state-of-the-art faithfulness.
- Contributed retrieval and verification components to a multi-agent QA platform deployed in production, serving 1.9M+ requests in under a year.
- Mar 2025 — Oct 2025Research Assistant
- Implemented attention-based interpretability methods to deliver actionable insights for improving LLM architecture (Date Fragments).
- Designed agentic AI frameworks and workflows for efficient data extraction from large documents in medical and financial domains.
- Sep 2022 — Sep 2024Research Assistant
- Engineered and trained a foundation model (Qalam) for Arabic OCR, achieving a 3-point F1 improvement over state of the art.
- Built and benchmarked multimodal LLMs (FinTral) using distributed training on a multi-GPU cluster.
- Oct 2021 — Sep 2022Machine Learning Engineer
- Owned the full lifecycle of a Vision Transformer system for image manipulation detection, from training to deployment via a REST API on AWS for a national defence client.
- Architected and deployed a scalable document intelligence pipeline on AWS processing over 10,000 documents daily, automating OCR, table, and text extraction.
- May 2021 — Dec 2021Software Engineer
- Enhanced the core NLU model for AWS Lex, improving intent recognition accuracy by 10% through targeted fine-tuning.
- Deployed a production-ready XGBoost model on AWS SageMaker for a healthcare application, automating detection of critical symptoms from streaming data.
Awards & Service
- EACL 2026Outstanding Reviewer Award
Contact
Research & collaboration
I welcome enquiries about research internships and collaborations in LLM interpretability, evaluation, and reliable AI systems.
gbhatia880@gmail.com