Research Engineer · NLP & LLMs

Stefano
Campese

Research engineer with ~10 years of experience across LLM alignment research and production systems. I work on making models more honest and reliable — from preference optimization to RAG pipelines serving real users.

01 — Research

What I work on

My research shifted to LLM honesty and calibration — models that give consistent answers and reflect their actual confidence

LLM Calibration & RL

GRPO with custom reward signals to penalize hallucinated claims.

QA Coherence

q-RAG: improving LLM coherence up to 28pp by augmenting prompts with equivalent questions. QUADRo: database QA over 6.3M pairs.

Retrieval Systems

Billion-scale semantic similarity and ranking pipelines.

Data Curation

Multilingual data generation, large-scale annotation pipelines, quality filtering.

02 — Publications

Selected papers

NAACL '25 Analyzing and Improving Coherence of LLMs in QA
AACL '25 Improving Document Retrieval Coherence for Semantically Equivalent Queries
IntelliSys '25 Domain-Specific and Cross-Lingual Synthetic Data Generation for Information Retrieval Training in RAG Applications
EACL '24 Pre-training Methods for Question Reranking
EMNLP '24 Datasets for Multilingual Answer Sentence Selection
EMNLP '23 QUADRo: Dataset and Models for QA Database Retrieval
03 — Experience

Where I've been

Oct 2024 — Apr 2025

Applied Scientist II

Amazon AGI — Milan

Preference optimization pipelines (DPO), human annotation systems for calibration evaluation, hallucination detection classifiers.

2022 — 2025

PhD Researcher

University of Trento

LLM calibration via RL (GRPO/DPO/ORPO, up to 72B params on 8×L40S), QA coherence, dataset curation. 6 publications.

2015 — Present

AI Team Leader / Senior ML Engineer

OMNYS — Vicenza

Leading 10 engineers building Omnyscient, a multi-agent RAG platform. Hybrid retrieval, LangGraph, AWS Bedrock. Hundreds of thousands of users.

2021 — 2023

Applied Scientist Intern (×3)

Amazon — Alexa & AGI, LA & Torino

Scalable retrieval and ranking pipelines for database QA. Dense retrieval, billion-scale semantic similarity.

04 — Stack

Tools & technologies

Python PyTorch Hugging Face DeepSpeed vLLM TRL GRPO / DPO / ORPO LangGraph OpenSearch Pyserini / BM25 FastAPI AWS (Bedrock, ECS, Terraform) Docker CI/CD
05 — Open Source

Projects

Daedalus AI

MCP plugin for ML experiment lifecycle: hypothesis-driven design, distributed training (multi-GPU, SSH), convergence detection, DPO/GRPO dataset validation. 478 tests.