Data & ML Talk

Evaluating models when nobody agrees on the answer

Name: Evaluating models when nobody agrees on the answer
Start: 2027-05-20T11:00:00+02:00
End: 2027-05-20T11:45:00+02:00
Location: Kiln Room, The Foundry Hall

When your labels come from humans who disagree, accuracy is a number with no meaning. Dae-won presents a practical approach to measuring a model against disagreement instead of pretending it away.

Covers inter-annotator agreement, calibrated confidence and how to report results to a product team without lying by omission.

Evaluation
ML

Speaker

Dae-won Seo

Machine Learning Engineer, Parallax Labs

Dae-won builds evaluation tooling for language and ranking models and cares more about disagreement than accuracy. He holds a PhD in statistics he rarely mentions.