Evals 0/1

Evals

Eval sets

A set of real inputs with expected outcomes that every prompt or model change runs against. It replaces checking a few examples by eye with a comparison you can repeat.

Reading

WhiteBox lessons2

Videos1

Building with LLMs Various speakers 1 video

People to follow2

  • Hamel Husain

    Teaches AI evals

    Practical writing on how to debug, analyze and measure an AI product, including a guide that answers common questions about evals.

  • Eugene Yan

    Writes about AI systems and products

    Detailed, practical posts on building and evaluating AI systems, such as running product evals in three steps.