Class 13
Evaluating LLM Outputs and Building Evidence
Description
This session develops methods for evaluating LLM-generated classifications, extractions, and interpretations against underlying source material and other forms of evidence. Validation is important because plausible model output is not necessarily accurate, complete, reproducible, or sufficient to support a professional conclusion.
Motivation
How do we know whether the model is right?
Review
The prior session expanded our analytical capabilities by allowing LLMs to interpret complex financial language. This session reconnects those capabilities to the earlier concept of data veracity by distinguishing an AI-generated answer from validated analytical evidence.