Skip to main content
Back to top
Ctrl
+
K
Search
Ctrl
+
K
Overview
GenAI Cookbook
Learn
1. Agents overview
2. Agent fundamentals
2.1. Data pipeline
2.2. Retrieval, augmentation, and generation (aka RAG Agent)
2.3. Evaluation & monitoring
2.4. Governance and LLMops
3. Agent quality knobs
3.1. Data pipeline
3.2. Retrieval, augmentation, and generation (aka RAG Agent)
4. Evaluating Agent quality
4.1. Defining “quality”: evaluation sets
4.2. Assessing performance: Metrics that Matter
4.3. Enabling Measurement: Supporting Infrastructure
5. Evaluation-driven development workflow
Product demo
10 minute demo of Mosaic AI Agent Framework & Agent Evaluation
Implement
Prerequisite:
Gather requirements
Step 1:
Clone code repo & create compute
Step 2:
Deploy POC to collect stakeholder feedback
Step 3:
Curate an Evaluation Set from stakeholder feedback
Step 4:
Evaluate the POC’s quality
Step 5:
Identify the root cause of quality issues
Debugging retrieval quality
Debugging generation quality
Step 6:
Iteratively implement & evaluate quality fixes
Implement data pipeline fixes
Step 6:
Deploy & monitor
Index
