Essential session storage is required to sign in. Optional analytics remain off unless you allow them.
Build a prompt that extracts STRUCTURED JSON (a fixed schema) from messy free text — e.g. pull name/skills/years from resumes, or vendor/total/date from invoice text. Define a GOLDEN eval set: ~15 inputs with expected outputs. Score the prompt's accuracy field-by-field, then improve it with few-shot examples, an explicit schema, and (where it helps) brief reasoning before the JSON — re-running the eval to prove the change helped and didn't regress. Instruct the model to return ONLY valid JSON and handle missing fields explicitly.
Returns valid JSON matching the schema reliably (only-JSON instruction, example shape); missing-field handling defined.
A real golden set with expected outputs; field-level scoring; runs before/after a change to catch regressions.
Few-shot examples and/or reasoning-then-JSON applied where they measurably improve accuracy.
Output parses programmatically; edge cases (empty, ambiguous) handled rather than crashing.
Use this README structure to make the work reviewable.
DevPath fetches a read-only, size-capped snapshot. Submitted code is never executed.
# A structured-extraction prompt with an eval set ## Goal Build a prompt that extracts STRUCTURED JSON (a fixed schema) from messy free text — e.g. pull name/skills/years from resumes, or vendor/total/date from invoice text. Define a GOLDEN eval set: ~15 inputs with expected outputs. Score the prompt's accuracy field-by-field, then improve it with few-shot examples, an explicit schema, and (where it helps) brief reasoning before the JSON — re-running the eval to prove the change helped and didn't regress. Instruct the model to return ONLY valid JSON and handle missing fields explicitly. ## Acceptance criteria - [ ] Structured output: Returns valid JSON matching the schema reliably (only-JSON instruction, example shape); missing-field handling defined. - [ ] Eval set & scoring: A real golden set with expected outputs; field-level scoring; runs before/after a change to catch regressions. - [ ] Technique use: Few-shot examples and/or reasoning-then-JSON applied where they measurably improve accuracy. - [ ] Robust parsing: Output parses programmatically; edge cases (empty, ambiguous) handled rather than crashing. ## Verification - [ ] Document installation and run commands. - [ ] Record automated checks and their exact commands. - [ ] Add representative output, screenshots, or a short demo where useful. - [ ] Confirm failure paths and known constraints. ## Decision log ### Decision title - Context: - Choice: - Alternatives considered: - Tradeoffs: ## Evidence - Link each rubric criterion to the file, test, or artifact that demonstrates it. ## Known limitations - List what is intentionally out of scope and what should be improved next.