This project assembles a checked incident brief from a six-minute dispatcher call and a seven-minute depot camera clip. The artifacts are fictional and independently identified. The deliverable is a brief that separates what was heard, what was seen, what the access log reports, and what remains unknown. It must not turn a blurry frame or noisy word into a confirmed cause.
Project: produce a checked multimodal incident brief
Build the evidence packet
Transcribe the call with segment IDs, speaker labels, and uncertainty flags. Mark the device serial near minute three as unreadable. Review the video around the cited gate event and keep a time range for every visual claim. Add the access-log record as a separate source. Ask the model to draft impact, observed sequence, unknowns, and next checks with evidence IDs. Validate the IDs against the packet and preserve a review state when audio and video appear to conflict.
Review the result
Construct 33 test cases by varying noise, speaker overlap, clip sampling, and missing access logs. Reserve 11 cases as a holdout. A reviewer listens to uncertain audio spans and scrubs cited video ranges. Check that the brief never claims the gate sensor failed without telemetry, never invents a serial, and never makes a maintenance ticket from uncertain identity. The final case packet records the prompt version, artifacts, missed-event count, and any human correction.
Artifacts: CALL-31, VIDEO-BAY-22, ACCESS-LOG-44.
Output fields: observed_audio, observed_video, log_fact, unknowns, next_check.
Audio gate: uncertain serial stays unresolved.
Video gate: each visual claim has clip ID and time range.
Release gate: no unsupported cause or unauthorized work-order effect.Performance and operating cost
Processing cost grows with recording duration, frame density, and the number of reviewable segments. Lower sampling may reduce cost but miss short events, so report missed-event rates on the holdout. Human review can focus on flagged spans rather than reading every transcript word, provided those flags are reliable. Keep media retention and access within the product policy. A compact brief saves operator time only if its evidence links allow disputed claims to be checked.
Common Mistakes
- Do not merge a reported pressure reading with a sensor measurement.
- Do not turn a sampled frame into a continuous timeline.
- Do not execute a work order from an uncertain asset ID.
Connected lessons
- Audio prompts: preserve speakers, timestamps, and uncertain words
- Video prompts: cite the moment and the observation
- Multimodal prompts: separate what an image shows from what it suggests
- Evidence IDs: make generated claims auditable against supplied records
- Human handoff: preserve evidence and the reason for uncertainty
- Applied prompt engineering decisions
Continue with: Project: reconcile a warehouse evidence packet.
