Validation ladder
Turn the camera concept into replayable field evidence
This is the planned validation route: bench radiometry and safe small-hot-target tests, then low-altitude flights, controlled-burn evidence, deliberate false-positive scenes, and acceptance review. None of these rungs has been climbed yet.
Validation ladder
Each stage adds operating realism without losing calibration, truth, and replayability.
Evidence boundary
A flight only counts if the result can be replayed after detector, optics, calibration, or threshold changes.
The field plan uses the existing calibration kit, raw-frame pipeline, survey model, dataset-validation policy, and payload constraints as a single evidence contract.
Missing references, unknown route state, absent target truth, or missing raw frames keep a flight in engineering-diagnostic status rather than stakeholder detection evidence.
Test stages
The first pass separates bench confidence, surrogate target detection, and operational-style survey behavior.
| Stage | What it proves | Minimum output |
|---|---|---|
| Bench radiometry | Detector warm-up, references, shutter/NUC logging, window offset, and calibration grade are known before flight. | Bench manifest, reference residuals, warm-up curve, window-offset note, and quality decision. |
| Small-hot-target bench | Safe measured targets reveal the detection curve before the project depends on fireground access. | Detection versus target size, temperature contrast, background, threshold, focus, and calibration grade. |
| Field surrogate | The full payload workflow can find measured warm targets and reject rural clutter. | Raw frames, target truth, geotag check, alert packets, misses, false alarms, and reviewer labels. |
| Validation flight | The 35 m high-overlap route gives benchmark-grade target and no-fire evidence. | Frame, reference, route, weather, target-truth, alert, and failure-packet manifests. |
| Candidate sweep | The 80 m survey setting can be judged by target detection and false alarms per area. | Operational-style metrics per flight minute and per surveyed hectare. |
| Controlled burn | Real smoke, heat, crew coordination, and response workflow are tested only after approvals. | Controlled-burn evidence packet clearly separated from surrogate and public-data results. |
Targets and negatives
The first field day needs measured warm targets and deliberate no-fire backgrounds.
Measured target set
Use non-flaming high-emissivity targets with known size, temperature, material, location, deployment time, and removal time.
Background variety
Include soil, grass, bark, rock, metal-adjacent clutter, mixed sun/shade, and matched empty routes.
False-positive terrain
Capture hot rocks, bare soil, roads, gates, vehicles, machinery, people, livestock, buildings, water, dust, and reflective surfaces.
Failure packet
Keep missed targets, false alarms, rejected hot clutter, and excluded frames beside the headline metrics.
Route families
Use the survey model as the field-test scaffold.
Start with the 35 m, 2 m/s, 90 percent overlap route for calibration checks, close inspection, and target truth. Move to the 80 m, 8 m/s, 30 percent overlap sweep for the first operational-style survey case.
Keep the 120 m, 12 m/s broad sweep as a coverage-biased scenario until radiometry proves the centimetre-scale target case at that GSD. Every route should bracket survey frames with reference passes and a no-fire terrain pass matched for solar loading.
Data capture
The test output is a manifest-backed evidence packet, not a folder of interesting images.
Flight manifest
Route, pilot, operator, payload, detector, firmware, optics profile, software version, storage root, and weather source.
Frame manifest
Timestamp, raw checksum, calibration grade, shutter/NUC state, GPS/pose, route segment, and storage pointer.
Reference manifest
Target ID, type, emissivity assumption, measured temperature, frame IDs, position, validity flag, and residual.
Alert manifest
Candidate ID, frame IDs, geotag, threshold band, calibration version, reference age, crop pointer, and review label.
Acceptance gate
Stakeholder evidence requires calibration state, target truth, route context, and failed examples.
The first validation report should separate safe surrogate, public dataset, and controlled-burn evidence. It should report target detection by size and contrast, false alarms per flight minute and hectare, geotag error, calibration residual, alert latency, replay agreement, and exclusion counts.
Flights with missing references, unknown operator notes, unknown target truth, or missing raw frames can guide engineering choices, but they should not be described as operational detection performance.