Pilot Report &
FAR Resolution Evidence

Executive Summary

The current draft draws primarily on US-based validation studies conducted by Virridy and academic partners (Boulder Creek CO, Seine River FR, Yampa River CO, plus historical comparison data from Kenya and Malawi). These data establish the sensor's intrinsic measurement performance and the deployed regression model. They will be augmented and, where appropriate, replaced by Rwanda and Kenya field data from the Amazi Meza pilot deployment as it accumulates over the verification period.

FAR resolution status (current snapshot)

FAR	Requirement	Status	Primary Evidence
FAR 1	Sensor Validation & Calibration Protocols	In Progress	n=209 paired Lume↔Colilert (R²=0.881); n=303 paired Lume↔Membrane Filtration (R²=0.872); calibration protocol documented; field-rotation protocol pending Rwanda
FAR 2	AI/ML Implementation & Validation	In Progress	Resolved by design: deployed model is a transparent linear regression (no AI/ML); model card with full coefficients in Appendix B; training-set provenance from Bedell et al. (2022) and Knopp et al. (2026); cross-validation results; version control via Lume desktop dashboard releases
FAR 3	Manual ↔ Digital Integration Protocol	In Progress	Cross-check protocol drafted; US duplicate-sample QA data; Rwanda routine QA cadence pending Phase 2 deployment
FAR 4	SDWS 23 & 27 Exploration	In Progress	US bench classifier validated on two test setups (Closed Pipe Flow 95.3% / Bucket 90.7%, κ ≥ 0.85); SDWS 23 & 27 recommended for inclusion in the monitoring plan; Rwanda field-validation pending

No FAR is currently resolved; all four are in progress with US-pilot evidence in place and Rwanda/Kenya field evidence accumulating. Final closure of each FAR requires the field-deployment data identified in the per-FAR sections below.

Pilot Implementation Status

What has been deployed

The pilot is structured in two phases per the approved Implementation Plan:

• Phase 1 — Mobile Lume Validation: US baseline validation studies (complete or in progress); Rwanda field validation rotations TBD: schedule confirmed for [INSERT MONTH] 2026.
• Phase 2 — Permanent Installation: 30–50 permanent sensors planned at Amazi Meza institutional sites in Rwanda TBD: deployment cohort and dates.

Deviations from the approved plan

TBD — Document any scope, schedule, or methodology deviations from the approved plan here. As of this draft, no formal deviations have been submitted.

Sensor and site inventory snapshot

Cohort	Sensors deployed	Sites	Active since	Data points (cumulative)
US validation (Boulder Creek)	3	BC-CU, BC-55, BC-Can	April 2026	~50,000+ continuous readings
US bench / lab (multi-sensor)	10+	Lab fixture, Yampa, Seine	2022 → present	n=512 paired lab samples (combined Colilert + MF)
Rwanda Amazi Meza — Phase 1	TBD	TBD	TBD	TBD
Rwanda Amazi Meza — Phase 2	TBD	TBD	TBD	TBD

FAR 1 — Sensor Validation & Calibration In Progress

1.1 Paired sensor–reference comparison evidence

The Lume sensor's measurement performance against laboratory reference methods has been characterised across multiple independent studies. The two relevant gold-standard methods are Colilert (IDEXX defined-substrate technology) and membrane filtration (MF, US EPA Method 1604). Full source: thelume.ai/research.

Reference method	Paired n	R²	Binary accuracy at 10 CFU/100 mL	Cohen's κ	Source
Colilert (IDEXX)	209	0.881	0.92 (balanced 0.92)	0.84	Knopp et al. (2026); thelume.ai/research
Membrane Filtration (EPA 1604)	303	0.872	—	—	MF-trained model, method-agnostic validation
Three-way (Lume / Colilert / MF)	153	—	Lume κ=0.88 vs. Colilert	Colilert↔MF κ=0.40	Method-comparison subset

Notable: the Lume↔Colilert agreement (κ=0.88, almost perfect) is substantially stronger than the Colilert↔MF agreement (κ=0.40, fair) on the same n=153 split-sample set. This indicates the Lume's reproducibility against either reference method is on the order of, or better than, the inherent reproducibility between two accepted laboratory methods.

1.2 Performance by WHO risk category

WHO-defined drinking-water risk bands (Low <1, Intermediate 1–10, High 10–100, Very High >100 CFU/100 mL):

Risk band split	Threshold	Overall accuracy	Balanced accuracy	Cohen's κ
Safe vs. any contamination	1 CFU/100 mL	0.91	0.91	0.82
WHO Low/Intermediate vs. High+	10 CFU/100 mL	0.92	0.92	0.84
3-category (<10, 10–100, >100)	multi	0.91	0.85	0.60
Recreational binary (Seine R., held-out)	900 CFU/100 mL	0.968	0.94	—

1.3 Calibration protocol

The Lume sensor is calibrated at the operating point led_power = 512, sipm_bias ∈ [2960, 3040] (target 3000) — these are the parameters under which the deployed CFU regression and turbidity (NTU) regression were trained. Sensors falling outside this window automatically fall back to the (LED, bias) combo nearest the target via the Lume backend; readings from the fallback combo are flagged as “provisional” on operational dashboards.

Calibration check	Frequency	Tolerance / Pass criterion	Action on failure
Operating-point combo (LED 512, bias ~3000)	Continuous (every reading)	bias ∈ [2960, 3040]	Fall back to nearest combo; dashboard flags “Provisional”; replace sensor if fallback persists > 7 days
Turbidity (ToF) zero-baseline check	Continuous (per-sensor 10th-%ile in-water)	Sensor-relative anomaly: NTU = max(0, 2.05 × (sps − baseline))	Re-baseline automatically from rolling in-water minimum
Field paired CBT or Colilert grab-sample	Per institutional visit during Phase 1; at least quarterly during Phase 2	Within 1 WHO risk band of Lume estimate	Investigate; flag period; re-train if systematic
Sensor swap / retirement	On detection of persistent fallback, low battery (<3.8 V steady), or repeated air-exposed flag	—	Replace in field; data continuity preserved via Blues check-in chain of custody

1.4 Drift monitoring

The Lume Fleet Health dashboard (internal operations tool, requires login) tracks each sensor's:

• Battery drift (V/week) — trend tag fires above 0.05 V/week loss; alert above 0.15 V/week.
• Data gap — flags any sensor with no Pumphaus telemetry for >6 hours despite Blues check-in.
• GPS drift — flags >100 m from expected location (warn) and >500 m (alert).
• Calibrated-combo coverage — flags when the firmware bias-sweep skips the calibrated window.

1.5 What's still needed

FAR 2 — Model Documentation (Linear Regression, no AI/ML) In Progress

2.1 Model card

Attribute	Value
Model family	Linear regression with one interaction term (quantitative CFU); logistic regression (binary > 10 CFU/100 mL). No AI, no decision trees, no ensemble methods, no neural networks in the deployed pipeline.
Output (primary)	E. coli concentration (CFU/100 mL), continuous regression with sensor-derived uncertainty bounds
Output (secondary)	Categorical risk class (WHO Low / Intermediate / High / Very High)
Input features	Tryptophan-like fluorescence (TLF, mon2_val), water temperature (floor_temp), turbidity proxy (Time-of-Flight return per SPAD, tof_mean), and the TLF×temperature interaction term
Operating-point filter	Rows accepted only at led_power = 512, sipm_bias ∈ [2960, 3040] (post-2026-04-27 widening); rows outside the window fall back to nearest combo and are flagged provisional
Pre-processing	Hampel filter on TLF outliers; ±2 min annotation buffer; top-2% residual trim on training set; centred rolling-mean low-pass on bias-corrected TLF
Quantitative regression form	Linear regression with interaction (ported from Lume desktop dashboard src/model/e_coli.rs ACTIVE_MODEL): CFU = 11.477 − 0.0146 · mon2 − 0.671 · temp + 0.0172 · tof + 0.000785 · mon2 · temp, floored at 0.1 CFU/100 mL
Classifier (logistic)	P(>10 CFU/100 mL) = sigmoid(−26.32 + 0.00974 · mon2 − 0.186 · temp + 0.204 · tof)

2.2 Training data provenance

Dataset	n	Reference method	Locations	Use
Lume v1.2 multi-site validation	~512 paired (combined Colilert + MF)	Colilert / MF	US (Colorado: Boulder Creek, Yampa); France (Seine); historical Kenya, Malawi	Primary regression training + cross-validation
Bedell et al. (2022) Water Research	Published	Culture-based	Kenya groundwater (37 sites, Sorensen et al. 2018 cohort)	Foundational TLF↔E. coli relationship; 83% reported accuracy
Knopp et al. (2026) EarthArXiv	Published	Colilert + MF	Multi-site (US + France)	Lume v1.2 sensor design + multi-site validation results
Demaree et al. (2026) ES&T Water	Published	Colilert	Upper Yampa River, CO	Sensor-informed predictive models
Nowicki et al. (2020)	Published	Culture	Malawi	TLF reproducibility (14% RPD vs. ≥26% for culture)

2.3 Cross-validation results

CV scheme	RMSE	MAPE	R² / Accuracy	Notes
Temporally-structured global CV (training)	0.58 log units	22.5%	—	Across all sites, blocking on time
Temporally-structured global CV (test)	0.59 log units	18.8%	—	Held-out time blocks
Boulder Creek held-out test	—	7.1% (log space)	R² = 0.67	>75% of predictions within Colilert analytical uncertainty
Seine River held-out test (binary <900 / >900 CFU/100 mL)	—	—	96.8% accuracy, 94% balanced	Cross-domain generalisation (saline-influenced river)

2.4 Procedures for retraining and version control

• Single source of truth. The deployed regression coefficients live in functions/js/ecoli-model.js.js in the Virridy code repository (model version 2026-04-27-turbidity-relative). The same coefficients are mirrored in the offline Lume desktop dashboard (src/model/e_coli.rs ACTIVE_MODEL); both copies must move together.
• Versioning. Each model release carries a date-stamped MODEL_VERSION string. The shared model file is served with Cache-Control: no-store so every dashboard fetches the latest on every page load — no per-page cache busting required.
• Retraining trigger. Re-fit is performed when (a) ≥100 new paired samples are accumulated from a new geography or water type, (b) systematic residual bias is detected in any verification audit, or (c) a sensor hardware revision changes optical or thermal characteristics.
• Audit trail. All training notebooks, paired-sample CSVs, and fitted coefficients are committed to the version-controlled SweetSenseInc/lume_desktop_dashboard repository on the pc-sandbox branch.

2.5 What's still needed

FAR 3 — Manual ↔ Digital Integration In Progress

3.1 Cross-check protocol (draft)

1. Routine manual sampling cadence. A Compartment Bag Test (CBT) grab sample is collected at every Amazi Meza institutional visit (target: monthly per site during Phase 2). Sample is taken from the same point-of-consumption tap as the Lume install, within a ±5 min window of a Lume reading.
2. Independent laboratory analysis. CBT results are read at 24–28 h incubation following the Aquagenx CBT protocol; results are recorded as a most-probable-number (MPN) value and the corresponding WHO risk band.
3. Pairing & storage. Each (CBT, Lume) pair is uploaded to mWater under the project's "Lume 1.2 — 2026 Validation Data" datagrid, including: site ID, sample timestamp, CBT MPN, Lume CFU prediction at sample time, water temperature, turbidity, and enumerator name.
4. Discrepancy thresholds. A pair is flagged as discrepant if the Lume prediction disagrees with the CBT result by >1 WHO risk band.
5. Discrepancy resolution. Flagged pairs are reviewed by the Virridy water-quality lead within 7 days. Resolution paths: (a) confirmed true sensor error → enter the next retraining batch and replace sensor if persistent; (b) confirmed true CBT error → annotate dataset, exclude from accuracy stats; (c) ambiguous → schedule duplicate manual sample on next site visit.
6. Verifier-facing log. All discrepancies (raised, resolved, outstanding) are exposed in this report's Appendix C and updated continuously through the verification window.

3.2 US-pilot duplicate-sample QA evidence

The Boulder Creek validation work has accumulated paired-sample QA data with both Colilert and MF reference methods. TBD: insert duplicate-sample replicate spread, per-replicate CV, and discrepancy rates from the US pilot here once the Lume mWater datagrid for Rwanda has the same structure.

For US bench-comparison context, the Colilert↔MF discrepancy rate on the same paired dataset (n=153) was κ=0.40 (“fair agreement”) — which sets the practical floor for inter-method discrepancy. A Lume↔CBT discrepancy rate similar to Colilert↔MF should be considered nominal rather than a sensor failure.

3.3 What's still needed

FAR 4 — SDWS 23 & SDWS 27 Exploration In Progress

4.1 Why these parameters are relevant

SDWS Parameter 23 (volume of safe water treatment) and SDWS Parameter 27 (operational days) are the two SDWS parameters most amenable to digital substitution by an in-line Lume sensor. The Lume's existing on-board channels — UVLED / SiPM / board temperatures and ToF turbidity — change predictably when the sensor's optical interface transitions between air-exposed, still water, and flowing water. Mapped to the methodology:

• SDWS 27 (operational days) reduces to a daily binary classification: was the treatment system in active service, yes or no? The Lume's air-vs-water discrimination is the direct sensor for this — a day with sufficient sub-aquatic minutes is operational; a day spent dry is not.
• SDWS 23 (water volume) reduces to a continuous time integral: volume = Σ (flowing seconds × calibrated flow rate). The Lume's flowing-vs-still discrimination is the direct sensor for this — flowing time is what gets multiplied by a per-site flow-rate calibration to recover litres dispensed.

4.2 US bench evidence — two test setups

An end-to-end US bench study built and validated a per-point flow-state classifier on Lume sensor #50051 across two distinct fixtures. Full results, methodology, snapshot data, and the static analysis page are at piped-flow-test.pages.dev/analysis/ (snapshot 2026-04-27).

• Test 1 — Closed Pipe Flow (2026-04-13 → 04-16, 696 sample points): pump-driven flow through a closed pipe loop, alternating ~15 min flowing + ~45 min still per hour. Designed primarily to validate the flowing↔still discrimination that drives SDWS 23.
• Test 2 — Filling/Draining Bucket (2026-04-17 → present, 905 sample points): bucket dispenser cycling through fill, hold, and drain phases with intentional Air exposure between fills. Designed primarily to validate the air↔water discrimination that drives SDWS 27.
• Classifier: per-point KNN (k=3, distance-weighted, class-balanced) on seven temperature-dynamics features, with leave-one-region-out cross-validation. Air predictions are gated by a turbidity threshold (signal_per_spad_kcps ≥ 80) so the classifier cannot call Air without optical evidence.

4.3 Headline performance against each SDWS parameter

Test setup	Primary SDWS target	Overall accuracy	Cohen's κ	Key per-class result
Closed Pipe Flow (n=696 points)	SDWS 23 (volume)	95.3%	0.89	Flowing recall 96.3%, Still recall 98.1%
Bucket Dispenser (n=899 points)	SDWS 27 (operational days)	90.7%	0.85	Air recall 96.0%, Air precision 100.0%
Combined (n=1,595 points)	—	93.0%	0.85	All three classes ≥ 85% recall

For an integral-over-time deployment metric, this corresponds to ~7% time-budget error per measurement period across both setups combined: roughly 5 min of misclassified state per 100 min on the Closed Pipe Flow rig (relevant to SDWS 23) and roughly 9 min per 100 min on the Bucket rig (relevant to SDWS 27). Both are well within the precision needed for monthly carbon-credit verification cycles.

4.4 SDWS 27 (operational days) — feasibility evidence and approach

Feasibility: the Bucket Dispenser test directly demonstrates SDWS-27-grade air-vs-water discrimination. Air precision is 100% (every Air prediction was correct — zero false-positives) and Air recall is 96% (96 of every 100 actual air-exposed minutes are correctly labelled). For a binary daily question — "did this site have water for ≥ N minutes today?" — this exceeds the precision needed to meet Gold Standard's audit requirements. The 4% of missed Air minutes are biased toward conservatism (counting borderline air-exposed periods as Still under-counts air-exposure days, never over-counts).

Proposed deployment formula:

• Aggregate the Lume's per-point air-vs-water classification at daily resolution.
• Define an operational day as one in which the sensor reports water for ≥ 120 minutes (configurable; baseline aligned with Amazi Meza institutional schedules of 4 h+ daily service windows).
• Per-sensor calibration of the air/water threshold during the post-install 4–6 h equilibration window (per the install-validation pattern documented in the bench-annotations study).
• Cross-check against Amazi Meza in-person attendance logs at every quarterly site visit.

4.5 SDWS 23 (water volume) — feasibility evidence and approach

Feasibility: the Closed Pipe Flow test directly demonstrates SDWS-23-grade flowing-vs-still discrimination. Overall accuracy is 95.3% with Flowing precision 94.0% and Still precision 96.4%. The residual error is dominated by Flowing → Still under-counts (a directionally favorable bias for a conservative volume estimate; see below). The classifier is therefore sensor-side ready for SDWS 23 estimation, conditional on per-site flow-rate calibration.

Proposed deployment formula:

• Use the per-point classifier to label each Lume reading as Flowing or Still, integrated across the day to recover total flowing time.
• Calibrate the dispensed flow rate once at Phase-2 install per site via a manual fill test (graduated bucket, 60 s repeated 3×).
• Daily volume = Σᵢ (flowing duration_i × calibrated flow rate_site), with classifier-side error budget of ≤ 5 min per 100 min observed and a separately-reported uncertainty contribution from the flow-rate calibration repeats.
• The classifier's bias is conservative: 49 of the 219 Flowing points in the Bucket test were misclassified as Still, and 6 of 162 Flowing points in the Closed test were under-counted similarly. For SDWS-23 verification this is the favorable direction (lower-bound volume estimate), but for an unbiased report a per-class recall correction matrix can be applied at the integration step.

4.6 Sensor-cadence dependency

The dominant bottleneck across both tests is sample cadence. At the snapshot rate of one sensor reading per ~6 min, 15-min Flowing windows yield only 2–3 samples per event, leaving the temperature-derivative features statistically underpowered. The straightforward operational fix is to return the firmware to 1-min sample cadence (the configuration the original 2026-04-18 closed-pipe-flow study used), which would put 15+ samples in every Flowing event and is expected to lift Flowing recall on both setups well above 95%. This will be implemented before the Phase-2 Rwanda deployment.

4.7 Recommendation for the Monitoring Plan

4.8 What's still needed for full FAR 4 closure

Phase 1 Results — Mobile Lume Validation

US sites (baseline)

The US-side mobile validation work establishes the Lume's intrinsic measurement performance and the linear-regression model that Rwanda Phase-1 inherits (no AI/ML — see FAR 2). Headline metrics are summarised under FAR 1 and FAR 2 above. Live US deployments include three Boulder Creek sensors (BC-CU, BC-55, BC-Can) currently streaming continuously to the Boulder Water dashboard (boulder-water.pages.dev).

Rwanda sites (Amazi Meza Phase 1)

Phase 2 Results — Permanent Installation

Planned Phase-2 reporting (pre-populated structure)

Metric	Aggregation	Target	Result
Sensor uptime	% of expected check-ins received	≥95%	TBD
Calibrated-combo coverage	% of readings at led=512, bias∈[2960,3040]	≥90%	TBD
Battery longevity	Median V/week drift	<0.05 V/week	TBD
CBT cross-check rate	Paired samples per site per quarter	≥3	TBD
Discrepancy rate	% of CBT pairs >1 WHO band off	≤15% (matched to Colilert↔MF baseline)	TBD
Operational-day coverage (SDWS 27)	Days/site with ≥120 min in-water flag	≥28 / 30 days	TBD

Findings, Limitations & Recommendations

Findings (current draft)

1. The Lume sensor's measurement performance against laboratory reference methods is quantitatively at parity with, or stronger than, the agreement between two accepted laboratory methods (Lume↔Colilert κ=0.88 vs. Colilert↔MF κ=0.40 on n=153 paired samples).
2. The deployed estimation model is, by explicit design choice, a transparent linear regression with one interaction term (no AI, no ML ensemble, no neural network). The full coefficient set is in Appendix B and is portable across the Lume desktop and cloud implementations from a single source-of-truth file.
3. Cross-domain generalisation is empirically demonstrated by the 96.8% binary accuracy on the held-out Seine River (Paris) dataset, a saline-influenced urban river materially different from the US training sites.
4. Flow-state and air-vs-water discrimination from the existing ToF channel is feasible at the bench level (KNN on temperature-derivative features), supporting an in-principle path to SDWS 23 and SDWS 27 estimation.

Limitations

• Geographic. Current evidence is dominated by US and European sites. Rwanda-specific water-source generalisation has not yet been demonstrated at scale.
• Sample size by category. The 3-category (Low/Mid/High) classifier shows κ=0.60 vs. κ=0.84 at the binary 10-CFU split — most of the disagreement is in the mid band where lab reference methods themselves disagree.
• SDWS 23/27. Bench-level proof-of-method only; no field accuracy numbers yet.
• Reference-method floor. Colilert↔MF agreement on the same samples is κ=0.40, so a portion of any Lume↔CBT discrepancy reflects the inherent variability between any two microbial water tests rather than sensor error.

Recommendations

1. Schedule the first Rwanda Phase-1 mobile rotation within the next quarter, paired with at least 30 CBT samples per site.
2. Ship Phase 2 hardware for the first 5 institutional sites in parallel with Phase 1 to compress the verification timeline.
3. Lock the model version at 2026-04-27-turbidity-relative for the duration of the verification window unless a documented retraining trigger fires; any mid-window model change requires an audit-trail entry.
4. Adopt the Colilert↔MF baseline (κ=0.40 / 1-WHO-band tolerance) as the formal discrepancy threshold for FAR 3 in this report and in the verifier submission, and document this calibration explicitly.

Appendices

Appendix A — Live data references

• thelume.ai/research — full validation evidence, confusion matrices, regression plots, training-set provenance.
• thelume.ai/validation — live mWater field-validation datagrid (currently 4 entries; auto-refreshes on page load).
• thelume.ai/inventory/fleet — internal Fleet Health dashboard (login required) with per-sensor calibration combo, drift, and uptime telemetry.
• piped-flow-test.pages.dev/analysis/ — static snapshot (2026-04-27) of the SDWS 23 / 27 classifier results, with confusion matrices and per-class metrics for both test setups. Live tool at piped-flow-test.pages.dev.
• lume-bench-annotations.pages.dev — long-running bench data (TLF, ToF, temperature) with operator-annotated air / still / twist-shake conditions.
• boulder-water.pages.dev — public Boulder Creek dashboard (City of Boulder Utilities × CU Boulder × Virridy) showing live sensor data.

Appendix B — Model coefficients (verifier reference)

Authoritative copies live in virridy-lume-summary/functions/js/ecoli-model.js.js and SweetSenseInc/lume_desktop_dashboard:src/model/e_coli.rs.

Model	Coefficient	Value
CFU regression (linear + interaction)	intercept	11.4773
	mon2_val	−0.0146124
	floor_temp	−0.6708344
	tof_mean	0.0171708
(continued)	mon2_val × floor_temp	0.0007850
Logistic classifier (P > 10 CFU/100 mL)	intercept	−26.317203
	mon2_val	0.009744
	floor_temp	−0.185900
	tof_mean	0.204381
Turbidity (NTU) regression — bench, sensor 50031	intercept (absolute, single-sensor)	−145.89
	slope (transfers across sensors)	2.0488

Note: The absolute NTU intercept is sensor-specific (calibrated on bench unit 50031). Field deployments use a sensor-relative anomaly form: NTU = max(0, 2.05 × (sps − per-sensor-baseline)), where the baseline is the rolling 10th-percentile of in-water sps for that unit.

Appendix C — Discrepancy log

Appendix D — Sensor inventory & chain of custody

Appendix E — Document version history

Version	Date	Author	Changes
0.1 (Draft)	2026-04-27	Virridy	Initial draft. US-pilot evidence populated for FARs 1, 2, 4.3; Rwanda / Phase-2 sections marked TBD.