| Field | vs the bar | vs raw NBM | Our error ■ vs theirs ■ | Average error | Graded on |
|---|
| Field | Base forecast (L1) | Production | Correction | Field | Base forecast (L1) | Production | Correction |
|---|---|---|---|---|---|---|---|
| Loading current tick… | |||||||
chcl · cmt · h · dp · ws · wg · wd · sr · cccl · cm · cht · h · dp · ws · wg · wd · sr · cct · h · dp · ws · wg · wd · sr · cc · cht · cm · ch · cccl · pr · pph · dp · ws · sr| Field | Total LiftProd vs Public Baseline | Difficulty7d raw MAE ÷ 90d ref · >1 = harder than usual | Pipeline Lift(L1 pick − Prod) / L1 pick | HRRR Pipeline Skill(raw − Prod) / raw | NBM Pipeline Skill(raw − Prod) / raw | Win Ratewins / (wins + losses); ties excluded | Selector Scoreachieved ÷ achievable | n |
|---|---|---|---|---|---|---|---|---|
| loading per_field_scoring.json… | ||||||||
error_l5_nbm (killed 08-25) + error_l6_nbm (scaffold). HRRR-side counterfactual deferred (needs per-field disable logic because HRRR layer slots are field-dependent).windows.24h.per_field block. Same aggregation rules as the 7d table (dp/cc derived-omitted; pa/pp non-MAE excluded from Median/Mean).| Field | Total LiftProd vs Public Baseline | Difficulty7d raw MAE ÷ 90d ref · >1 = harder than usual | Pipeline Lift(L1 pick − Prod) / L1 pick | HRRR Pipeline Skill(raw − Prod) / raw | NBM Pipeline Skill(raw − Prod) / raw | Win Ratewins / (wins + losses); ties excluded | Selector Scoreachieved ÷ achievable | n |
|---|---|---|---|---|---|---|---|---|
| loading per_field_scoring.json… | ||||||||
windows.12h.per_field block. Same aggregation rules as 7d/24h.| Field | Total LiftProd vs Public Baseline | Difficulty7d raw MAE ÷ 90d ref · >1 = harder than usual | Pipeline Lift(L1 pick − Prod) / L1 pick | HRRR Pipeline Skill(raw − Prod) / raw | NBM Pipeline Skill(raw − Prod) / raw | Win Ratewins / (wins + losses); ties excluded | Selector Scoreachieved ÷ achievable | n |
|---|---|---|---|---|---|---|---|---|
| loading per_field_scoring.json… | ||||||||
| Field | Pipeline · HRRR | Pipeline · NBM | Selector (fallback on blended fields) | Status |
|---|---|---|---|---|
t temperature |
L1 → L2 additive | raw_nbm → l2_nbm (l3_nbm dropped 08-25 v0.6.472 — walkforward −2.2% agg; l6_nbm v0.6.453 scaffold ENABLED=False) |
loading… | Served: 3-way blend of corrected HRRR, corrected NBM and raw ECMWF IFS, one weight per lead band, refit daily (v0.7.74 bands, v0.7.75 ECMWF, v0.7.85 ECMWF-timing fix). L2 additive station correction feeds the HRRR side and is worth ~5h. Live 6–11h loses to HRRR 5–15% (coin-flip by day): v0.7.75 HOLD, re-read 11-05. The Selector column is the fallback, not what is served. |
h humidity |
L1 → L2 additive | raw_nbm → l2_nbm (l3_nbm dropped 09-05 v0.6.551 — sentry HOT + 14/15 cells help_fresh negative) |
loading… | Served: 3-way blend (HRRR / NBM / ECMWF), weight per lead band (v0.7.82; ECMWF v0.7.75). The app's h equals the blend (v0.7.72). L3_NBM dropped 09-05 v0.6.551, so the NBM cascade is raw_nbm → l2_nbm. L2 gain is mixed day to day. |
ws wind speed |
L1 → L2 direct (L3 dropped 08-08 v0.6.397) | raw_nbm → l2_nbm (l3_nbm dropped 08-25 v0.6.472 — walkforward −2.5% agg) |
loading… | Served: 3-way blend (HRRR / NBM / ECMWF) per lead band (v0.7.81), replacing the 6–11h terminal blend (dropped v0.7.69) and the frontal-only 24–47h blend. 11-05 gate tests dropping ECMWF. L2 wind blend is useful to ~3h only. wsbp calm-only gate ENABLED=False. |
wg wind gust |
L1 → L2 direct → L3 lead-decay → L4 diurnal (09-24 v0.7.1) → wg residual persistence (dormant) | raw_nbm → l2_nbm → l3_nbm |
loading… | Served: HRRR / NBM blend, one weight (v0.7.71, refit daily). Stack: L3 + L4 with SKIP tables; NBM L3_NBM runs nw_flow/6-11 (v0.7.66). L2 useful to ~3h. wg residual persistence ENABLED=False. |
dp dew point |
L1 → L2 additive → L3 skip-table → dpbp → dp residual persistence (dormant) (derived from corrected t, h via Magnus) | raw_nbm → l2_nbm → l3_nbm (v0.6.440) |
loading… | Derived, not blended: Magnus(blended t, blended h) on every blended hour (v0.7.72). No independent skill chain. dpbp LIVE since 08-04. L2 gain holds out past 9h. |
cc cloud cover (derived) |
L1 → L2 blend → Lc → Ccd = max(cl_l6, cm_l6, ch_l6) (SKIP_REGIMES: se_flow, unknown → Pirate fallback) | raw_nbm → l2_nbm (l3_nbm dropped 09-10 v0.6.577 — sentry HOT + walkforward DROP two-tool; l4_nbm dropped 09-08 v0.6.563) |
loading… | Derived via Ccd = max(cl_l6, cm_l6, ch_l6) except SKIP_REGIMES. Fully removed from NBM cascade layers 09-08 + 09-10. Combine walker tracking random-overlap replacement for the hardcoded max formula. |
ch high cloud |
L1 → L2 blend → L3 lead-decay → L4 diurnal → Lc (07-17) → chp (07-19) | raw_nbm → l2_nbm → l3_nbm → l4_nbm → chp_nbm (l4_nbm v0.6.451, chp_nbm v0.6.454) |
loading… | Best-performing field vs raw. Full stack L1→L2→L3→L4→Lc→chp. chp runs 0–5h only since v0.7.76; its dynamic cell gate refits daily in the cloud. Router drops ne_flow/12-23 (v0.7.45), nw_flow/24-47 (v0.7.68), se_flow/6-11 (v0.7.70). Static _CELL_SKIP still forces some mid/long-lead cells back to L4. |
cl low cloud |
L1 → L2 blend → Lc → clp (dormant; cl in _FIELD_SKIP; cl_l6 = raw) | not extracted (NBM CO does not publish cl) |
HRRR (no NBM cascade) | Off Lc (in _FIELD_SKIP) since 07-30. clp gate wired Stage 3, ENABLED=False. Open: EMA/Kalman shift tracker to un-skip Lc. |
cm mid cloud |
L1 → L2 blend → L3 lead-decay → Lc (L3 dropped 09-15 v0.6.625 — walkforward pooled net-negative; Lc 07-17) | not extracted (NBM CO does not publish cm) |
HRRR (no NBM cascade) | Stable win + Lc live. L3 dropped 09-15 v0.6.625 (walk-forward pooled net-negative). Selector picks HRRR (no NBM cascade). |
pp precip probability |
L1 only (L3 dropped 07-04 — hurts Brier despite MAE gain) | not extracted (POP deferred pending unit audit) |
HRRR (no NBM cascade) | L1 only. All recalibration paths PARKED 08-02 (non-stationary). Reopen path: physical-feature gating (CAPE, RH-500mb, front proximity). |
pr pressure |
L1 → L2 additive (gated on nw_flow/0-5h + 6-11h since 08-10) | not extracted (NBM CO does not publish pressure) |
HRRR (no NBM cascade) | L2 SHIP gated on nw_flow/{0-5h, 6-11h} since 08-10 v0.6.401. |
sr solar radiation |
L1 → Lsr synoptic-regime (skip: ne_flow + calm) → Lsb (sea_breeze cc<25 override) | raw_nbm → l2_nbm (l3_nbm dropped 08-25 v0.6.472; l5_nbm KILLED 08-25 v0.6.471 — sentry +238% MAE + walkforward −126%/−145% agreed; sr has no HRRR L2 so l2_nbm ≈ raw_nbm) |
loading… | Served: HRRR / NBM blend, weight per 4-hour local block (v0.7.73), refit daily. ECMWF solar logged only (v0.7.84); the 3-way was killed on aligned data. Lsr (skip ne_flow + calm) and Lsb (sea_breeze cc<25) stay on the HRRR side; l3_nbm / l5_nbm killed for sr. |
pa precip amount |
L1 only (no L2/L3/L4 apply) | not extracted (APCP deferred pending unit audit) |
HRRR (no NBM cascade) | L1 only. No truth from 2026-10-21 to 11-04 (scoreboard n=0): observed pa read WU only. v0.7.90 adds Tempest (max fresh station rate, x60 to in/hr). pa rows score again as rain days accrue. |
wd wind direction |
L1 → L2 wind_blend → wdp (07-27) | raw_nbm → l2_nbm → l3_nbm (sin/cos) → wdp_nbm (08-19 v0.6.437) |
loading… | L2 wind_blend + wdp both LIVE. NBM cascade full (l2/l3/wdp_nbm). |
mae_over_time.json's last_7d block (with an unweighted-across-leads fallback when that block is empty). It is NOT the top scoreboard's source. The scoreboard's Total Lift / Pipeline Lift / Notable Calls / High-Conf tiles all read per_field_scoring.json's paired-pool numbers (v0.6.492+), which don't touch this fallback. Kept here as a per-field cross-check; retire when the paired-pool numbers are trusted for the same purpose.
mae_over_time.json…
observed for t/h/dp/ws/wg is the station network shrunk toward the model: model_t + K × weighted station bias (K 0.4 / 0.65 / 0.9). The 11-02 audit put it at 40–90% station, the rest model. That flatters any layer that stays close to the model.observed_stations (K=1, stations only) for t, h, dp (v0.7.86) and ws, wg (v0.7.87). The scoreboard tiles above still grade on the shrunk truth; the table below grades the same rows on both so the gap is visible (v0.7.91).observed_stations from lead 2h on; the scoreboard tiles switch once observed_stations has enough history (~11-17 / 11-20 rescore). See L2 below for what L2 is actually worth.analysis/urma_field_score.py. Caveat: NBM is calibrated to URMA, so URMA flatters NBM, most on moisture. Cloud cover: the scored observed is HRRR shrunk toward the KBOS/KBVY METARs (ceilometers blind above 12,000 ft) and sits 13–15 points from URMA, so cc is graded on URMA cloud cover (TCDC, satellite + surface) since v0.7.102: the cc line under the table.analysis/ship_rule.py, 11-02): day-block bootstrap, 95% interval, holdout days = valid day ≥ 2026-11-03 (never tuned on), ≥ 8 holdout days before a verdict; ledger analysis/runlog/ship_tests.jsonl. analysis/holdout_tracker.py runs served vs raw NBM / HRRR by band through it. Every future ship call uses it.runtime_tables/; a table older than 3 days switches the blend off and the older per-cell blend / selector path serves.BLENDER_APPLIED_FIELDS is still empty; the L1b / terminal per-cell paths still apply on cells the field blend does not cover.myweather-collector every 10 min at :07 (4 GB since v0.7.77 memory-leak fix), myweather-nbm-ingest hourly, myweather-refitter 04:30 ET (12 tables: selector, L3/L4_NBM, Lc + recent-bias gate, Lsr bias, sr sea-breeze, chp cell gate, selector regime walker, sr learned GBM, the field blend, the URMA t shadow; 16 GB, one process per table since v0.7.100), myweather-publisher hourly (backstamped pair log + mae_over_time.json), myweather-nbm-backfill (one-shot). The Fitter (L3/L4 tables) runs in the collector at 03:07 and 15:07.~/Developer/myweather, 3.13 venv) since 11-01; the digest runs there. Bucket protection: weekly copy to myweather-data-backup (first run 11-08, verified), monthly permanent snapshot of the backstamped log to archive/ (next 12-06).gpt-oss-120b, fallback gpt-oss-20b (v0.7.48), then template. Gemini is disabled.analysis/urma_shadow_score.py.analysis/refitter_status.py fails the digest when any table is more than a day old.analysis/t_station_truth_wf.py). Flip only if it passes the ship rule.station_readings, then the airport floor, split-network L2 and wg stronger L2 (held, +2.4% on shrunk truth).cluster_spread_tempest_log.json, then point the confidence layer at it.Two lenses on the same object. Applicability (top) describes what's configured to fire and under what gates — built each tick from describe_applicability() in each correction module, so the page can never drift from the code. Runtime firing frequency (bottom, 7-day rolling) shows what's actually firing per operator × field × regime, so silent dormancy (config says "enabled" but the code path never mutates a value — the class of bug that hid ws L3 for 4 days after v0.6.279) surfaces as an ★ on cells with 0 fires despite ≥5 ticks in that regime.
describe_applicability()
| field | applies when | gated by | current state |
|---|---|---|---|
t |
Always — additive bias from station network, scaled by Kalman gain K | always on | applies |
dp |
Always — additive bias from station network | always on | applies |
h |
Always — additive bias scaled by Kalman gain K | always on (K-taper 1.0 → 0.4 by lead 24h) | applies |
pr |
RE-ENABLED 2026-08-10 v0.6.401 with regime gate — K=1 additive, τ=8h fitted, applied on (regime, lead_band) ∈ {(nw_flow, 0-5h), (nw_flow, 6-11h)}; every other cell stays raw. Cell selection from analysis/pr_l2_regime_lead_retro.py 08-10 (Jaccard 0.50 STAGE 1 SHIP, both-halves winners A +21.8%/B +41.6% and A +10.3%/B +13.1% on 8,596 shadow rows). History: DISABLED 2026-07-01 v0.6.276 after pooled Production +2.4% BAD — but that pooled read was regime-blind. The nw_flow short-lead win was real all along; needed regime-conditional cross-cut + halves verification to surface. Shadow-wire since 2026-07-29 v0.6.389 supplied the fresh data. |
regime-gated (current-tick regime × lead band) | applies on nw_flow 0-11h; other cells raw |
cc |
Always — Kalman-blended KBOS + KBVY METAR override on hourly[0] (current-hour value used by cards). Not propagated across forecast leads — promotion to a true per-lead L2 bias correction is queued (see Open architectural questions). |
always on (needs KBOS or KBVY at current hour) | applies |
cl / cm / ch |
Derived — same Kalman K applied to L/M/H splits on hourly[0] to keep them self-consistent with cc. Not propagated across leads, same as cc. |
always on (derives from cc) | applies |
ws / wg |
Always — direct selection from per-octant median (no per-station bias track); KBOS+KBVY authoritative-source floor when both agree at >1.4× the octant median | always on (direct-selection, not additive) | applies (direct) |
sr / pp / pa |
N/A — no station network for this field; L2 row is structurally empty | n/a — no obs network | n/a |
wd |
Always — SHIPPED 2026-07-20 v0.6.368a. Circular unit-vector blend in wind_blend.py: obs wd + fc wd → (sin, cos), weighted mean by linear decay over 24 leads, atan2 back to degrees. Solves the wrap-around that made "N/A — L2's linear math doesn't apply" true prior to this ship. Calm-floor guard WIND_DIR_MIN_SPEED = 3.0 mph skips cells where both obs and fc speed are below floor. raw_wind_direction preserved for baseline. Complementary wd persistence gate (Stage 2 07-20 v0.6.365) targets long-lead regime-transition cells where L2 decay has expired. |
always on (needs both obs+fc speed ≥3 mph) | applies (circular) |
| field | applies when | gated by | current state |
|---|---|---|---|
t / dp / h |
Always — receives HRRR L2's Kalman-scaled additive bias delta | rides on HRRR L2 | applies |
ws / wg |
Always — direct-selection delta from HRRR L2 wind blend rides across | rides on HRRR L2 | applies (direct) |
wd |
Always — circular unit-vector delta from HRRR L2 wind_blend rides across (calm-floor 3 mph inherited) | rides on HRRR L2 | applies (circular) |
cc / ch |
hourly[0] only — inherits HRRR L2 KBOS+KBVY METAR delta on current hour; not propagated across leads (same limitation as HRRR L2 cloud fields) | rides on HRRR L2 | applies on hourly[0]; other leads pass through |
sr |
Passes through raw — HRRR L2 has no sr correction (no station network), so delta is 0. | n/a — HRRR L2 sr is n/a | l2_nbm = raw_nbm |
cl / cm / pp / pa / pr |
Not emitted by NBM CO grib — permanent HRRR-only. Selector never picks NBM for these; no L2_NBM row exists. | n/a | n/a |
Two forecast sources run in parallel for every hour: Open-Meteo HRRR/GFS on one side, direct NBM (National Blend of Models) on the other. Each side runs its own full correction cascade. The 🎯 selector below picks per (field, lead-band) which cascade's output becomes user-visible. Multi-kilometer grid resolution on both — still knows nothing specifically about Wyman Cove; that's what the layers below add.
APPLIED_CELLS live since v0.7.20 · rest shadow · fresh-data since 2026-09-24 16:00 UTC
What our — nearby weather stations say the model is getting wrong right now — and how much of that signal we trust.
What L2 is worth, and what it is graded on · last manual run 2026-11-04 · pair log (backstamped 08-24→11-03 and primary) · scored on the older shrunk observed unless noted
⚠ These are internal pipeline values, not user forecasts. After the aggregate-bias correction is applied but before downstream layers and physical-bounds clamping (FIELD_BOUNDS in decay_apply.py). Values can legitimately fall outside physical ranges here — cloud_cover 121%, precip_probability −6%, precip_amount −0.025 in — because the bias offset is additive and the clamp comes downstream. If any of these look wrong for user display, check the L3 / L4 / cloud-clamp path, not this section.
How the model's error tends to grow with each hour into the future — and the per-field nudge we apply to push back against that drift.
analysis/decay_tau_tuning.py shows ≥5% MAE improvement vs the default. Current overrides: pp (POP) at τ=28d (+11.1% held-out, 2026-06-21 v0.6.167; note: today's read shows only +3.2% vs τ=14, now below the 5% floor — flagged for next τ-audit day). pa (precip amount) reverted 2026-07-19 v0.6.358 after today's read landed +0.9% vs τ=14 (was +5.9% yesterday); pa's best-τ swung 28→42→7 across three IMPLEMENT reads. See [[feedback_tau_streak_gate_limits]].
A separate correction for the part of model error that follows the sun — e.g., bias that's different at 3 AM than at 3 PM. Currently applied to high cloud and wind gust.
{ch, cc} (cc added 2026-06-24 v0.6.214). Fits for fields outside the applicability set are still computed below for diagnostic purposes. See the Applicability map for the live trigger state.
A per-regime W/m² delta applied to direct solar radiation. The classifier reads current wind direction, speed, pressure trend, hour, and temp; the regime label keys into a calibrated per-regime delta. L1–L4 are trained on general bias correction; Lsr is the first layer trained on synoptic-state stratification — different "kinds of weather" get different corrections. Skip regimes (v0.6.280): ne_flow and calm return 0.0 — l5_solar_analysis showed Lsr hurt sr in those regimes (+32% and +10% worse) while helping everywhere else. Skip regimes are a targeted patch until Fix B / refit against L4 baseline lands.
(cc < 25) OR (cc >= 75) after 07-24 halves re-run showed the overcast half (75-100 cc-bin) actively regressed (Δ=−17.6%, halves +6%/−23%) while the clear-sky half held robustly (Δ=+35.8%, halves +33%/+40%). The "thick attenuation missed" hypothesis for overcast was wrong; data says model attenuation is fine or over-corrected. Middle-and-overcast cc bins (≥25) now all fall back to Lsr's original direct-beam correction. Narrowed-shape Stage 2 re-run (07-28) PROMOTES cleanly: pooled +31.50%, halves +43.78% / +22.69% (both above +10% ship gate), 4/4 lead-bands SHIP. Landed ENABLED=False with fresh 7-day live-layer gate 07-28 → 08-04. Broader unit-mismatch across other regimes (pre_frontal +12, unknown +367 W/m²) still open. Shortwave shadow-log infrastructure stays alive (underpins future sr L2 work once unit mismatch resolves per project_sr_unit_mismatch). Divergence-report LSR_ENABLED source bug fixed 07-20 v0.6.365 (was reading l5_solar_analysis, now routes through .cache_l5_gate_history.json) — status now correctly AGREE, live gate 100% SHIP for 30+ days.ENABLED=True after the Lsr gate cleared 7/7 ship days (12-cycle SHIP streak).direct_radiation_post_l4 before mutation and adding sr_l5 to the snapshot writer._layersFor() filters the Lsr entry to the sr card only.compute_solar_correction. Per today's l5_solar_analysis: overall Lsr net −7.5% MAE across 8 regimes, but only 3 clear the +3% improvement threshold (sea_breeze, nw_flow, se_flow) — verdict HOLD gates further refinement, not the shipped layer.raw_direct_radiation was being captured AFTER Lsr mutated direct_radiation, so the "raw" baseline in the debug page + Production accumulator was itself Lsr-corrected on all post-06-28 pair rows. Fixed by extracting raw preservation into preserve_raw_forecast_arrays() and calling it BEFORE stamp_solar_correction. Structural guard added v0.6.291 (2026-07-04): weather_collector/processors/raw_integrity.py deep-copies every raw-counterpart hourly array at pipeline start and asserts equality at pipeline end. Any future layer that mutates a source array before its raw_* copy exists will now fire a drift event on the next tick and land in the daily digest, instead of hiding for a week.stamp_solar_correction was computing a single delta from the CURRENT tick's raw solar + hour, then applying that scalar to all 48 forecast leads. At pre-dawn / dawn ticks the current raw was below the 50 W/m² sun-up threshold, delta = 0, no Lsr applied to any lead. Fixed by iterating over the forecast array with each lead's own raw value + parsed local hour. Lsr now fires at every daytime lead in every non-skip regime.L5_VALID_FROM = 2026-06-28T07:05, but the two collector bugs above mean the pair-log record of Lsr in the wild before 07-03 is only a sliver of what Lsr will do post-fix. First actually clean 7-day audit window closes ~2026-07-10. Today's simulate_windows HOLD verdict + today's l5_solar_analysis HOLD verdict both read against the corrupted window — treat as informational until the 07-10 re-run.A per-(field, value_bin) percentage-point shift applied to the L4-corrected forecast for cc / cl / cm / ch. The bias pattern Lc unwinds is saturation: the models systematically overshoot at 0-5% cloud and undershoot at high fractions. The same shape holds across all four cloud fields but with different magnitudes. Lc reads each L4-corrected value, keys into a curated (field, value_bin) → shift table where the fitter's SHIP verdict is set, adds the shift, and clamps to [0, 100]. Emergency intervention 2026-07-30 v0.6.389d-g — pooled shift table diverged from recent regime-conditional truth, causing 8× cl MAE blow-up on 07-30. Live SHIP surface after intervention: cl fully off (_FIELD_SKIP); cc ships at 0-5 (all regimes), 50-80 + 80-95 (all regimes except ne_flow), 95-100 KILLED universally; cm unchanged (20-50 / 50-80 / 80-95 / 95-100, all regimes); ch unchanged (20-50 / 50-80 / 80-95, all regimes). SKIP cells (bins that stayed within the ±5 pp noise floor across the 30-day window) pass through unchanged. See Engineering status below for the walk-forward evidence that motivated the interventions.
_FIELD_SKIP or _CELL_SKIP frozenset in cloud_saturation_correction.py. See [[project_lc_regime_conditional]] for the full pipeline state.
LC_ENABLED READY on divergence report, and no cc/cl/cm/ch ANOMALY in the pair-log anomaly detector. First live tick (15:27 EDT): 113 cells fired — cc 46/48, ch 39/48, cm 19/48, cl 9/48; mean |Δ| in the expected 28-42 pp range per field. Predicted biggest Prod MAE lifts: cl 80-95 −55%, cl 95-100 −47%, ch 50-80 −37%. Two-gates-per-layer cross-check via [[feedback_two_gates_per_layer]]. 07-31 clean close superseded by the 07-30 intervention above.
weather_collector/processors/cloud_saturation_correction.py at ENABLED=False. Fit script analysis/lc_fit.py emits lc_correction_table.json with per-(field, value_bin) shift + SHIP/SKIP verdict + n_samples.lc_fit's own SHIP-set stability gate. Cross-check both before flipping. Applied here: divergence gate cleared 07-10 (7/7) while lc_fit gate remained False through 07-16 because cm 20-50 flipped into the SHIP set 07-11 (inside the 7-day stability window).lc_fit gate True (07-10 rolled out of the 7-day window today) + 16 SHIP cells identical 7 consecutive days (07-11 → 07-17) + no cc/cl/cm/ch ANOMALY. Deploy sequence: line 29 flip → make deploy-collector → :07 tick verified → GCS confirmed → frontend push. 14-day post-ship watch begins.Two lenses on the same question. First (Accuracy over time, below): the per-obs-day trajectory of Raw vs Production — is the stack drifting? Did a recent ship move the needle? Second (per-field tables, further down): the current-window breakdown by lead band per field — where in the 0-47h horizon is each correction layer helping or hurting? The over-time view is the drift detector; the per-band tables are the shipping-decision granularity. Production in both views IS a real per-row aggregate. Per-band tables: per_layer_mae_by_lead[field].production where the per-lead sample count clears n≥30, keyed on applied_layer stamps. Over-time chart: mae_over_time.json's "prod_real" series, bucketed per obs-day from the same applied_layer stamps (v0.6.371). Both surface the same "what users actually saw" quantity, just at different aggregation grains. Specialist lines (Lsr, Lc, chp, clp) remain visible on the over-time chart as intermediate series showing gate-fired-only MAE — smaller sample than Raw/L2/L3/Prod, useful for isolating each specialist's per-lead lift during 14-day watches.
Core comparison: pipeline error vs. raw L1 error on the same forecast-observation pairs. L1 = HRRR-side (Open-Meteo HRRR/GFS) baseline. NBM cascade runs in parallel. What the user sees is a field-level blend of the two corrected cascades for t / h / ws / wg / sr (t, h, ws with raw ECMWF as a third input; dp derived from t and h), and the 🎯 L1 selector's per-(field, lead-band) pick for the rest. Scoreboard baseline = the normal source: NBM where NBM publishes the field, HRRR only where it does not (never always-HRRR on an NBM field). Every metric on this page averages those errors differently.
Truth caveat (11-02 audit, 11-04): the scored observed for t/h/dp/ws/wg is shrunk toward the model, and L2 is graded on the stations that feed it (lead 0–1h is a tautology). observed_stations is the model-free column; grade L2 and the short-lead layers on it from lead 2h on once it has history. Ship calls go through analysis/ship_rule.py (day-block bootstrap, holdout days ≥ 11-03).
"Observed" per field (best local measurement, not absolute truth):
model_t + K × weighted bias, so the scored value is partly the model (11-02 truth audit: 40–90% station). From 2026-11-02 the pair log also carries observed_stations (K=1, stations only) for t, h, dp, ws, wg; use it to grade layers that use the stations (L2) from lead 2h on.daily_extremes.py reads wu_stations for this; pa had no truth from the WU outage (2026-10-21) until v0.7.90, which reads Tempest too. The observed current-hour precip (the pp truth) also fell back to the model's own forecast while WU was dark; v0.7.90 uses the Tempest network first (same wet rule as the app: at least 3 wet stations and 25% of those reporting).Three metrics per NWS/ECMWF conventions:
pp uses Brier score, not MAE (binary obs vs. probability forecast). Brier = mean of (fc/100 − obs/100)². Same "% erased" framing.
Measurement gaps:
h_persistence_skill.py). Baseline: 5 ADD VALUE (t/h/pr/ws/sr), 4 MIXED (dp/wg/cc/pp), 3 NO SKILL — cl/cm/ch lose to persistence at every band despite L3+L4. Scorecard integration shipped 07-12 v0.6.328 — the "vs Persistence" line in the scorecard banner (above) is live-updated from persistence_skill.json. Response for ch: ch persistence gate FLIPPED LIVE 07-19 v0.6.358 (14 SHIP cells post-emergency-demote 07-27 v0.6.382t (was 27) on refreshed windows; 14-day watch CLOSED CLEAN 08-02). Landmark answered 07-14 v0.6.351a: persist_only ties gate pooled but gate wins halves-hedge; keep the gate. cl persistence gate wired Stage 3 07-24 v0.6.379 (replaces retired cl_persistence_short_lead — narrow 0-5h hypothesis disproven by halves-verified Stage 2). 08-09 Stage 2 rerun — 6 SHIP / 22 SKIP whitelist shape ready for flip decision. SHIP cells: calm/0-5 (−35.7%), ne_flow/0-5 (−54.4%), ne_flow/12-23 (−16.6%), nw_flow/0-5 (−18.7%), pre_frontal/0-5 (−17.8%), sea_breeze/0-5 (−32.9%). Flip = cl_persistence_gate.ENABLED=True. 08-09 c1_stage4_mixture_check: cl/6-11h [transition] b3 no longer in DEGRADED list (07-29 escalation window CLOSED CLEAN, never reached 3× ≥+200%); cm/12-23h [transition] IMPROVED; t/24-47h [transition] newly DEGRADED (unrelated). Joint cl/cm correction hypothesis rejected 07-29 (cl has no correction stack; cm corrections are stable improvers; drift is raw-HRRR-side on top-fc transition rows and fitter absorbs via c1 confidence bands).Historical wording caveat: section text written pre-v0.6.325 (2026-07-10) treats MAE as the only measure. Read alongside RMSE + bias — disagreements usually mean occasional big misses or systematic drift.
The bright white "Production" column is the forecast users actually see. Lower is better. For each field, one combined table shows how far off the forecast tends to be at each lead band — MAE (typical error), RMSE (occasional big misses that MAE averages away), bias (systematic drift, signed) — per correction layer. Compare a MAE row across columns to see where each layer helps or hurts.
cluster_spread_log.json) has no entries after 10-21, so the live axis is silent. v0.7.90 logs a Tempest-cluster spread (west / central / east by longitude) to cluster_spread_tempest_log.json and stamps cluster_spread_tempest. It is a different measure, so the confidence layer does NOT read it: its Q1/Q23/Q4 cuts must be refit on 14+ days of the new log (earliest ~11-19) before the axis comes back.frontal_events_log.json).L4_FIELDS = {ch} at the time (now {ch, wg}: wg added 2026-09-24 v0.7.1) — cc dropped 2026-08-28 v0.6.515 after 7-day drop-gate cleared (cc is Ccd-overwritten downstream; L4 cc was code-hygiene deadweight).h_asymmetric_l3.pyh_run_time_bias.pytype='cold' classifications since deployment because _classify_type's cold branch required dp_drop ≥ 8.0°F — threshold sat above observed p99.9 (6.3°F, max 6.8°F). DP_DROP_THRESHOLD lowered 8.0→4.0°F; cold branch now reachable. Warm-front separation still requires the reverse dp signature (dp rising) which no signal captures yet. First live type='cold' tag expected within a few days. Re-test h_front_type.py once the events log accumulates a mix of cold/sea_breeze/unknown.h_precip_obs.pyh_cloud_composition.pyh_ws_wd_error.pystation_history.json, not forecast_error_log.jsonl). Re-test after adding lightning_proximity_km to pair-log rows. h_lightning_proximity.pyThings we built, ran, and stopped running. Two kinds live here: hypotheses the data answered no, and settled tunings where the sweep concluded "current value is fine." Kept as institutional memory. Scripts in analysis/ can be re-run if conditions shift. Retired layers (code intact but permanently no-op) stay in R&D — see Lt above.
myweather-refitter at 04:30 ET refits 10 live tables — selector, L3/L4_NBM, Lc fit + recent-bias gate, Lsr bias, sr sea-breeze, chp cell gate, selector regime walker, sr learned GBM — guards each one and publishes to runtime_tables/. The collector loads GCS copy → last good → bundled. No digest or deploy needed. Ship decisions (skip tables, APPLIED_CELLS, LIVE_DEMOTED, the frozen chp/wdp/wg-residual/wg-L3-asymmetric cell sets) stay in the repo and change only through a ship.ne_flow/12-23 + se_flow/12-23 as of the 10-09 13:44Z refit (nw_flow/12-23 and /24-47 held out by LIVE_DEMOTED; pre_frontal/24-47 fell out of that fit — the cell set changes with each daily refit) · L1 static blender (L1b) applied on h/nw_flow/24-47 + h/sw_flow/24-47, dp still shadow. All three look cells up by the per-hour pre-swap regime, recorded in the pair log as state_fc.regime_runtime since v0.7.35.{ch} · cc dropped 2026-09-08 v0.6.563 (30d marginal lift +0.79% pooled, best skip-table only +1.38% — not worth the maintenance)regime == "sea_breeze" AND cc < 25 · 14-day watch CLOSED CLEAN 2026-08-19 (+11.4% pooled, n=15,985)raw_nbm for 9 fields; native L2 for all 8 NBM-scope fields (t/h/dp additive Kalman+decay, ws/wg/wd wind_blend, cc/ch cloud_obs_blend hourly[0]; sr identity — no L2 either side); l3_nbm for wg/ch/sr (scope trimmed 08-25 v0.6.472; sr re-added 09-04 v0.6.549 then per-cell skipped 09-29 v0.7.11 on nor_easter/12-23h + /24-47h; h dropped 09-05 v0.6.551; cm dropped 09-15 v0.6.625; cc dropped 09-10 v0.6.577); l4_nbm for ch only (cc dropped 09-08 v0.6.563); l5_nbm for sr KILLED 08-25 v0.6.471; l6_nbm for t scaffold (ENABLED=False); chp_nbm for ch; wdp_nbm for wd. · Field-level pick distribution changes daily — see the per-field Selector column above and the National Source tile for real-time per-band picks. The static summary sentence that used to live here drifted from reality within days of every ship. · skip_table_nbm_curated.json 22 cells post-v0.7.24, counted from the file (v0.7.17 removed one, v0.7.24 added three: wd se_flow/24-47 and TEMPORARY nor_easter/12-23, /24-47; the older "17 post-v0.7.11" figure was not recounted after v0.7.17). Symmetric ADD + REMOVE audit loop runs daily in the digest.max(cl_l6, cm_l6, ch_l6) except SKIP_REGIMES {se_flow, unknown}_CELL_SKIP · FLIPPED 07-19 v0.6.358calm/12-23 (cell set frozen in the repo since v0.7.30) · FLIPPED 2026-07-27 v0.6.382 · 10-09: still −18.0% vs L1 on all calm/12-23 rows, but its older half now loses on firing rows — see UpcomingBLENDER_APPLIED_FIELDS = frozenset() · curated table stale-window fit · re-curate needed · sibling: v0.7.6 L1 static blender (2026-09-26, separate mechanism — universal ω per field, L1 seat, cascade BYPASS, curated for h/dp only). PARTIALLY LIVE since v0.7.20 (2026-10-02) — the v0.7.3 blender above is still fully shadow; v0.7.6 now applies on two cells (h/nw_flow/24-47 since v0.7.20; h/sw_flow/24-47 since v0.7.23, 10-03). v0.7.6 10-02 shadow retro: 9 SHIP-READY / 10 HOLD / 1 KILL / 0 THIN on 20 curated cells (was 2/13/7/7 on 09-29 — the pre_frontal KILLs collapsed to HOLD; only dp/pre_frontal/12-23 still KILLs). Shipped: h/nw_flow/24-47 on 10-02 (n=436, +35.5%, halves 38.7/33.9 — tightest spread of the nine) and h/sw_flow/24-47 on 10-03 (n=830, +39.0%, halves 41.8/35.7). Later reads: 10-03 retro 9 SHIP-READY / 11 HOLD / 0 KILL / 0 THIN; l1_static_blend_shadow_verify curated cells 14/6/0/0 on 10-04 and 16/4/0/0 on 10-05. Held: dp/nw_flow/24-47 cleared n and lift (+17.5%) but halves 7.7/42.4 too wide to flip same-day. Mechanism is ENABLED=True + per-cell APPLIED_CELLS allowlist via is_applied(); the curated table was deliberately NOT stripped, so all 20 cells keep stamping shadow telemetry. Reversal = APPLIED_CELLS = {}. nor_easter remains off-curated (best-ω HRRR-favoring vs universal 0.27/0.44) — a curation gap, not a plumbing bug.forecast = ω · HRRR_terminal + (1−ω) · NBM_terminal, per (field, regime, band) cell. Fields in BLENDER_APPLIED_FIELDS (currently empty) apply the blend to the served forecast; every other field gets shadow telemetry only. Un-curated cells fall through to the selector unchanged.forecast_error_log_backstamped.jsonl had been frozen since 2026-08-21 (uploaded manually once, then no auto-refresh). l1_blender_stage1.py reads that URL via cached_path(); every fit since Aug 21 saw data ending 2026-08-20T20:07. The 13 STABLE cells shipped v0.7.0 and today's v0.7.2 dp APPLY flip were both fit on that frozen window. Fresh-data refit results: 11 of 13 cells fail halves-stable. Survivors: h/pre_frontal/24-47 (A+25.7% / B+8.1%, ω̄ 0.44-0.55) and t/se_flow/24-47 (A+11.8% / B+14.9%, ω̄ 0.37-0.46). One new candidate not in the shipped set: wg/pre_frontal/12-23. v0.7.3 (this session) reverts BLENDER_APPLIED_FIELDS to empty and wires analysis/nbm_backstamp_append.py into the publisher CF (runs first each hour, appends new pair-log bytes to the GCS backstamped file via bucket.compose(), HWM tracked in gs://myweather-data/backstamp_hwm.json) so this staleness class can't recur silently. l1_blender_curated.json intentionally not re-curated in v0.7.3 — since applied-fields is empty, no cell fires and shadow telemetry now accumulates against fresh data. Re-curation is a follow-up ship. Related audit: per-obs classifier v2 (v0.6.644-646) fit on the same stale window; fresh-data refit shows STAGE 1 HOLD on both h and t with every cell marked degenerate (fNBM 67-99% — the fit rubber-stamps NBM rather than selecting per-obs).raw_nbm → l2_nbm → l3_nbm → l4_nbm (ch) → l5_nbm (sr) → l6_nbm (t, ENABLED=False) → chp_nbm (ch) / wdp_nbm (wd). Each layer has its own fitter (analysis/l{3-6}_nbm_*.py) writing a curated JSON that the collector loads at import; per-hour stamps land in the snapshot log and per-row error_l{n}_nbm columns land in the pair log. L1 selector picks argmin(MAE(HRRR-deepest-Prod), MAE(NBM-deepest-Prod)) per (field, lead-band). Bins reached L3 maturity via analysis/nbm_backstamp.py (walks pair log + backfilled NBM point extracts, reconstructs l2_nbm exactly from snapshotted HRRR values); deeper layers warm naturally over the next ~30 days as error_l3_nbm/error_l4_nbm/error_l5_nbm rows accumulate. See per-layer fit-status tiles below.confidence.cells[field][band].c1h. Per-cell co-axis ortho gate (v0.6.321) suppresses cells non-orthogonal to the currently-firing co-axis — cl fires freely, ch 24-47h + t × 3 never fire.nor_easter +3.79°F ★ but n=279 — magnitude passes floor, sample thin. Gate: 3 consecutive reads with n growing.confidence.cells[field][band].c1d. Fires when live cloud_inter_source_sigma ≥ Q3 (44.55). cc 24-47h MARGINAL NARROW −7.06% is a documented outlier — flagged for Stage 4 review.weather_collector/data/pre_frontal_curated.json. v0.6.372b matched-regime fix HOLDS PROMOTE with the corrected baseline; SHIP set expanded 5→7. Digest currently at streak 1/7 due to cell-set drift + population still THIN.cl_persistence_short_lead (all-9-regimes design gate not met). Broader regime × lead_band gate: 12 SHIP / 8 MARGIN / 16 SKIP / 1 THIN. Whitelist: persistence on calm/se_flow/unknown all-leads + nw_flow 24-47h + short-lead in sw_flow/ne_flow/nw_flow/pre_frontal + pre_frontal 6-11h. Wired after Lc. 7-day flip gate: SHIP cell-set Jaccard ≥ 0.8 across 7 daily reads. See [[project_cl_persistence_investigation]].fc_l2, replaces L3-corrected wg on gate-fired cells. Short leads always SKIP (L2's Kalman handles close-in). 07-27 flip HELD — h_wg_residual_persistence_stage1 flipped PROMOTE → MARGINAL 07-26 (+20.24%, was +17.74%). Persistence hypothesis's aggregate weakened; hold until Stage 1 recovers PROMOTE. (Divergence report drop-{wg,ws} was stale-tool artifact per [[feedback_walkforward_skip_table_blind]]; fix v0.6.381 landed 07-26.)fc_l2 + fitted L2-residual). Stage 2 preview: 8 SHIP / 2 MARGIN / 26 SKIP / 1 THIN — SHIP cluster long-lead only (frontal 12-47h, nw_flow 24-47h, pre_frontal 12-47h, sw_flow 6-47h); zero SHIP at 0-5h. Best cells sw_flow 24-47h (−20.4%), sw_flow 12-23h (−18.6%). Clamp |Δ| > 10°F. Same Jaccard ≥ 0.8 streak gate as wg..skip.py on 08-10 with the daily-digest retirement pass (10 dead-verdict scripts total). Structural next step if pp ever reopens: physical-feature gating (CAPE, RH-500mb, front proximity), not calibration parameter fitting.LC_RECENT_BIAS_GATE_ENABLED = True live in cloud_saturation_correction.py. Applies existing lc_correction_table.json shift only where recent 3-day observed bias still agrees with the historical fit (sign match + magnitude ≥ 0.5×|hist|). Smaller surface than the EMA/Kalman rewrite alternative — no new lookup table, just a per-cell gate on the live shift. 08-18 refresh (v0.6.430): ch per-field streak now 11 days; cl and cm remain CHURN (in and out of promoted set day-over-day). [10-07: both tables now refit daily in the cloud refitter (v0.7.30) — no deploy. fields_cleared has been [] since at least 09-13 and is still [] on the first cloud fit, so the gate currently suppresses nothing and Lc applies its historical shift on every cell. The 08-18 text below is as of 08-18.] Live cells suppressed today = 0/3 — all three ch per-bin cells (20-50, 50-80, 80-95) have gate_apply=True because recent 3d bias tracks historical (or is thin) on every bin. cc excluded (derived via Ccd). Ship is a live no-op today but the self-healing mechanism is in place for any future per-bin drift — the anti-scar-tissue counterpart to hand-curated skip frozensets. Runtime telemetry verified 10:48 UTC tick: weather_data["cloud_saturation_correction"]["recent_bias_gate"] = {enabled: true, fields_cleared: ["ch"]}. See [[project_lc_regime_conditional]]. Follow-on: cl remains in _FIELD_SKIP (Lc architecture can't rescue cl regime) — needs EMA/Kalman fallback, separate workstream [[project_lc_cl_unskip_investigation]].weather_collector/data/ch_persistence_gate_curated_vs_l6.json. L6-baseline Stage 2 rebuild (h_ch_persistence_blend_stage2_vs_l6.py) says chp's honest SHIP set is 5 cells: calm/0-5 (−70%), pre_frontal/0-5 (−41%), sw_flow/0-5 (−30%), se_flow/0-5 (−26%), ne_flow/6-11 (−17%). Currently-live chp is at 14 SHIP + 8 MARGIN post-demote; 5 more halves-disagreement/small-magnitude cells still fire but should also fall away. 7-day live-layer change gate closed 08-03; verdict tracked continuously via auto-rolling windows (v0.6.398, 08-09). Adopt condition: 7 daily reads of the L6-baseline Stage 2 continue to agree on the 5-cell SHIP set + no new frontal-only-recalibration interaction.sr/ne_flow/12-23 + sr/se_flow/12-23. pre_frontal/24-47 dropped: its A half held out at +9.6% (07:37 data: +24.0%), failing the held-out ≥ ½ train-lift check — fit-to-fit churn, but label change vs newer data not yet separated (rerun old vs new v5 on the same file). After a few ne_flow days, paired read on ne_flow/12-23 (served vs error_l5_nbm on its learned rows, keyed by regime_runtime); demote via LIVE_DEMOTED if negative. Also still open: the v0.7.33 effect (sr rows in runtime nw_flow/12-23 stamp band_pool) — no daytime nw_flow since the deploy as of 10-09.regime_runtime, one ship each with a before/after read: selector by-regime walker; L1 blender fitter + l1_static_blend_shadow_verify (its "calm" bucket and 16 off-curated cells are this label mismatch); wdp. Use analysis/_cache.RegimeRuntime (stamped label, else rebuilt from the run's pre-swap HRRR wd/ws/t/cc). Same class, not yet scoped: the learned classifier's wind/cloud/solar features — the fitter reads post-swap state_fc, the runtime reads the raw hourly arrays (forecast_snapshot.py _build_learned_features).calm/12-23 — the only live wdp cell. 10-09 h_wd_persistence_gate_stage2 went HOLD: on all calm/12-23 rows the gate still beats L1 by 18.0% (n=429), but on firing rows the 09-09..09-24 half now loses (+13.7%). The cell set is a frozen ship decision, so nothing changes on its own. Pull the cell (a ship) if the all-rows lift goes negative.scripts/ch_v075_live_verdict.py once the 09-30..10-02 event (51% of the rows) has aged out. Pull any cell still losing to always-HRRR at n≥150 — ne_flow/12-23 and the three se_flow cells are the candidates; a cell removal is a ship. Rollback = flip IMS_SELECTOR_SHADOW_ENABLED.selector_source, the loss is entirely on NBM-picked rows (h 12-23h: served 6.26 vs HRRR-L2 2.43, n=531; t 6-11h: 1.81 vs 1.01, n=195). Replaying the fit rule each morning shows every NBM period in both cells was the 7d override (t: 30d says HRRR every day; h: 30d lift +0.5..+3.1%, under the 3% bar). The selector table used to change only on deploy; since v0.7.28 it refits daily in the cloud. New digest tool l1_selector_override_walkforward: first read all FLAT (override −1.3% vs 30d-only, halves −0.2/−3.5; daily refresh +0.6% vs deploy-frozen). No selector rule change until ~7 consistent daily reads; any change must run in the cloud. h's alert should fade as 09-29..10-01 leaves the 7d window (~10-09), t's ~10-10. [10-09: h's alert is gone; only t/production@12-23h remains (+12.2% vs raw), the tail of the stale bundled override (runs 10-05 09Z..10-07 08Z), due to roll off ~10-10. Override walkforward 3 reads, all FLAT (−1.3%, −1.1%, −0.6%). The 10-09 cloud refit moved h 0-5 / 6-11 / 24-47 and dp 6-11 from NBM back to HRRR.]h/calm/24-47 62, h/nw_flow/24-47 24, h/ne_flow/24-47 1) — the stamp is landing. The 62 under "calm" are probably v0.7.23's sw_flow cell, bucketed by the pair-log regime label (l1_static_blend_shadow_verify.py keys on state_fc, not the runtime regime) — inferred, not confirmed. Still unobserved: (a) the stamp's value on a real applied row (expect nbm); (b) the user-visible writeback — hourly.corrected_humidity at an applied lead must equal the stamped blend, not L4; (c) the verifier's per-cell lift for the two applied cells (cut from the digest tail). [10-09: not re-checked. The "calm" bucket is the pair-log regime-label mismatch found 10-08; the verifier moves onto regime_runtime in the migration above.]dp/nw_flow/24-47 is still split: halves +4.5% / +39.9% (n=456 7d) — leave shadow. dp/nw_flow/12-23 now reads SHIP-READY with tight halves (+53.0% / +49.9%, n=357 7d, 507 30d) — below the 400-row apply gate on 7d, and the verifier still buckets by the old regime label. Re-read after the verifier moves onto regime_runtime; extend APPLIED_CELLS only if it holds.l1_static_blend_curated.json to support per-regime ω override (nor_easter isn't compatible with the field's universal ω=0.44). Also revisit dp/nor_easter (best-ω=1.00, halves unstable). If no cells clear, hold and revisit when regime accumulates more sample.h_ch_persistence_blend_stage2_vs_l6 (WATCH, 7 losing cells on 10-09) does not read the cloud gate, so its count overstates the live problem. The gate itself turned off 12 cells on 10-09. Live cells losing that day but not yet 7/7: pre_frontal/6-11 (6/7), calm/6-11 and ne_flow/12-23 (5/7) — the gate removes them on its own at 7/7. Open: teach the tool the cloud gate so its WATCH counts only live cells.se_flow/24-47 CONFIRMED 10-07; nor_easter cells untested. Verified through gate_firing_log (the pair log records the l3_nbm counterfactual regardless of the skip, so it cannot show this): the only post-deploy se_flow tick (10-07 08:58) shows wd L3_NBM fires 0 / skips 46 (pre-deploy 10-03 18:58: 22/24). No nor_easter tick since the deploy; those two cells reach the 10-13 re-review unexercised unless one occurs.wd × nor_easter × 12-23h / 24-47h (v0.7.24, single-event evidence: 50d adds 42 and 2 rows over 14d) and sr × nor_easter × 12-23h / 24-47h (v0.7.11). Keep as formal skips if the regime recurs and the audit confirms on independent rows; remove if it faded. Reversal = delete the entries in skip_table_nbm_curated.json.walkforward_l3l4_validator now proposes adding sr (L4_FIELDS = {sr, ch, wg}, 2/7 on 10-09); dp and h dropped out of the proposal 10-07. Needs 7/7 plus a second tool-group before it is a ship.l2_nbm on 86% / 67% of 0-5h / 6-11h rows, overwriting Ccd; where the applied layer is l6, served == derived-max exactly, so Ccd works. The gain sits in rows where raw cc ≥ 90: derived-max beats served by 35–52% there (0-5h 21.0 vs 10.1), while on non-saturated rows it is a wash (worse at 24-47h). The 08-04 saturation guard (SAT_THRESHOLD=90, raw ≥ 90 keeps raw) was justified on rows selected by observed cc 95–100, which conditions on the outcome; the guard acts on the forecast. The sentry's "worse than raw" is a saturated-overcast lucky-baseline artifact: during the 09-26..09-29 nor'easter raw cc = observed = 100 (every row had raw ≥ 90 on 09-27/28; raw MAE 0.0), so any other value reads as hundreds of percent worse. 🟢 Auto-wired Stage 0 (analysis/h_cc_sat_guard_stage0.py; runs in the digest once committed). First real read 10-05: PROMOTE — 10,923 saturated quads, derived beats served by +52.1% / +57.6% / +43.9% / +35.2% (0-5h → 24-47h), 20 of 24 regime×band cells STABLE (unstable: ne_flow 12-23 and 24-47, se_flow 12-23, sw_flow 24-47). Raw ≥ 90 splits into two halves: observed ≥ 95 (raw 0.08, served 12.01, derived 3.29) and observed < 95 (raw 61.73, served 34.96, derived 23.87) — raw is perfect on true overcast and terrible otherwise; derived hedges and beats served on both. Caution: the edge is concentrated in 09-21..10-01; the newest four days net out flat (served 25.02 vs derived 25.01, n=1,470) and derived lost badly on 09-24 and 10-03 — the 7-day read matters. Needs: its 7-day read, then a selector refit with the derived value as a candidate source, and the KNOWN_LIVE_PIPELINES label that suppresses h_cc_derivation's PROMOTE corrected. Also open: Lc l6 vs ch in se_flow (l4 ≈10.5 → l6 14.7–18.7). [10-06, day 2: PROMOTE holds — +42.5% overall (was +42.0%), 20/24 cells still STABLE, same unstable set. Two consecutive PROMOTE days toward the 7-day gate.] [10-08 day 3: PROMOTE +47.1%, last 4 days +0.7%.] [10-09 day 4: the overall read is still PROMOTE +44.2% (17/23 STABLE), but the edge has reversed — last 4 days −29.4% (n=432), last 7 days −7.3% (n=1,741). The overall figure is carried by the 09-26..10-02 overcast days. Do not advance to SAT_THRESHOLD / a selector refit; keep reading.]nw_flow/24-47 cell was held on a wide halves spread, so dp is still entirely shadow and this workstream does NOT close yet. Revisit when dp gets its first applied cell (see the ~10-09 row above); if dp never clears, re-scope NWS coherence as the fallback.ws/calm/12-23, ws/nw_flow/12-23). Under the fixed gate (sum(n_today) ≥ 60 across 3d), three PPP cells sit below threshold and will clear as sum_dn accumulates: wg/sw_flow/0-5 (sum_dn=32, lift +7.4%), ws/calm/0-5 (sum_dn=6, lift +21.4%), ws/calm/6-11 (sum_dn=2, lift +28.7%). Digest watch — no action.stagnant_high × t × 0-5 cell clears the wire naturally. Once the walker routes t at that regime/band, the named gate becomes redundant. Remove HRRR_PBL_MORNING_OVERSHOOT_* block from l1_selector.py; drop the hour_local kwarg call from forecast_snapshot.py. Was "4-7d" (09-13); regime rarely active recently.confidence_layer.ENABLED = False — no C1d band is displayed to users (calibration audit HOLD, 13% pass vs 75%). The cc/0-5h sample-floor watch was closed with no ship: c1d_calibration uses a rolling 14-day window so n_low plateaus (832) and never reaches the 1000 floor, and the premium swung +145% → +762% → +464% over 21 windows with halves that disagree. Revisit only with a confidence-layer enable plan.ch 24-47h, cl 24-47h). Only 3 frontal passages in the 45d window; most cells THIN. Re-run when autumn front cadence returns.h_frontal_t_bias_stage0.py mirroring l6_fix_b_refit: requires ≥7 distinct days, no HOLD days, ship_bands STABLE, ≥1 band. Verdict escalates to STAGE 1 CLEAR — the trigger to scope frontal_t_bias.py specialist. Earliest clear 2026-08-19.regime_runtime (opened 10-09). v5 (sr) and v2 (t) bucket rows by the regime the runtime looks cells up by, via _cache.RegimeRuntime (rebuild = stamped label on 130/130 routed rows). Before/after on the same data: sr cells pre_frontal/24-47 + se_flow/12-23 → those plus ne_flow/12-23; t HOLD both ways. Watch: live from the 10-09 13:44Z refit (rev 00005-qez): cells ne_flow/12-23 + se_flow/12-23; pre_frontal/24-47 fell out of that fit (see Upcoming). Then the new cell's paired read. Reversal = revert the two fitters.regime_runtime in the pair log (opened 10-08). The collector stamps the per-hour pre-swap regime; the pair log carries it from obs 10-08 19:07 (1,820 rows by 10-09 07Z). Watch: rows joined to pre-deploy forecasts still lack it; full coverage by ~10-10 20Z (48h lead). No served change.L1b registered via describe_applicability(); Joe confirmed it on the page 10-09 (v0.7.35a put the map in cascade order and moved the blender status tile to the L1 section). Closes on the next sweep.nw_flow/12-23 demoted (opened 10-08). Added to LIVE_DEMOTED (paired −44.2%, n=207; −41.8% re-keyed by runtime regime). Watch: sr rows in runtime nw_flow/12-23 stamp selector_mechanism = band_pool — not yet observed (no daytime nw_flow since deploy, as of 10-09).ENABLED AND the latest refit verdict is PROMOTE (the 08-05 flip gate). The current fit is HOLD (0 hourly cells, pooled test −0.66%), so it is off; would-have-fired leads are logged as skips in gate_firing_log, and it re-arms on its own when a daily refit says PROMOTE. Before this the processor ignored the verdict and applied the overall +87 W/m² at every clear sea_breeze hour. Stamp: sr_sea_breeze_correction.verdict. The other 9 cloud tables were checked: their consumers already honor their own verdict/gate fields. Watch: sea_breeze ticks show applied: false while HOLD; it re-arms on the first PROMOTE fit.myweather-refitter (rev 00003-fid, 8 GB, daily 04:30 ET) owns 10 live tables; collector rev 00613-jut reads them through runtime_tables.get(). First full cloud run 17:19Z, all 10 published: selector 307,671 rows, 0 pick changes; L3_NBM 274,544 pairs; L4_NBM 25,788; Lc 11 SHIP cells; Lsr 8 regimes; chp gate 6 cells off; regime walker clears hrrr 6 / nws 2 / nbm 0; learned sr 2 cells. Four churning cell sets (chp, wdp, wg residual, wg L3 asymmetric) frozen as ship decisions; their tools now write candidates to analysis/output/candidates/. Watch: (a) _status.json all published each morning; (b) any guard refusal; (c) collector RSS. [10-08, 10-09: all 10 tables published both mornings, last_run_ok true.] Rollback for one table: copy runtime_tables/history/<date>/<name> back over runtime_tables/<name>.wd skips l3_nbm in se_flow/24-47 (normal, two-window CONFIRMED) and nor_easter/12-23, /24-47 (TEMPORARY, single-event evidence). Deployed rev 00609-hey 12:33:28Z. Watch: (a) pair-log wd rows in those cells show applied_layer = l2_nbm; (b) nbm_regression_sentry wd stays clean; (c) the joint nor_easter re-review on 10-13. Reversal = delete the three entries and redeploy.h/sw_flow/24-47 joins h/nw_flow/24-47 in APPLIED_CELLS (n=830, +39.0%, halves 41.8/35.7). Deployed rev 00608-jot 11:28 UTC; committed 10-04 (4aa17f6). Watch: lift holds on post-flip data (counterfactual baseline); the stamp/writeback checks listed under the v0.7.20/21 entries apply to both cells. Held: h/sw_flow/12-23 (n=380 on 10-03; should cross the 400 gate within days), h/nw_flow/12-23, h/sw_flow/6-11. Reversal = remove the cell from APPLIED_CELLS.h/nw_flow/24-47 applies via APPLIED_CELLS (joined by h/sw_flow/24-47 in v0.7.23, 10-03); the other 18 curated cells stay shadow. [as of 10-05: 24 applied rows on this cell in the digest, 0 excluded.] Confirmed firing on the 08:57 tick (9 rows, blend math exact). Watch: (a) the user-visible writeback on the next nw_flow hour — hourly.corrected_humidity at an applied lead must equal the stamped blend, not L4; (b) whether the cell's lift holds on post-flip data once v0.7.21's counterfactual baseline has a full 7d window; (c) dp/nw_flow/24-47 halves tightening enough to extend the allowlist (~10-09). Reversal = APPLIED_CELLS = {} + redeploy.{f}_preempted_source_shadow and rebuilds the served counterfactual for applied rows. Watch: (a) n_excluded_no_counterfactual on h/nw_flow/24-47 should be >0 for about a day (rows applied between the 08:33 and 18:42 deploys) then fall to zero as they age out of the 7d window — if it stays non-zero past ~10-04 the stamp isn't landing [as of 10-05: 0 excluded across 87 applied rows — landing]; (b) the stamp's value on applied rows should read nbm, matching the selector's pick at those leads [still unobserved as of 10-05]; (c) nws/blend writeback paths stay inert until a cell clears. See feedback_apply_flip_invalidates_shadow_verifier.sr × nor_easter × 12-23h and 24-47h to l3_nbm in skip_table_nbm_curated.json. Circuit-breaker per feedback_fresh_fire_vs_circuit_breaker_frames: known layer/regime/direction, two-day worsening, n=220 above threshold. Watch: (a) tomorrow's pair-log 12-23h backstamps show applied_layer=l2_nbm (not l3_nbm); (b) nbm_regression_sentry sr.l3_nbm exits HOT; (c) 24h/12h sr Total Lift recovers toward raw NBM MAE. Re-review 2026-10-13 (joint with the v0.7.24 wd nor_easter cells): either remove (regime faded, walkforward proposes cleanly) or keep as formal skip. Reversal = delete the two entries in the JSON.CHP_CELL_GATE_ENABLED = True in ch_persistence_gate.py. 9 cells cleared 7-day gate (5 dynamic-only new suppressions: ne_flow/6-11, ne_flow/12-23, ne_flow/24-47, nw_flow/24-47, se_flow/12-23). Watch: (a) h_ch_persistence_blend_stage2_vs_l6 WATCH count drops from 4 live losing cells to ≤1 within a week; (b) any of the 5 new dynamic-only cells flipping OUT of gate = churn signal, revisit threshold; (c) ch pooled Prod-vs-L6 delta should improve ~5-10% MAE per 09-21 66-day anchor. Reversal: flag back to False; static _CELL_SKIP retained. [10-05 read: (a) NOT MET — count 6, not ≤1 (see Upcoming); the tool's 10-day window still has pre-09-28 rows; recheck ~10-09.] [10-09: superseded as a verify — the gate refits daily in the cloud and had 12 cells off; see Upcoming "chp gate".]_FIELD_SKIP. cl held out (walk-forward: pool_vs_raw −22.15%, reg_vs_raw −30.37%). cc retired v0.6.390 architecturally → Ccd derives max(cl_l6, cm_l6, ch_l6). cm + ch untouched, +34-50% vs raw held-out. Next step for cl: EMA/Kalman shift tracker OR recent-bias gate. See [[project_lc_regime_conditional]] + [[project_lc_regime_stage1_pool_prereq]].LIVE_DEMOTED), v0.7.16 v5 writer fix, sr L3_nbm nor_easter watch (now the v0.7.11 TEMPORARY skip, re-review 10-13). Full history → Archive → Post-ship watchesweather_data["marine_layer_correction"] every tick, ENABLED=False. In-bin cc bias collapsed 06-30 (real break, not the 07-07 "cliff" the anomaly detector reported — that was cumulative-window artifact). Split at 07-04 (cm HRRR-anomaly onset): stratum-local, pre-HRRR, rules out HRRR-anomaly causation. Companion cl signal points seasonal. Hold OFF indefinitely. Will not re-arm when cm HRRR anomaly clears — different event. Redesign candidate: time-of-year gating on the MLC bin.docs/CHANGELOG.md, trimmed on next curation)calm/12-23 lost its older half. v0.7.35a DASHBOARD — applicability map in cascade order (L1b › L2 › … › L2_NBM › NBM cascade › C1); blender status tile moved to the L1 section. v0.7.36 PIPELINE — learned-selector fitters bucket by regime_runtime via new _cache.RegimeRuntime; sr gains ne_flow/12-23. v0.7.37 INFRA — retired walkforward_lc_regime_ship_stability: v0.6.592 retired its parent 09-12 and missed it, so "Stage 2b READY" re-read a frozen file for four weeks. Closed: the "wg calm/24-47 audit gap" — the cell is already skipped; the walkforward lists already-skipped cells and the audit correctly drops them. Found: the learned classifier's features have the same fitter/runtime mismatch as its regime label (not yet scoped).nw_flow/12-23 demoted via LIVE_DEMOTED (paired −44.2%, n=207). v0.7.34 DASHBOARD — the L1 static blender registered on the applicability map; stale layer-tuple test fixed. DISCOVERY — the pair log's regime label differs from the runtime regime on ~20% of selector-routed rows (the snapshot classifies before the selector swaps NBM into t/ws/wd/cc; the error log reclassifies after), so every regime-keyed fitter trains part of its rows in the wrong cell. v0.7.35 — the pair log records regime_runtime. Closed: t FRESH FIRE = the stale override tail; pa WATCH +598.7% = a near-zero dry baseline. Backstamp prune deferred to ~10-24.myweather-refitter function · publisher redeployed (00034-geg) · Claude ran the deploys at Joe's request. v0.7.27 SHIP — dropped sr learned GBM nw_flow/24-47 via a new LIVE_DEMOTED set (paired 7d −21.5%, n=193); the 10-06 unpaired read was wrong. DISCOVERY — the h/t τ-suspect is the selector's 7d recency override, and the selector table only ever changed on deploy; new digest tool l1_selector_override_walkforward (first read all FLAT). v0.7.24 se_flow/24-47 skip confirmed firing via gate_firing_log. v0.7.28 INFRA — Joe's rule: tables meant to refit on their own refit in the cloud, and the model never needs the Mac digest. New myweather-refitter (daily 04:30 ET) and runtime_tables loader; selector table first. Backstamp appender, frozen since 09-24 (off-by-one offset + daily prune), fixed; my rewrite left a NameError that crashed one run, fixed in 186166ae. Pair-log duplicates (09-06..09-24 counted twice by 18 fitters) filtered in _cache.pair_log_paths(). v0.7.29 INFRA — L3/L4_NBM move to the cloud refit; their 7-day stale rule is now checked on every call. v0.7.30 INFRA — 7 more tables move (10 in all); chp/wdp/wg-residual/wg-L3-asymmetric cell sets frozen as ship decisions. First full cloud run 17:19Z, all 10 published; learned sr cells now se_flow/12-23 + sw_flow/0-5. Found in the v0.7.31 sweep — the sr sea-breeze override was applying a HOLD fit with 0 hourly cells. v0.7.32 SHIP — the override now follows its refit verdict (applies only on PROMOTE); off while HOLD.docs/CHANGELOG.md and git log.decay_apply.py for historical documentation.h_cc_blend_formula.py Stage 0 grid on 123K quads: max wins in 8/10 regimes; frontal +4.3% for random but halves-unstable (A=−2%, B=+8%); nor_easter +30.9% but one-sided (A=8, B=233). Only clean per-cell candidate: pre_frontal/0-5h random +3.5% n=2,115. Ccd stays max-only. Re-check when frontal + nor_easter data accumulates. See [[project_cc_blend_tuner]].analysis/h_regime_l3.py)analysis/h_l3_regime_mismatch.py)analysis/h_lead_c1a.py)analysis/h_solar_cloud_selfcheck.py)analysis/h_weekday_temp.py)analysis/h_l4_window_size.py)analysis/h_lead_granularity.py)analysis/h_mesonet_conf.py)h_rh_saturation.py) showed cm +134%, ch +149%, cl saturating +67% MAE elevation. Promoted to Stage 1 Tier 2, then killed same week by h_c1g_orthogonality.py: 1 ortho / 69 redundant / 0 confounded / 2 ambiguous. The elevation was sampling-driven — fog (obs_humidity ≥95) heavily co-occurs with both rain-forecast (C1f) and cc_fc ≥95 (cc-saturation). In the F=False or S=False subsets, fog rows show ratios 0.02–0.25× (smaller MAE than non-fog) — i.e. C1g either reduces error (when conditions disagree with model) or merely tracks F/S (when conditions agree). No independent widening signal. (analysis/h_rh_saturation.py, analysis/h_c1g_orthogonality.py)l3_regime_lead_analysis on 694k rows showed calm-regime L3 WINS every band (+15% to +44%). Real ws L3 losers: ne_flow all bands + short-lead sea_breeze. Gate would have skipped where L3 wins biggest. Removed v0.6.267. Real fix: per-(field, regime, lead_band) skip table.l6_l2_double_counting.py on 19,975 pairs: L2 only erases 3.7% of L1's MAE on cove rows. Real cause: L1 cold ~2.25°F at cove (HRRR microclimate gap). Cooling branch on cold baseline → 74.9% MAE increase in Δ ≤ −2°F bucket. Cooling branch disabled v0.6.259; warming retained (later killed too).blend_observed_into_hourly. Nothing actionable.h_pp_bin_calibration (per-decile lift), h_pp_platt_calibration (2-param logistic |Δb|=0.06 stable, |Δa|=0.30 unstable), h_pp_kalman_recalibrate (rolling a_t), h_pp_source_blend (HRRR+GFS near-collinear α→~1.0), h_pp_platt_by_regime (weighted agg WORSE +21%/+25%), h_pp_frontal_platt_stage1 — the one Stage 1 SHIP (frontal × 6-11h fixed-b=0.6, initial Brier lift −28.56%/−18.73% at n=292) has since re-flipped HOLD in 08-02 halves-strict re-cut. Root problem: recalibration parameters don't transfer across time at this sample size. pp is L1 in production; scoreboard 0.0% is legitimate. Re-check earliest 08-05 → 08-15 as more frontal passages accumulate; if frontal-narrow re-attempt fails too, structural (physical-feature gating: CAPE, RH-500mb, front proximity) may be the only path. Live section retired from Research & Diagnostics. (analysis/pp_brier_reliability.py, analysis/h_pp_*.py)