Session: 2026-05-26

Summary

Extended and refined the forecast performance analysis across two new notebooks. Notebook 14_forecast_performance.ipynb received a series of metric and cosmetic improvements. A new notebook 15_trigger_performance.ipynb was created to evaluate GloFAS reforecast performance directly against Floodscan flood events (rather than reanalysis), disaggregated by return period level.

Analysis outputs

Forecast performance refinements (`14_forecast_performance.ipynb`)

Several changes to the exceedance detection and plotting logic:

Shared wet months: All analyses now filter to the intersection of months available in both GloFAS and Google GRRR reforecast data (shared_wet_months), and all plot titles include the month range.
Cumulative activation logic: At a given leadtime lt, a year is counted as activated if any forecast with leadtime ≤ lt exceeded the threshold. This ensures activation counts are monotonically non-decreasing across leadtimes. Years are counted at most once (no double-counting of multiple exceedances within a year).
Met-convention metrics: Switched from hit rate / POFD to POD (TP / (TP+FN)) and FAR (FP / (TP+FP)). Activation count (TP+FP) replaced activation return period.
Fixed-threshold plots: Exceedance analysis at the defined trigger thresholds from STATE_CONFIG (GloFAS: 3,132 m³/s; Google: 1,195 m³/s), showing POD and activation count vs leadtime.
Probability of activation plot: GloFAS-only panel with twin y-axis showing probability of activation (TP+FP)/N alongside FAR vs leadtime.
Leadtime shading: A light blue band shades the GloFAS operational action window (≤ 5 days for Adamawa) on all leadtime-axis plots.
GloFAS issue frequency caveat: Added a markdown note explaining that the Wuroboki GloFAS reforecast is issued twice-weekly (approximately every 3–4 days), yielding ~35 issue dates per wet season rather than ~120 for a daily product. This results in roughly 1/10th the valid-time density per leadtime compared to Google, making conditional skill estimates at higher RP levels (n ≤ 5) highly unstable.

GloFAS reforecast trigger performance (`15_trigger_performance.ipynb`)

New notebook evaluating the GloFAS reforecast trigger (3,132 m³/s at Wuroboki) against observed Floodscan flood events for Adamawa:

Truth: Floodscan annual wet-season (Aug–Nov) maximum mean SFED across riverine pixels, classified at 3-year and 5-year RP thresholds using the full 1998–2025 record.
Trigger: GloFAS ensemble-mean cumulative max at each leadtime exceeds the trigger threshold (once per year, using the same cumulative logic as notebook 14).
Metrics vs leadtime: Accuracy, detection rate (POD), false alarm ratio, precision, and F1 score — one line per RP level.
RP level note: The 4-year RP level is excluded because the Floodscan years that distinguish it from the 5-year RP set (1999, 2023–2025) all fall outside the GloFAS reforecast window (2003–2022), making the two event sets identical within the evaluation period.

Notebook index

Notebook	Purpose
`14_forecast_performance.ipynb`	Skill metrics, exceedance detection, and activation analysis for GloFAS and Google GRRR reforecasts
`15_trigger_performance.ipynb`	GloFAS reforecast trigger performance vs Floodscan flood events — accuracy / POD / FAR / precision / F1 vs leadtime

Open questions

GloFAS reforecast issue frequency

The Wuroboki GloFAS reforecast data contains only ~35 issue dates per wet season (July–October), spaced every 3–4 days, rather than daily issuances. This is a data availability issue rather than a product limitation — it is worth confirming whether a more complete reforecast archive exists and whether it is worth re-downloading. The sparse issue frequency substantially reduces sample sizes in the conditional skill and exceedance analyses.

Benue state trigger threshold

No trigger threshold is currently defined for the Benue state (Makurdi station) in STATE_CONFIG (glofas_thresh and google_thresh are both None). Selecting a threshold requires the same reanalysis-based RP analysis that was done for Wuroboki/Adamawa. This should follow the approach in 08_readiness_trigger_wurobokki.ipynb: compute empirical RP from the full GloFAS and Google reanalysis records at Makurdi, identify the threshold corresponding to the target return period, and validate against Floodscan events.

Historical record length inconsistencies

Different datasets used in this analysis span different time periods:

Dataset	Coverage
Floodscan SFED	1998–2025
GloFAS reanalysis	1979–present
GloFAS reforecast (Wuroboki)	2003–2022
Google GRRR reanalysis	~2016–2023

This creates inconsistencies in return period estimates, event classification, and performance evaluation depending on which record is used as the reference. It is worth deciding whether to standardise on a common reference period (e.g. 2003–2022 or 2016–2023 for the most constrained analyses), and whether more recent years (2023–2025) should be included as validation years where data allows.