Copernicus EMS · Rapid Mapping · container global (dev) · generated 2026-09-04 04:45 UTC

CEMS Flood Archive Ledger

Rendered directly from _meta/products.parquet and friends — regenerate at any point (report.py) for the true current state of the harvest. Every row below is real.

302

flood activations

3,611

ledger rows

2,884 / 2,888

zips archived (99.9%)

31.5 GB

uploaded

failures on ledger

Blob layout

global/
└─ copernicus_ems/flood/
   ├─ bronze/
   │  ├─ code=EMSR009/ … code=EMSR927/     ← one folder per activation, original zip basenames
   │  └─ _meta/
   │     ├─ products.parquet      the ledger — one row per target, incl. what we can't get
   │     ├─ zip_contents.parquet  one row per file inside every uploaded zip
   │     ├─ transfers.jsonl       append-only journal of every attempt
   │     └─ activations.parquet   all flood activations, both portals
   ├─ silver/    harmonized GeoParquet — every polygon + acquisition datetimes
   ├─ gold/      label_index + labels (dissolved flood + valid mask per label set)
   └─ platinum/  Portolan catalog serving the labels viewer (PMTiles + index)

From activations to labels

Every EMSR flood activation either contributes label sets or is listed below with the reason it can't. Nothing is silently dropped.

302

flood activations found

284

with zips archived

283

with flood labels

2,715

label sets in gold

code	why it has no labels (19 activations)
EMSR040	only pre-event Reference maps were produced — no delineation to harvest
EMSR049	only pre-event Reference maps were produced — no delineation to harvest
EMSR060	only pre-event Reference maps were produced — no delineation to harvest
EMSR085	only pre-event Reference maps were produced — no delineation to harvest
EMSR129	products never migrated to the archive portal
EMSR188	zips archived, but no product contains a flood extent layer (flood receded / hydrography-only delineation)
EMSR321	products never migrated to the archive portal
EMSR364	activation delivered nothing
EMSR384	products never migrated to the archive portal
EMSR451	products never migrated to the archive portal
EMSR473	products never migrated to the archive portal
EMSR474	products never migrated to the archive portal
EMSR519	products never migrated to the archive portal
EMSR693	activation closed without delivering products
EMSR701	activation closed without delivering products
EMSR710	activation closed without delivering products
EMSR792	activation closed without delivering products
EMSR846	activation closed without delivering products
EMSR926	activation closed without delivering products

The status model

Every ledger row carries exactly one status. Nothing is silently skipped — a target we chose not to fetch or physically cannot fetch keeps its row and says why. Pending by class: .

Status	Set by	Meaning	Rows
uploaded	harvest	In blob, size-verified; sha256 + member inventory recorded	2,884
excluded_ref	discovery	Pre-event Reference map — inventoried, deliberately not fetched	414
unavailable_status_N	discovery	New-portal product closed without delivery (never published)	300
unavailable_not_migrated	discovery	Legacy activation whose products were never migrated to the archive portal	7
failed_download	harvest	Download failed; HTTP status + error kept; retried via --retry-failed	4
unavailable_no_products	discovery	Activation delivered nothing	1
unavailable_no_url	discovery	Product not (yet) delivered, not closed	1

`products.parquet`

the ledger · one representative real row per status · re-running discovery merges, never drops

target_id	Status	class	source	aoi	delivery	size	files	sha256	error
EMSR009/EMSR…DETAIL03-MONIT03_v1_vector.zip	uploaded	DEL	archive	AOI01 Mariannelund	2012-07-12	3.19 MB	73	e59ab291…	—
EMSR069/EMSR…OVERVIEW-MONIT01_v1_vector.zip	failed_download	DEL	archive	AOI01 Maidenhead	2014-02-13	—	—	—	BadZipFile: File is not a zip file
EMSR009/EMSR…FERENCE_DETAIL03_v2_vector.zip	excluded_ref	REF	archive	AOI01 Mariannelund	2012-07-12	—	—	—	—
EMSR129/__unavailable_not_migrated__	unavailable_not_migrated	—	archive	—	—	—	—	—	API metadata: n_aois=4 n_products=8
EMSR659/product_id=63	unavailable_status_N	DEL	new_portal	AOI01 Spazzate-Sassatelli	2023-04-24	—	—	—	status_code=N feasible=False

Most recent fully-recorded upload, in full

target_id

EMSR773/EMSR773_AOI30_GRA_PRODUCT_v1.zip

code

EMSR773

source

new_portal

aoi

AOI30 Valencia Sud

title

product_class

GRA

delivery_time

2024-11-16 11:11:25.719258

url

https://rapidmapping.emergency.copernicus.eu/backend/EMSR773/AOI30/GRA_PRODUCT/EMSR773_AOI30_GRA_PRODUCT_v1.zip

blob_path

copernicus_ems/flood/bronze/code=EMSR773/EMSR773_AOI30_GRA_PRODUCT_v1.zip

status

uploaded

attempts

attempted_at

uploaded_at

sha256

8dddf8a13b63a3d816e6cb450e0b99bf9bcb6e9ca1238bf6fc27dcf404a57730

size_bytes

10472119

n_members

`zip_contents.parquet`

member inventory, built during download · largest members of the upload above

member	file_size	compress_size
Maps/EMSR773_AOI30_GRA_PRODUCT_20000_map_v1.pdf	10,358,105	10,264,248
EMSR773_AOI30_GRA_PRODUCT_summaryTable_v1.xlsx	121,303	111,958
EMSR773_AOI30_GRA_PRODUCT_observedEventA_v1.json	79,009	19,445
EMSR773_AOI30_GRA_PRODUCT_observedEventA_v1.dbf	38,484	590
EMSR773_AOI30_GRA_PRODUCT_transportationL_v1.lyr	35,840	5,136
EMSR773_AOI30_GRA_PRODUCT_builtUpP_v1.dbf	33,042	472
EMSR773_AOI30_GRA_PRODUCT_observedEventA_v1.shp	27,616	18,371
EMSR773_AOI30_GRA_PRODUCT_transportationL_v1.dbf	26,913	536
… of 45 members in this zip; 164,746 member rows across 2,884 inventoried zips

`transfers.jsonl`

append-only journal · 2,963 attempt records so far · latest record:

{
  "ts": "2026-09-01T03:14:24+00:00",
  "outcome": "uploaded",
  "target_id": "EMSR427/EMSR427_AOI06_DEL_MONIT01_r1_RTP01_v1_vector.zip",
  "source": "copernicus_ems",
  "category": "reference",
  "dataset": "Copernicus EMS EMSR427 DEL product",
  "provider": "Copernicus Emergency Management Service (rapid mapping)",
  "licence": "CEMS free re-use with attribution (\u00a9 European Union)",
  "origin_url": "https://cems-mapping-website.s3.eu-west-1.amazonaws.com/static/activations/EMSR427/EMSR427_AOI06_DEL_MONIT01_r1_RTP01_v1_vector.zip",
  "blob_path": "copernicus_ems/flood/bronze/code=EMSR427/EMSR427_AOI06_DEL_MONIT01_r1_RTP01_v1_vector.zip",
  "stage": "dev",
  "size_bytes": 85778670,
  "sha256": "5a563d46e9109885a5c1825c8b116e27205cbaa1ed52f76f33e848e80786ed7a",
  "error": null
}

`activations.parquet`

all EMSR flood activations, cross-validated across both portals · newest three:

code	name	countries	activated	AOIs	products	closed
EMSR927	Flood in Nepal	Nepal	2026-08-26	4	4	no
EMSR926	Flood in Latvia and Lithuania	Latvia; Lithuania	2026-08-22	4	9	yes
EMSR871	Flood in Abruzzo, Molise and Basilicata regions, Italy	Italy	2026-04-01	6	15	yes