An interactive benchmark comparing lossless and lossy compression of ERA5-Land climate data, built with Zarr and Blosc-Zstd. It runs entirely in your browser.
Quick tour — pick a region, forecast window, variable and algorithm, then compare size, ratio and reconstruction error.
Drag the selectors to pick a region, forecast window, variable and algorithm, then compare file size, compression ratio, and reconstruction error. The pre-loaded Asia dataset is already cached in your browser, so everything is fast — no server needed.
Earth Data Compression Benchmark is a working playground for asking one practical question: how small can a climate dataset get before it stops being useful? It takes a real ERA5-Land subset over Asia, encodes it with traditional lossless codecs, quantized precision levels and a neural implicit representation (INR / SIREN), then benchmarks every option on size, compression ratio, encode/decode time and reconstruction fidelity (RMSE, PSNR, SSIM). Built as a standalone FastAPI + Next.js service using xarray, Zarr v3 and PyTorch.
| Attribute | Value |
|---|---|
| Provider | ECMWF Copernicus Climate Data Store (CDS) |
| Region | Asia — 10°N–55°N, 60°E–150°E |
| Variables | 2 m temperature (K) + total precipitation (m/h) |
| Temporal / spatial | Hourly · default 48 h (24–72 h) · 0.25° (~181×181 grid) |
| Raw payload | ~25 MB float32 |
| License | CC-BY-4.0 · free via cdsapi |
The logic behind the formats is straightforward. The dataset itself is unchanged — a real ERA5-Land climate grid — but how it is stored changes how fast and how small it gets. NetCDF4 keeps everything in a single .nc file that any tool can read. Zarr splits the same data into small, independently compressed chunks, so you decompress only the region you query and new chunks load lazily. The benchmark shows how each storage choice trades size, speed and fidelity — and lets you try it live in your browser.
The original 25 MB float32 tensor is the reference. Best lossless comes in at 2.40× (NetCDF4 + zlib, bit-exact), while a 10-bit quantization reaches 5.26× at RMSE 0.0066 K — inside ERA5's own measurement uncertainty. The neural INR pushes it much further, to roughly 625× at PSNR > 57 dB. A full comparison run finishes in under 60 seconds — a practical, reusable yardstick for anyone storing or serving large climate grids.