dmrpp as a chunk reference format

Thu, 12 Nov 2026 11:00:00 -0600

I keep coming back to DMR++. It is a good old idea: instead of making every client parse an HDF5 granule from scratch, publish the map ahead of time. A sidecar gives you dimensions, variables, chunk positions, offsets, and byte lengths. The client follows the references and only fetches what it needs.

That is a proxy. It is not a copy, and it is not a proof.

The reference notebook parses a small DMR++ document, turns the offsets into a VirtualiZarr chunk manifest, and puts the result next to an Icechunk-style manifest.

DMR++ as a virtual chunk manifest

Thu, 12 Nov 2026 00:00:00 +0000

DMR++ is a metadata sidecar used by OPeNDAP/Hyrax. For an HDF5 granule, it describes the dataset structure and, importantly, the byte ranges of each chunk inside the original file. A client can therefore build a virtual view of the dataset without parsing the granule from scratch.

This notebook makes the mapping explicit with a small synthetic DMR++ document. It then builds a VirtualiZarr ChunkManifest from the same references and compares the result with the shape of an Icechunk-style manifest.

Dmrpp on betolink

dmrpp as a chunk reference format

DMR++ as a virtual chunk manifest