nczarr_compression.c File Reference

Demonstrate deflate+shuffle compression in a local NcZarr dataset. More...

#include <netcdf.h>
#include <stdio.h>
#include <stdlib.h>

Macros
#define	FILE_URL   "file://nczarr_compression.zarr#mode=nczarr"
#define	NY   4
#define	NX   5
#define	NDIMS   2
#define	CHUNK_Y   2
#define	CHUNK_X   5
#define	DEFLATE_LEVEL   1
#define	ERR(e)

Functions
int	main (void)

Detailed Description

Demonstrate deflate+shuffle compression in a local NcZarr dataset.

This example extends nczarr_chunking.c by adding compression filters on top of explicit chunking. The recommended workflow for compressed storage is: (1) set the chunk shape with nc_def_var_chunking(), then (2) enable compression with nc_def_var_deflate(). Both steps must occur in define mode before nc_enddef().

The program creates the same 4x5 temperature grid with 2x5 chunks, then applies the shuffle filter and deflate (zlib) compression at level 1. After writing, it reopens the dataset and verifies the compression metadata via nc_inq_var_deflate(), then reads and validates all data values. Decompression is fully transparent — nc_get_var_float() returns the original data without any extra API calls.

Learning Objectives:

• Apply deflate compression and the shuffle filter to a NcZarr variable
• Understand the recommended order: set chunks first, then compression
• Query compression metadata (shuffle, deflate, level) after reopen
• Verify that decompression is transparent through the netCDF API

Key Concepts:

• nc_def_var_deflate(ncid, varid, shuffle, deflate, level):
  • shuffle=1: enables the shuffle filter (reorders bytes for better compression of typed data)
  • deflate=1: enables deflate (zlib) compression
  • level=0..9: compression level (0=none, 9=max; 1 is fastest compression)
• Chunk + compress workflow: Compression requires chunked storage; always set chunk shape explicitly for predictable behavior
• nc_inq_var_deflate(): Returns current shuffle, deflate, and level

Related Examples:

• nczarr_chunking.c - Chunking without compression
• nczarr_simple.c - Basic NcZarr create/read/write
• compression_performance.c - Comparing compression levels and filters (performance example)

Compilation:

gcc -o nczarr_compression nczarr_compression.c -lnetcdf

Usage:

./nczarr_compression
ncdump 'file://nczarr_compression.zarr#mode=nczarr'

Expected Output: Creates the directory nczarr_compression.zarr containing:

• 2 dimensions: y(4), x(5)
• 1 variable: temperature(y, x) of type NC_FLOAT, chunked [2, 5], with shuffle + deflate level 1
• Attributes: units="K", long_name="Temperature", _FillValue=-999.f
• A 4x5 temperature data grid stored compressed in 2 chunks

Note

Companion code for "The NetCDF Developer's Handbook: The Authoritative Guide to Writing High-Performance Programs for Scientific Data Management, Second Edition" (https://www.amazon.com/dp/B0H7Q1Z75L)

Author

Edward Hartnett, Intelligent Data Design, Inc.

Date

2026-06-26

Macro Definition Documentation

CHUNK_X

#define CHUNK_X   5

CHUNK_Y

#define CHUNK_Y   2

DEFLATE_LEVEL

#define DEFLATE_LEVEL   1

ERR

#define ERR

(

e

)

Value:

    do { \
    if (e) { \
        fprintf(stderr, "Error: %s at line %d\n", nc_strerror(e), __LINE__); \
        return 1; \
    } \
} while(0)

FILE_URL

#define FILE_URL   "file://nczarr_compression.zarr#mode=nczarr"

NDIMS

#define NDIMS   2

NX

#define NX   5

NY

#define NY   4

Function Documentation

main()

int main

(

void

)