quantize.c File Reference

Demonstrates how lossy quantization pre-filters interact with lossless compression to affect compression ratio, I/O throughput, and precision loss on a realistic scientific dataset. More...

#include <netcdf.h>
#include <netcdf_filter.h>
#include "nep.h"
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/time.h>

Data Structures
struct	filter_config_t

Macros
#define	NZ   500
#define	NY   180
#define	NX   360
#define	TMP_FILE   "quantize_tmp.nc"
#define	CHUNK_Z   10
#define	CHUNK_Y   45
#define	CHUNK_X   90
#define	NVALS   ((size_t)NZ * NY * NX)
#define	UNCOMPRESSED_BYTES   ((double)NVALS * sizeof(float))
#define	ERRCODE   2
#define	ERR(e)   {printf("Error: %s\n", nc_strerror(e)); exit(ERRCODE);}
#define	NSD_MIN   1
#define	NSD_MAX   6
#define	N_NSB   6
#define	NUM_FILTERS   (sizeof(filters) / sizeof(filters[0]))

Functions
static double	get_time (void)
	Return current wall-clock time in seconds.
static int	apply_filter (int ncid, int varid, const filter_config_t *filter)
	Apply the specified lossless filter to the variable.
static int	run_one (float *orig, float *read_buf, const char *q_alg, int quantize_mode, int nsd_or_nsb, const filter_config_t *filter)
	Run one quantize × lossless filter combination and print a CSV row.
int	main (void)
	Main entry point.

Variables
static const int	nsb_values [] = {3, 6, 9, 13, 16, 19}
static const filter_config_t	filters []

Detailed Description

Demonstrates how lossy quantization pre-filters interact with lossless compression to affect compression ratio, I/O throughput, and precision loss on a realistic scientific dataset.

Quantization is a lossy pre-filter that zeroes excess bits of the IEEE 754 floating-point significand before a lossless compression step. Because adjacent quantized values share identical trailing bits, lossless compressors achieve substantially higher compression ratios. Unlike the shuffle filter, quantization introduces a bounded but irreversible precision loss.

Three quantization algorithms are tested:

• BitGroom (NC_QUANTIZE_BITGROOM): applies a uniform bit mask derived from the requested number of significant decimal digits (NSD), alternating excess bits between 0 and 1 to preserve the array mean.
• GranularBitRound (NC_QUANTIZE_GRANULARBR): computes the minimum required bits per value using IEEE rounding, achieving better compression at the same NSD than BitGroom.
• BitRound (NC_QUANTIZE_BITROUND): user specifies the number of binary significand bits (NSB) to retain directly; free from BitGroom multipoint statistics artifacts. NSB values {3,6,9,13,16,19,23} correspond to the same worst-case decimal precision as BitGroom/GranularBitRound NSD 1–7.

Each quantize algorithm × NSD/NSB level is paired with all five lossless filters at their sprint-10 optimal settings (all with shuffle enabled). Baseline rows (lossless only, no quantization) are printed first.

The program measures max absolute error by reading back written data and comparing element-wise to the original array. Baseline rows report 0.000000.

Call order: shuffle → lossless filter → nc_def_var_quantize() → nc_enddef()

Output: CSV printed to stdout with columns:

quantize_alg,nsd_or_nsb,filter,compressed_bytes,ratio,write_s,read_s,max_abs_err

Typical workflow:

./quantize > quantize_results.csv
python3 plot_quantize.py   # produces quantize_performance.jpg

Learning Objectives:

• Understand lossy quantization as a pre-filter that improves lossless compression
• Learn the three quantization algorithms: BitGroom, GranularBitRound, BitRound
• See how NSD (Number of Significant Digits) controls precision vs compression
• Measure the interaction between quantization and each lossless filter
• Quantify precision loss (max absolute error) to make informed trade-offs
• Recognize that quantization + lossless can achieve 10–50× compression ratios with bounded, predictable precision loss

Key Concepts:

• Quantization: Lossy pre-filter that zeroes excess significand bits of IEEE 754 floats; the zeroed bits compress dramatically better because adjacent values share identical trailing bit patterns
• NSD (Significant Digits): Number of significant decimal digits preserved; higher NSD = less compression but less precision loss
• NSB (Significant Bits): Number of binary significand bits retained (BitRound only); provides finer control than NSD
• BitGroom: Alternates excess bits between 0 and 1 to preserve array mean; simplest algorithm, well-established
• GranularBitRound: Computes minimum required bits per value using IEEE rounding; better compression than BitGroom at same NSD
• BitRound: User specifies NSB directly; free from multipoint statistics artifacts; most control over precision/compression balance
• Call Order: shuffle → lossless filter → nc_def_var_quantize() → nc_enddef()

Prerequisites:

• simple_nc4.c - NetCDF-4 file creation basics
• chunking.c - Understanding chunked storage
• lossless.c - Understanding the lossless filter landscape
• deflate.c - Understanding baseline lossless performance

Related Examples:

• lossless.c - Lossless-only comparison (baseline for quantize improvements)
• deflate.c - Deflate performance without quantization
• zstandard.c - Zstandard performance without quantization

Key API functions:

• nc_def_var_quantize() Enable quantization pre-filter (BitGroom/GranularBitRound/BitRound)
• nc_inq_var_quantize() Query quantization settings on an open variable
• nc_def_var_deflate() Enable shuffle and/or DEFLATE compression
• nc_def_var_zstandard() Enable Zstandard compression
• nc_def_var_szip() Enable SZIP compression
• nc_def_var_lz4() Enable LZ4 compression
• nc_def_var_bzip2() Enable BZIP2 compression
• nc_put_var_float() Write entire variable in one call
• nc_get_var_float() Read entire variable in one call

Note

The program is intended for local performance profiling. Build with ENABLE_BENCHMARKS=ON (CMake) or –enable-benchmarks (Autotools); it is excluded from regular CI.

Companion code for "The NetCDF Developer's Handbook: The Authoritative Guide to Writing High-Performance Programs for Scientific Data Management, Second Edition" (https://www.amazon.com/dp/B0H7Q1Z75L)

Author

Edward Hartnett, Intelligent Data Design, Inc.

Macro Definition Documentation

CHUNK_X

#define CHUNK_X   90

CHUNK_Y

#define CHUNK_Y   45

CHUNK_Z

#define CHUNK_Z   10

Chunk shape: matches all v1.10.0 performance examples for consistency.

ERR

#define ERR

(

e

)

{printf("Error: %s\n", nc_strerror(e)); exit(ERRCODE);}

ERRCODE

#define ERRCODE   2

N_NSB

#define N_NSB   6

NSD_MAX

#define NSD_MAX   6

NSD_MIN

#define NSD_MIN   1

NSD range for BitGroom and GranularBitRound (decimal digits). NC_FLOAT has ~6.92 significant decimal digits (23-bit significand). NC_QUANTIZE_MAX_FLOAT_NSD is 7, but NSD=7 triggers a boundary condition in BitGroom where bits_to_zero=0 and the alternating mask produces NaN on odd indices. Cap at 6 to stay within meaningful float precision.

NUM_FILTERS

#define NUM_FILTERS   (sizeof(filters) / sizeof(filters[0]))

NVALS

#define NVALS   ((size_t)NZ * NY * NX)

Total number of float values in the dataset.

NX

#define NX   360

Longitude dimension: 360 degrees.

NY

#define NY   180

Latitude dimension: 180 degrees.

NZ

#define NZ   500

Time dimension: 500 time steps.

TMP_FILE

#define TMP_FILE   "quantize_tmp.nc"

Temporary NetCDF file created and removed for each measurement.

UNCOMPRESSED_BYTES

#define UNCOMPRESSED_BYTES   ((double)NVALS * sizeof(float))

Uncompressed size in bytes.

Function Documentation

apply_filter()

static int apply_filter ( int  ncid, int  varid, const filter_config_t *  filter  )

static

Apply the specified lossless filter to the variable.

For DEFLATE, shuffle is embedded in nc_def_var_deflate(). For all other filters, the caller has already applied the shuffle-only call before invoking this function.

Parameters

ncid	NetCDF file ID.
varid	Variable ID.
filter	Lossless filter configuration.

Returns

0 on success, 1 on error.

get_time()

static double get_time ( void  )

static

Return current wall-clock time in seconds.

Returns

Seconds since the epoch as a double.

main()

int main

(

void

)

Main entry point.

Allocates two 500×180×360 NC_FLOAT buffers (original data and read-back buffer), then iterates:

1. Baseline rows: no quantization, each lossless filter at optimal setting (5 rows).
2. BitGroom NSD 1–7 × 5 filters (35 rows).
3. GranularBitRound NSD 1–7 × 5 filters (35 rows).
4. BitRound NSB {3,6,9,13,16,19,23} × 5 filters (35 rows).

Total: 110 CSV data rows + 1 header row.

Returns

0 on success, 1 on any error.

run_one()

static int run_one ( float *  orig, float *  read_buf, const char *  q_alg, int  quantize_mode, int  nsd_or_nsb, const filter_config_t *  filter  )

static

Run one quantize × lossless filter combination and print a CSV row.

Creates quantize_tmp.nc with shuffle + lossless filter + (optionally) quantization, writes a 500×180×360 NC_FLOAT temperature variable, stats the file to obtain the compressed size, reads the variable back to compute max absolute error, then removes the file.

When quantize_mode is -1 (baseline), no quantization is applied and max_abs_err is reported as 0.000000.

Parameters

orig	Pre-allocated buffer of NVALS floats (original data).
read_buf	Pre-allocated buffer of NVALS floats (read-back buffer).
q_alg	Quantize algorithm name for CSV ("none", "bitgroom", "granularbr", or "bitround").
quantize_mode	NC_QUANTIZE_BITGROOM, NC_QUANTIZE_GRANULARBR, NC_QUANTIZE_BITROUND, or -1 for no quantization.
nsd_or_nsb	NSD (BitGroom/GranularBitRound) or NSB (BitRound) value; 0 when quantize_mode is -1.
filter	Lossless filter configuration.

Returns

0 on success, 1 on any error.

Variable Documentation

filters

const filter_config_t filters[]

static

Initial value:

= {
    {"deflate",   "DEFLATE",   1},
    {"zstandard", "ZSTANDARD", 1},
    {"szip",      "SZIP",      2},
    {"lz4",       "LZ4",       1},
    {"bzip2",     "BZIP2",     1},
}

Optimal lossless filter configurations (from sprint 10).

nsb_values

const int nsb_values[] = {3, 6, 9, 13, 16, 19}

static

NSB values for BitRound, equivalent in decimal precision to NSD 1–6. NSB=23 (NSD=7 equivalent) is excluded for NC_FLOAT: it retains all 23 significand bits and is indistinguishable from no quantization.