Your quantizer

Auto is a fixed demonstration preset, not an automatic sample-min/max estimate. All presets round calibration endpoints to 0.001. Clamping limits a code to 0–15 by choosing the closest boundary.

16 codes = 15 gaps · s = (max − min) / 15 · z = clamp(round(−min/s), 0, 15)

Real value to reconstructed level

Three rows share the same real-number scale, including sources beyond the levels. The top row has all 16 reconstructed levels; code 8 represents zero. The dot is x, the square is x̂. Exact positions have no minimum drawn error.

Reconstructed endpoints: code 0 = -0.16000 · code 15 = +0.14000

These can differ from calibration min/max after the zero point rounds to an integer. Each unclamped rounded code has an error bound s/2 = 0.01000. The bound does not apply to clamped values.

Arithmetic trace

q = clamp(round(x/s) + z, 0, 15) · x̂ = s(q − z) · error = x̂ − x

x / s

2.65000

Rounded offset

3

Raw code = offset + z

11

Clamped code q

11

Reconstructed x̂

+0.06000

Signed error

+0.00700

Raw code was not clamped. Round nearest, with half ties toward +∞ (for example −2.5 → −2). Authored decimal inputs are treated exactly for this tie decision; ONNX uses ties-to-even instead.

Input cell after clamping: +0.05000 ≤ x < +0.07000. Endpoint codes include arbitrarily far tails; not every input assigned an endpoint code was clamped. This describes the mathematical decimal rule, beyond the bounded inspector.

Pack two codes

A nibble is 4 bits; a byte is 8. The first code occupies the high nibble, the fixed second code 6 the low nibble. This pair is a packing demonstration, not two selected block samples.

11 = 1011 · fixed 6 = 0110 → 0xB6 (10110110, one byte)

Codes alone: 32-bit FP32 → 4-bit payload = 8× reduction. Illustrative block format: 51 payload bytes + 4-byte FP32 scale + 1-byte zero point = 56 bytes, versus 408 FP32 bytes (7.29×). An odd sample count pads the last half-byte; real formats can add alignment, containers or other metadata. This does not measure model accuracy or speed.