Neural block texture compression with CUDA

Neural block texture compression, single 1k image, right=compressed, with CUDA on a 5090 (~850 line .CU file), 40.78 dB in 55 secs. 4 bpp selectors + 2 bpp latent + .05 bpp MLP.

The 2 latent layers (containing the per-block parameters) are sampled with nearest sampling.

• Format: per-pixel selectors + one latent per 4×4 block + tiny MLP block decoder
• Selectors: 1024×1024×2, 3 bits + 1 bit per pixel, picked by exact exhaustive search each iteration
• Block latent: 256×256×4, 8 bits/value, trained by ES with footprint attribution
• Decoder: 8→36→36→3 MLP, 1767 weights; inputs = 2 selectors + 4 block values + 2 block-local coords
• Decoder training: antithetic ES (64 pairs, 4096-pixel minibatches) then central finite differences for the last quarter
• Latent training: 4 antithetic pairs/iteration on the full image, no backprop anywhere
• 3000 iterations, lr annealed 1× → 0.05× over the second half
• 40.78 dB fp32, 40.71 dB with the block latent quantized to 8 bits
• 6.03 bpp raw (4 bpp selectors + 2 bpp block latent + 0.05 bpp MLP), 5.54 bpp entropy-coded
• 96 bits per 4×4 block: 64 selector bits + 32 latent bits
• 55 s total on an RTX 5090, everything on the GPU (CUDA 13.1)  

Left: original, right: reconstruction 

The block artifacts are there if you look closely: