This is something I prototyped for fun in an evening: Toy neural texture compression, trained entirely with Evolution Strategies (no backprop, no derivatives). A 128×128×8 latent texture + a 1035-weight MLP decoder reconstruct a 512×512 image (kodim23 crop) at 32.2 dB PSNR. Latent quantizes to 8 bits with only 0.04 dB loss: ~4 bpp raw, ~3.3 bpp entropy coded. Trained in 150 s on CPU. Single C++ file, ~600 lines, stb_image for I/O.
The trick that makes ES viable: each latent texel only affects a small pixel footprint through bilinear filtering, so one full-image decode pair yields a gradient estimate for every texel at once.
Rate/distortion sweep, all 3,000 iters, 8-bit latent:64×64×4 → 26.9 dB @ 0.47 bpp
64×64×8 → 28.2 dB @ 0.87 bpp
128×128×4 → 30.3 dB @ 1.65 bpp
128×128×8 → 32.2 dB @ 3.27 bpp
Output:
The latent:
Notes:
- No training framework. Dependencies are stb_image, stb_image_write, and OpenMP. The encoder and decoder share the same forward function.
- MLP training: antithetic ES with 32 perturbation pairs per step, each pair evaluated on the same random 4096-pixel minibatch. The estimated gradient goes through Adam.
- Latent training: all 131072 latent values perturbed at once, two full-image decodes per pair, 4 pairs per step. Each pixel's loss change is credited only to the 4 texels its bilinear tap reads, so every texel gets its own local estimate from one decode pair.
- Quantization is post-hoc.
- Trained for 3,000 iterations, each iteration one ES update of the MLP followed by one ES update of the latent. The curve was still rising slowly at the end; 6000 iterations gained another ~0.3 dB.
