Neural block textures vs. GPU textures

The existing neural texture work has made the entire space overly complicated.

The first step beyond GPU textures is to essentially keep the current GPU texture-style latent, but replace the fixed hardware generative decoder with a configurable MLP, and adopt PVRTC1-like block (low res) endpoint bilinear sampling:

Replace the classical GPU texture decoder (ASTC/BC1-7 etc.) with a neural net (MLP). Now you get as many output channels as needed. Then bilinear filter the "color" endpoints (like PVRTC1), and use a flexible (say 2-4) # of channels. 

I've been testing the "colors" latent (which is typically 2-4 channels) at 1/4, 1/6, 1/8 and 1/16 sizes relative to the texture's resolution.

Per-texel "selectors"/"weights" can be unchanged: use 1 channel for single textures, 2-3 for materials. I've tested 1-5 bits per channel, with potentially a different number of bits on each channel. 

The "encoder" is the training step which can use backprop or ES etc. It's amazingly flexible and looks remarkable.

Even on a single texture (not a material), the bitrate and quality is roughly competitive vs. transform methods on GPU texture latents. Except this method scales easily to materials and inference (decoding) cost is amortized across the material. The more channels the MLP outputs, the lower the effective bitrate.

Around 250-1200 MLP weights seems like a good range for this method. Tiny networks. I don't see why this can't be implemented directly in GPU hardware.

For inference on load (on the CPU) this approach seems entirely reasonable, using threading and SIMD to accelerate inference. No sidebands (like NTC uses) are needed for fast full-format BC7 encoding - the math is simple, and is already solved by bc7f. 

None of this is theoretical - I have this working now using CUDA (with a CPU SIMD fallback if needed). It's exciting.