A heavily perceptual LRDO XUASTC LDR encoder is coming soon

I've posted an example here:

 https://x.com/richgel999/status/2114046689581596751?s=20

NNTC PBR material set compressor: now supports CPU encoding

The NNTC (Non-Neural Texture Compression) PBR material set compressor now supports CPU encoding, as well as supporting CUDA. Encoding/viewing has also been tested on Windows ARM. 

https://github.com/richgel999/nntc

One major thing left: D3D12 Linear Algebra. It already supports VK_NV_cooperative_vector.

There is also an updated Prior Art disclosure, dated Sept. 19, 2026:

https://github.com/richgel999/nntc/blob/main/PRIOR_ART_DISCLOSURE.md

Latent-based GPU texture public Prior Art from 2012

Experiments in Luma-Optimized and Mipmapped DXT1 Compression:

https://web.archive.org/web/20140915122940/https://sites.google.com/site/richgel99/luma_chroma_texture_compression

The sample app's archive (with shader source code) is still available here.

Major concepts:

• It splits information by spatial frequency: full-resolution luma/detail plus lower-resolution chroma/color state. The page puts luma in mip 0 and the color/chroma image at half resolution and below.
• It reconstructs the final texel in the pixel shader from separately hardware-filtered components. The page explicitly says bilinear/trilinear/anisotropic filtering works normally because the reconstruction is linear.
• It deliberately makes the encoder aware of the runtime reconstruction. There is even an optional feedback pass that modifies the high-resolution luma to compensate for errors introduced by the compressed, upsampled low-resolution chroma.
• It treats the lower-resolution representation as something that gets interpolated by ordinary texture hardware and then combined with the high-resolution signal—not decompressed into a conventional texture first.
• It even says a preferable version would use DXT5A/BC4 for the high-resolution scalar field instead of abusing DXT1 for luma.

NNTC viewing example which uses VK_NV_cooperative_vector

The NNTC Vulkan sample is working (3000+ lines of .cpp later). It supports VK_NV_cooperative_vector for decoding.

So the same inference hardware the neural guys are using benefits NNTC, too. I'll have this checked into GitHub after testing on more hardware.

Notes on NNTC vs. neural texturing

NNTC (here on GitHub) uses a bilinear/degree-2 polynomial decoder fitted to each PBR material (i.e. a degree-2 polynomial whose only quadratic terms are the products between the two latents). Other solutions use full non-linear neural networks (MLP's).

In my testing, MLP's are usually but not always stronger, but not by much (low PSNR difference, like ~1-3 dB). MLP's also make it harder to get filtered samples - NNTC easily leverages existing GPU texture filtering hardware.

However, eventually as I optimized the NNTC-specific CUDA encoder, and made it BC4/BC5 aware, it started to beat my neural network based solution on raw PSNR. 

Training MLP's is a lot more expensive, and a harder problem to solve - and IMHO unnecessary for PBR textures if you configure the latent textures correctly. Encoding high frequency details into low-res feature channels and training MLP's to decode them is going to be quite expensive. The alternative is to do what GPU texture formats like BC1-7/PVRTC1/ASTC/etc. have been using for ages: use high resolution weight planes, and optionally (for more efficient distribution) supercompress the data in some way using RDO+LZ, DCT etc.

One of NNTC's current limits: it only supports up to 4 channels on each of the two latent planes. I may expand this to up to ~6-8 channels in the future. If I hadn't implemented a neural net prototype first, I couldn't have found a way to NNTC.

NNTC also encodes in seconds with CUDA. Even with backprop, I'm skeptical a neural solution can be competitive there.

NNTC repo now on GitHub

NNTC ("Non-Neural Texture Compression") is a two-latent PBR texture representation where the encoder fits the GPU sampling path, not a texel-only reconstruction. The GitHub repo is here:

https://github.com/richgel999/nntc/

It's a PBR texture encoder which outputs 2-3 standard .DDS files and a JSON file containing fitted coefficients. It doesn't use neural networks to decode correlated PBR material textures (between 3-18 channels, or 1-6 3-channel RGB textures). Instead it uses plain linear operations to decode (or remap, or expand the dimensionality) of the output channels after standard hardware texture sampling/filtering.

I got this working after implementing neural texture compression using ES (Evolution Strategies), then realizing (after taking inspiration from the old PVRTC1 texture format) the problem can be easily changed so no neural networks were needed to decode the material channels. The end result is far simpler and faster to encode and sample.

The first latent the tool encodes is at full texture resolution and uses BC4's or BC5's. The encoder's backend is BC4/BC5 aware. The second latent is at 1/4 resolution and uses 8-bits/channel, and is 8,16,24 or 32 bits. The encoder supports mipmapping, and the resulting latent textures can be sampled normally (with standard filtering). 

The encoder supports 1-4 channels per latent and up to 6 RGB textures.

The encoder supports Windows and Linux and currently requires CUDA. There's a D3D11 viewer (works on any GPU) showing how to sample and then decode by applying the fitted matrix coefficients. The 2nd latent texture (which is at 1/4 resolution) needs to be sampled with a MipLODBias setting of +2.

This allows PBR textures to be stored in VRAM at roughly ~2.5 bpp, not counting mipmaps. The actual bitrate in memory depends on the channel settings and the material size in textures (it's a matrix of possible bitrates).

The more correlated (or lower dimensional) the textures in the material, the better this method works. The less correlated the textures, the more latent channels are needed. 

Also see this excellent series here:

"Dimensionality reduction for image and texture set compression"

https://bartwronski.com/2020/05/21/dimensionality-reduction-for-image-and-texture-set-compression/

This repo also includes a Prior Art Disclosure, dated September 13, 2026:

https://github.com/richgel999/nntc/blob/main/PRIOR_ART_DISCLOSURE.md

BC4/BC5 could be transcoded near-losslessly to ASTC LDR 4x4 fairly easily for mobile/tablet usage. I may add this in the future.

A variant of this idea with lower in-memory bitrates, that doesn't support native GPU texture sampling but is more flexible (supporting more channels, with quantization aware encoding), is possible. This variant would still only use plain linear operations (i.e. no MLP's for decoding), but would require the developer to roll their own filtering.

Neural texture decoders can suppress DCT ringing artifacts within latent features

This is mirrored from my X post.

This is very image codec specific: Normally a DCT-based (JPEG style) codec has mosquito/ringing noise especially on text. But within a neural texture/image codec that places a quantized DCT in the training loop (via ES), the neural net can learn how to suppress these artifacts quite effectively. This is a libjpeg Q=5 AC quantization matrix, applied XUASTC/XUBC7 style on the level 0 latent's spatial values - super low DCT quality.

The first image is the full texel resolution IDCT decoded level 0 latent, and the second is the fully decompressed image (the output from the ~500 weight neural network, after decoding the 2 latents and local "cell" UV as inputs). 

The latent textures:

A latent can tolerate quantization artifacts that would be catastrophic in image space because joint training can rotate/warp the useful representation so those artifacts lie largely in low-sensitivity directions of a tiny learned decoder.