neural_upsampler: neural 2x upsampling as a tiny drop-in C++ library

On GitHub here:

https://github.com/richgel999/neural_upsampler

Neural JPEG artifact reduction in C++

Designed to be trivial to drop into any program, it filters an already decoded 32bpp RGBA JPEG image. A little helper is provided to extract the DCT quality factor, which is also fed into the MLP:

https://github.com/richgel999/jpegc_deblock

The model's ~2.2k weights are in a single header. It was trained on an RTX 5090, using libjpeg with a custom training engine.

"A 7x7-window MLP (52 inputs: the luma window as differences from the centre, the centre's luma, the 8x8 block phase, the quality factor; hidden 32 and 16, leaky ReLU; one linear output) that predicts a luma correction added to R, G and B, equivalent to a Y-only correction in YCbCr. Trained by finite differences on 514 PNGs round-tripped through baseline JPEG at qualities 30..95. On held-out images it gains about +0.95 dB luma PSNR at quality 30, +0.9 at 40, +0.7 at 80. About 20 Mpx/s on 32 threads, 1.6 Mpx/s on one."

A heavily perceptual LRDO XUASTC LDR encoder is coming soon

I've posted an example here:

 https://x.com/richgel999/status/2114046689581596751?s=20

NNTC PBR material set compressor: now supports CPU encoding

The NNTC (Non-Neural Texture Compression) PBR material set compressor now supports CPU encoding, as well as supporting CUDA. Encoding/viewing has also been tested on Windows ARM. 

https://github.com/richgel999/nntc

One major thing left: D3D12 Linear Algebra. It already supports VK_NV_cooperative_vector.

There is also an updated Prior Art disclosure, dated Sept. 19, 2026:

https://github.com/richgel999/nntc/blob/main/PRIOR_ART_DISCLOSURE.md

Latent-based GPU texture public Prior Art from 2012

Experiments in Luma-Optimized and Mipmapped DXT1 Compression:

https://web.archive.org/web/20140915122940/https://sites.google.com/site/richgel99/luma_chroma_texture_compression

The sample app's archive (with shader source code) is still available here.

Major concepts:

• It splits information by spatial frequency: full-resolution luma/detail plus lower-resolution chroma/color state. The page puts luma in mip 0 and the color/chroma image at half resolution and below.
• It reconstructs the final texel in the pixel shader from separately hardware-filtered components. The page explicitly says bilinear/trilinear/anisotropic filtering works normally because the reconstruction is linear.
• It deliberately makes the encoder aware of the runtime reconstruction. There is even an optional feedback pass that modifies the high-resolution luma to compensate for errors introduced by the compressed, upsampled low-resolution chroma.
• It treats the lower-resolution representation as something that gets interpolated by ordinary texture hardware and then combined with the high-resolution signal—not decompressed into a conventional texture first.
• It even says a preferable version would use DXT5A/BC4 for the high-resolution scalar field instead of abusing DXT1 for luma.

NNTC viewing example which uses VK_NV_cooperative_vector

The NNTC Vulkan sample is working (3000+ lines of .cpp later). It supports VK_NV_cooperative_vector for decoding.

So the same inference hardware the neural guys are using benefits NNTC, too. I'll have this checked into GitHub after testing on more hardware.

Notes on NNTC vs. neural texturing

NNTC (here on GitHub) uses a bilinear/degree-2 polynomial decoder fitted to each PBR material (i.e. a degree-2 polynomial whose only quadratic terms are the products between the two latents). Other solutions use full non-linear neural networks (MLP's).

In my testing, MLP's are usually but not always stronger, but not by much (low PSNR difference, like ~1-3 dB). MLP's also make it harder to get filtered samples - NNTC easily leverages existing GPU texture filtering hardware.

However, eventually as I optimized the NNTC-specific CUDA encoder, and made it BC4/BC5 aware, it started to beat my neural network based solution on raw PSNR. 

Training MLP's is a lot more expensive, and a harder problem to solve - and IMHO unnecessary for PBR textures if you configure the latent textures correctly. Encoding high frequency details into low-res feature channels and training MLP's to decode them is going to be quite expensive. The alternative is to do what GPU texture formats like BC1-7/PVRTC1/ASTC/etc. have been using for ages: use high resolution weight planes, and optionally (for more efficient distribution) supercompress the data in some way using RDO+LZ, DCT etc.

One of NNTC's current limits: it only supports up to 4 channels on each of the two latent planes. I may expand this to up to ~6-8 channels in the future. If I hadn't implemented a neural net prototype first, I couldn't have found a way to NNTC.

NNTC also encodes in seconds with CUDA. Even with backprop, I'm skeptical a neural solution can be competitive there.