In-loop deblocking of GPU textures

I posted this thread on X last night - mirroring it here:

On GPU textures the block lattice is a function of UV + known block size. This makes GPU texture seam artifacts entirely predictable, so filtering them is easy. This applies to all GPU texture formats, but is the most valuable on ASTC which supports up to 12x12 pixel block sizes.

No extra metadata is required. The geometry of GPU texture block artifacts is known before you even look at the compressed bits. It's also easy to apply the seam filter while preserving mipmapping, trilinear filtering, etc.:

https://github.com/BinomialLLC/basis_universal/wiki/Shader-and-In-Loop-Deblocking

For normal maps: you can apply the seam filter (which only kicks in near/on block edges) then renormalize the results.

Encoders can optimize the GPU texture block parameters taking into account the reconstruction operator (the seam filter) during compression (in-loop). We use a form of SCD: Stochastic Coordinate Descent.

ASTC 12x12 (DCT supercompressed to 0.633 bpp): No seam filtering

The same ASTC 12x12 texture data with 5-tap mip-stable seam filtering in pixel shader:

This technology has already fully shipped in Basis Universal v2.5 for ASTC or supercompressed XUASTC textures with block sizes >= 10x8 (by default - it can be enabled on any ASTC hardware block size). The seam filter can be applied during transcoding to other formats (like BC7 or ETC1/2) or in a simple pixel shader (we provide samples for numerous API's/shading languages).

These zooms were captured from our online tool:

What about BC7?

For desktop PC use, it's possible to emulate the essentials of large block size ASTC GPU textures using 2x2-3x3 BC7 blocks arranged as macroblocks. The encoder then treats these macroblocks as a single unit, and uses SCD to optimize for in-loop deblocking with seam filtering. A pixel shader applies a filter along the 8x8 or 12x12 macroblock edges. Inside a macroblock the BC7 endpoint colors are shared, reducing or eliminating block artifacts within the macroblock itself. Sharing endpoint colors also increases compression efficiency.

Macroblocks permit transform coding (using DCT/DST) on larger weight matrices (i.e. beyond 4x4), resulting in finer frequency domain control and greater coding gain (efficiency).

Today, when ASTC/XUASTC is transcoded to BC7, the seam filter is applied in the pixel shader on the ASTC block boundaries (i.e. 10x10 or 12x12 etc.) - not BC7's (4x4). So we've essentially already shipped the core of this idea, but we have to transcode the ASTC LDR latent to BC7's.

For areas of the image/texture with high detail density, 8x8 texel macroblocks can be adaptively subdivided into two 2x1, two 1x2, or four 1x1 collections of BC7 blocks. (This is how the original "crunch" .CRN library worked for BC1-5.) DCT's would then be used on these smaller blocks.

Correlated material stacks (texture arrays) could also be handled pretty easily. The DCT coefficients of each texture's weights will typically be correlated across the layer, and it's likely many blocks can use one set of weights across several layers.

Full BC7 is trivially transcoded quickly to very high quality ASTC (Basis Universal's transcoder already supports this).

This is roughly what we have planned for XUBC7 v2, time permitting. ASTC is significantly higher priority because it's the most deployed hardware format now. XUBC7 v1's (already shipped in basisu v2.5) sweet spot is around 1.2-2.5 bpp (lossy), and with macroblocks lossy bitrates will plummet. 

Note XUBC7's lossless BC7 bitrate is already competitive vs. the bitrates achieved with the older (heavily lossy) RDO BC7+LZ methods.