Block blurring (prefiltering)

Modern GPU texture compressors have a secret (but dangerous) superpower: prefiltering (blurring). Sometimes an encoder way overfits edges, causing overall perceptual quality to collapse. One way to overcome this is to blur the input and encode the block again. This is what we do in HDR on the very worst blocks (as measured by SSIM).

It's paradoxical: blurring can boost perceptual quality.

Large block size ASTC has been misunderstood

Most game developers misunderstand ASTC: the largest block sizes were intended for the largest resolution content (4k). GPU shader deblocking is easy: at the largest block sizes it's essentially free (because the bottleneck is typically memory bandwidth on mobile/tablets, not some ALU or cached tfetches).

The largest block sizes collapse memory consumption/bandwidth enormously (0.89-1.28 bpp for 12x12 or 10x10). 4 extra samples at block boundaries that are extremely likely to hit the texture cache (because they sample into a neighbor block) are going to be dirt cheap.

Once you add a form of well-specified deblocking, the next step is to make an encoder that is deblocking aware. Then you can heavily exploit deblocking - just like all modern image/video codecs have done for decades.

Unfortunately the deployment story with ASTC so far has been pretty spotty: There are few available full-format encoders because the format is so complex.

Intel's encoder in ispc_texcomp (now archived/no longer supported) didn't support 2-4 subsets (!!), only up to 8x8, and was broken (misusing/underutilizing modes in our analysis).

ARM's LDR encoder is good (in native, WASM story seems weaker) but it isn't deblocking aware and doesn't support supercompression.

Beyond ~2k ASTC 12x12 can look exceptional with a very simple deblocking shader (1+4 bilinear or trilinear samples only at block edges, mipmapping/filtering compatible), and the memory savings are huge.

Some related info is here:

https://github.com/BinomialLLC/basis_universal/wiki/List-of-Available-Software-ASTC-Decoders

Petascale changes everything

 Let's say you have multiple petabytes of JPEG's/WebP/etc. content and you need (not want - because your competitors are doing it already) to add GPU texture support to your app. What do you do?

• Encode 2-3x times without supercompression (BC7+ASTC, maybe ETC1 too, no supercompression): utterly impractical, explodes (8x-16x or more) overall content size. Even 1 format (say ASTC) without supercompression is impractical - explodes content size.
• Use supercompression to a universal texture format, transcode on device (native or plain WASM): Adds another ~1-1.5 petabytes. The tech is entirely free, has no driver dependencies and is standardized by Khronos.
• Use compute shaders, try to transcode on device: but now you're endlessly chasing ever-changing mobile GPU driver bugs until the end of time, outliers can't use your app reliably. You're also stuck with large textures in VRAM because you can't exploit the largest ASTC block sizes (beyond 6x6, and even 6x6 sacrifices quality due to compute shader issues). Supercompressed solutions can readily exploit ASTC 8x8 (2bpp in VRAM), 10x10 (1.28 bpp) and 12x12 (0.89 bpp), while compute shader solutions are limited to the smallest block sizes (3.56 bpp - 8 bpp) and have to make sacrifices (such as disabling dual plane support in some scenarios) to even achieve that.

At petascale many things that are taken for granted ("we'll just encode multiple times!") aren't practical. We knew this when we started Binomial to work on GPU texture tech. Our tech was already quietly deployed at petascale over a decade ago. At this scale supercompression is the way to go - the only approach that scales.

Once your customer base is 100's of millions to 1+ billion, even a ~.1% failure rate is intolerable. If a compute shader (driver dependent) failure prevents a firefighter, pilot, or war fighter from reliably seeing their map or geospatial content, you can't ship it.

My day job: optimizing this equation

 

XUASTC's next step: Intra-prediction of weight grids

Binomial has shown that image compression and GPU texture compression aren't separate fields. They're the same field, and the tools from one transfer directly to the other.

XUASTC is currently using JPEG-style DCT (from 1992) on ASTC weight grids:

https://github.com/BinomialLLC/basis_universal/wiki/XUASTC-LDR-Weight-Grid-DCT

We ported JPEG-style coding into ASTC, even preserving how libjpeg-style [1-100] Q factors are used to calculate quantization tables. (Our quantization table is the standard luminance JPEG table, with simple adaptive quantization added on top.)

This works, but it means the DCT has to carry the entire weight signal (just like JPEG). At the very lowest quality factors (Q levels 1-25 or so), the lowest spatial frequencies suffer (again, just like JPEG).

The next step is to port WebP-style intra-prediction into the weight grid domain. We can easily predict weight grids from nearby blocks, then code the weight residuals using DCT. It's the logical next step, and it'll push our bitrates even lower. While seemingly everyone is distracted by neural techniques, we're targeting billions of already shipped, hyper-efficient hardware decoders.

First XUASTC LDR 4x4 rate-distortion graphs

ASTC GPU texture blocks form a latent image space where JPEG techniques still work. (So does BC7.)

Here's a XUASTC LDR 4x4 (arithmetic vs. Zstd profile) bit rate vs. distortion graph across 151 test textures/images (the same test corpus we used to create bc7e.ispc). Distortion was measured using PSNR-HVS-M. 

XUASTC LDR 4x4 transcodes to standard ASTC LDR 4x4 in memory/VRAM (8.0 bpp). It supports all 14 standard ASTC block sizes up to 12x12.

Multiple block sizes, effort 9, arithmetic profile. It took ~10k invocations of the compressor-transcoder to make this graph.

ETC1S - effort 2:

Open Geospatial Consortium membership

I'm now a member of the OGC:

https://www.ogc.org/