RDO ETC1 texture compression prototype

I've now got a basic ETC1 RDO compressor working. Clusterization is now used on both the block colors/intensity table indices and selectors. This compressor supports the entire ETC1 format: 2 subblocks per block, flipping, and both differential and individual block color modes.

Here's kodim14 using only 256 unique selector vectors and 256 block colors/intensity table indices:

This is just a bare minimum prototype. It doesn't support crunch-style macroblock tiling, or required things like mipmaps, texture arrays, etc. It's a proof of principle prototype that crunch-style RDO compression is totally doable in the full ETC1 format.

Here are more examples. I have PSNR and SSIM stats, which I'm going to focus on next.

16 block color, 16 selector clusters:

 32, 32:

64, 64:

128, 128:

512, 512:

4096:

512 block color, 3072 selector clusters:

ETC1 block color clusterization progress

I've got block color ("endpoint") clusterization working pretty well with the full ETC1 format. (Not just a subset, like in last month's endpoint clusterization experiment.)

Here are some quick examples, using only 256 unique block color/intensity table values for each image, and RGB avg. error metrics. There are actually two tables for each image, one for differential and another for individual mode, each built from the same 256 clusters. The tables are closely related, so it's possible to store the block colors in 555 format and use them as predictors to delta code the 444 block colors.

This is the first (and trickiest) major step to full ETC1 CRN/RDO support in Basis (the successor to crunch). In practice I think 256 unique endpoints is too few, but I'm purposely limited the # of clusters to get a feel for how well the current algorithm works.

 kodim18 at 256 endpoint clusters, with tile and differential bit visualizations:

ETC1 optimization notes

I've been optimizing this function:

std::pair<etc1_bits, error> = ETC1Encode(pixels, options).

Which actually gives me a really fast way of accurately computing this:

error = ETC1Distance(pixelsA, pixelsB, options).

I'm seriously considering a SIMD implementation next. I wrote one for DXT1 just for fun last week.

I need this distance function to be fast in order to justify another series of bottom->up clusterization experiments, and on improving the clusterization process itself. 

ETC2 planar block only output created with etcpak

Bartosz Taudul (etcpak author) sent these ETC2 planar block only encodings in a reply to my previous post. For planar-only they look amazing!

Note: I've verified these images myself by hacking etcpak's ProcessRGB_ETC2() function to immediately "return result.first" after it calls Planar( src ); It returns all planar blocks in this case. I've verified this by generating a histogram of the used ETC1/2 modes in all the encoded blocks.

Hey GPU texture format engineers: Come on, give us more basis functions to play with! I'm starting to look more deeply at ETC2 encoded textures, and a surprising amount of blocks in some textures are using planar mode vs. the other ETC2 modes.

He also says that etcpak uses planar blocks quite often (blue indicates a planar block):

ETC2 texture compression using exclusively planar blocks

The usual explanation given for planar blocks is that they are intended for smoothly varying blocks (see my previous post on planar blocks). So it seems natural to model the block's pixels as three planes (separately R, G, and B) when trying to create a trial planar encoding.

etc2comp tries fitting several lines along the edges of the block to compute a trial plane definition, then it "twiddles" the quantized coordinates to minimize the quantization error. From what I've been told, in practice planar blocks aren't actually used that much in ETC2 compression, which seems sad to me.

I realized while looking at the planar mode's unpack function that the "offset" or "origin" component is equivalent to the DC component in the DCT. And, the H and V components are equivalent to two of the lowest frequency basis vectors (not counting DC) in the 4x4 DCT (circled in red):

So planar blocks actually support three basis vectors (including the DC component, in the upper left). So, why not try encoding each block in a test image as a ETC2-style planar block and see what the results looks like? In other words, look at using planar blocks from the perspective of transform coding.

In this experiment, I find the average block color, set the planar "O" color to that, then subtract that out from the block. I then dot (or inner product) the left-over pixel values against the H basis vector, then the V basis vector. I encode the resulting vectors into the ETC2 planar block H and V colors. I then use the same code to unpack this as I use to decode actual ETC2 planar block data. (Note because this is only a little fun experiment on Sunday night, I'm using 888 colors for O, H, and V, not 676, but I think the results should hold up in 676 with careful quantization/twiddling.)

The results are interesting. (See my newer post for much better looking planar-only encodings created by etcpak.) Remember, only planar blocks are used:

DC+H+V:

DC+H+V:

DC only:

DC+H+V:

DC only:

Rest are DC+H+V:

Visualization of random ETC2 Planar Mode blocks

For fun I've been poking around at the planar mode in ETC2. From this presentation:

Okay, they are intended for use on smoothly varying blocks. I'm intrigued by planar mode because the colors are stored at high precision (676) and there are no selectors like in the other modes. But what do planar blocks really look like though? This random planar block image, sorted by standard deviation, was pixel (box filter) upsampled by 400%:

Just the green channel:

This image was computed by poking random 8-bit bytes into an ETC2 block. If the block passed the planar ETC2 mode check it gets decoded and stored in a temporary image. After the temp image was full it gets sorted by standard deviation.

Hey - most of these random planar blocks are not actually smoothly varying! I'm unsure if this is actually useful in practice, but it's interesting.

Looking at how Planar blocks are actually unpacked, the H and V vectors are interpreted relative to the "origin" color, so they're actually signed and the unpacking code uses per-component [0,255] clamping. This is where the high frequency patterns come from.

// ro, go, bo - origin color, unpacked from 676
// rv, gv, bv and rh, gh, bh - vertical and horizontal colors, unpacked from 676
// unpack planar block's pixels - there are no selectors in this mode
for (int y = 0; y < 4; y++)
{
    for (int x = 0; x < 4; x++)
    {
        pDst->set(
            (4 * ro + x * (rh - ro) + y * (rv - ro) + 2) >> 2,
            (4 * go + x * (gh - go) + y * (gv - go) + 2) >> 2,
            (4 * bo + x * (bh - bo) + y * (bv - bo) + 2) >> 2,
            255);
        pDst++;
    }
}