## Saturday, October 15, 2016

### 2D Haar Wavelet Transform on GPU texture selector indices

I've been very busy refining my new ETC1 compressor, so I haven't been posting much recently. Today I decided to do something different, so I've been playing around with the 2D Haar 4x4 and 8x8 transforms (or here) on ETC1 selector bits. I first did this years ago while writing crunch on DXT1/BC1, but I unfortunately didn't publish or use the results.

To use the Haar transform on selector indices, I prepare the input samples by adding .5 to each selector index (which range from [0,3] in ETC1), do the transform, uniform quantize, then do the inverse transform and truncate the resulting values back to the [0,3] selector range. (You must shift the input samples by .5 or it won't work.)

The quantization stage scales the the floating point coefficient by 4 (to get 2 bits to the right of the decimal point, which in experiments is just enough for 4x4) and converts to integer. This integer is then divided by a quantization value, then it's converted to float and divided by 4

For this uniform quantization matrix:
1   1   1   2   2   3   3   4
1   1   2   2   3   3   4   4
1   2   2   3   3   4   4   5
2   2   3   3   4   4   5   5
2   3   3   4   4   5   5   6
3   3   4   4   5   5   6   6
3   4   4   5   5   6   6   7
4   4   5   5   6   6   7   7

I get this ETC1 image after 8x8 Haar transform+quantization+inverse transform:

The original ETC1 compressed texture (before Haar filtering):

Selector visualization:

1x difference image (the delta between the original and filtered ETC1 images):

There is error in high frequencies, which is exactly what is to be expected given the above quantization matrix.

Here's a more aggressive quantization matrix:

2   4   6   8  10  12  14  16
4   6   8  10  12  14  16  18
6   8  10  12  14  16  18  20
8  10  12  14  16  18  20  22
10  12  14  16  18  20  22  24
12  14  16  18  20  22  24  26
14  16  18  20  22  24  26  28
16  18  20  22  24  26  28  30

ETC1 image:

Selector visualization:

An even more aggressive quantization matrix:

3   6   9  12  15  18  21  24
6   9  12  15  18  21  24  27
9  12  15  18  21  24  27  30
12  15  18  21  24  27  30  33
15  18  21  24  27  30  33  36
18  21  24  27  30  33  36  39
21  24  27  30  33  36  39  42
24  27  30  33  36  39  42  45

Selector visualization:

I have some ideas on how the 4x4 Haar transform could be very useful in Basis, but they are just ideas right now. I find it amazing that the selectors can be transformed and manipulated in the frequency domain like this.