tag:blogger.com,1999:blog-7395797351911594965.post6856093723540846632..comments2024-03-24T23:19:53.674-07:00Comments on Richard Geldreich's Blog: ETC1 and ETC1/2 Texture Compressor BenchmarkRich Geldreichhttp://www.blogger.com/profile/14358203173986928600noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-7395797351911594965.post-53561820213901464732016-09-21T03:57:19.114-07:002016-09-21T03:57:19.114-07:00> etcpak: Can you add a single API to do compre...> etcpak: Can you add a single API to do compression with multithreading, like etc2comp? And have it return a double of how much time it takes to actually execute, excluding file I/O stuff. Your codec is so fast than I/O times will seriously skew the statistics.<br /><br />etcpak has a benchmark mode, which prints image load time and mean compression time measured over NUMCPU * 10 runs. Compression in benchmark mode is done to a heap-allocated memory, so there's no I/O dependency, which is present in normal mode of operation, due to usage of mmap-ed memory.<br /><br />Image load time is an important metric in etcpak, as it is a bottleneck compared to compression speed. I was able to optimize it slightly by removing checksum calculations in libpng and zlib, for a nice 12% speed boost.<br /><br />Making an API shouldn't be too hard, just look at Application.cpp:131-152 for an implementation of benchmark mode.<br /><br />Implementation of job scheduling is another important thing to take into consideration. My stupid simple TaskDispatch manages to slightly outperform Intel's Cilk and isn't as prone to interference from other processes as MSVC's std::async.wolf.pldhttps://www.blogger.com/profile/11888562673706089198noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-86417150562512422182016-09-20T20:11:58.992-07:002016-09-20T20:11:58.992-07:00I've still got the graph, but I want to keep t...I've still got the graph, but I want to keep this simple. I'll do a deeper Pareto Frontier analysis next.<br />Rich Geldreichhttps://www.blogger.com/profile/14358203173986928600noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-48971094427588668872016-09-20T12:38:20.267-07:002016-09-20T12:38:20.267-07:00Good! When you get to that later quality-vs-speed ...Good! When you get to that later quality-vs-speed post, try ETC2comp at Effort=40 too. While developing ETC2comp, the dev team found that this was a good sweet spot. It's also the default Effort setting.<br /><br />-JBAnonymoushttps://www.blogger.com/profile/09546712669654933611noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-20758173147396633752016-09-20T12:27:38.563-07:002016-09-20T12:27:38.563-07:00Yes, I'm also measuring encoding speed, althou...Yes, I'm also measuring encoding speed, although I've been saving that for another post. Super fast encoding is valuable too, so I figured I would include effort=1 for reference purposes.<br /><br />Also, I already had the effort=1 graph (by accident!) so I figured I would keep it.<br />Rich Geldreichhttps://www.blogger.com/profile/14358203173986928600noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-62506238565537558122016-09-20T12:16:30.855-07:002016-09-20T12:16:30.855-07:00I'm not sure it makes sense to show ETC2comp a...I'm not sure it makes sense to show ETC2comp at Effort=1 (lowest quality) unless encode speed is also being measured. I guess it might be worth comparing all codecs at lowest quality setting to show their quality range.<br /><br />-JBAnonymoushttps://www.blogger.com/profile/09546712669654933611noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-41743936327283343272016-09-20T11:21:34.832-07:002016-09-20T11:21:34.832-07:00Yes, I've verified that my PSNR's are exac...Yes, I've verified that my PSNR's are exactly the same way as ImageMagick's. I've also exported .KTX files, loaded them into Mali's Texture Tool, saved .PNG's and computed PSNR's - they match mine exactly.<br />Rich Geldreichhttps://www.blogger.com/profile/14358203173986928600noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-91516208743540895422016-09-19T08:31:22.792-07:002016-09-19T08:31:22.792-07:00This post has both RGB (average) and 601 Luma PSNR...This post has both RGB (average) and 601 Luma PSNR. I'm using the same open source PSNR code I used to create crunch on these post. I use RGB average, separate R,G,B, and Luma (either 601 or 709) PSNR to compare codecs. (I also display max component error and several other per-component metrics, like RMSE, etc.) While creating crunch I had to pay attention to Luma PSNR to be competitive against the other DXT encoders, which can be extremely perceptual.<br /><br />Sadly, it's 2016 and we still don't have standards for how to compute PSNR. I've been using a newer version of this code from crunch, which supports a variety of metrics:<br />https://github.com/BinomialLLC/crunch/blob/master/crnlib/crn_image_utils.cpp<br /><br />See error_metrics::compute(). "average_component_error" is set to true. It basically creates a histogram and computes the metrics from that. It uses these formulas to compute PSNR: <br /><br />https://web.archive.org/web/20090418023748/http://bmrc.berkeley.edu/courseware/cs294/fall97/assignment/psnr.html<br /><br />Luma PSNR is even more problematic. Do they use 601 or 709 luma, for example? I've been using 601 for years but I also support 709 in my latest code. My posts have been using 601. 709 has a higher green weight, so it's even more important for an encoder to favor green vs. the other channels. In perceptual mode crunch favors green massively and it massively de-emphasizes blue. (It has to, to be competitive vs. NVDXT.)<br /><br />I'll try ImageMagick.<br />Rich Geldreichhttps://www.blogger.com/profile/14358203173986928600noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-28638242726382909372016-09-19T08:16:40.084-07:002016-09-19T08:16:40.084-07:00Nice work Rich!
One thing to look at is how PSNR ...Nice work Rich!<br /><br />One thing to look at is how PSNR is being calculated. While developing ETC2comp, the engineers at Blue Shift, Inc. found that different codecs often calculate PSNR very differently from each other and their reported PSNR values could not be reliably compared against each other.<br /><br />For that reason, when comparing codecs, we typically use a numeric encoding (not perceptual) for each codec, and then calculate PSNR on the decoded images using ImageMagick, which is a robust, neutral way to evaluate quality.<br /><br />Do you know if your PSNR calculations match those of ImageMagick?<br /><br />-JB<br />Anonymoushttps://www.blogger.com/profile/09546712669654933611noreply@blogger.com