tag:blogger.com,1999:blog-7395797351911594965.post1237800454548692062..comments2024-03-24T23:19:53.674-07:00Comments on Richard Geldreich's Blog: zlib in serious danger of becoming obsoleteRich Geldreichhttp://www.blogger.com/profile/14358203173986928600noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-7395797351911594965.post-89979699022819837382016-01-21T01:02:26.980-08:002016-01-21T01:02:26.980-08:00Brotli and potentially Zstd (which I've just s...Brotli and potentially Zstd (which I've just started benchmarking), can replace zlib for open source use. The closed source world has an additional option (BitKnit).<br />Rich Geldreichhttps://www.blogger.com/profile/14358203173986928600noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-76532232461868841572016-01-20T12:46:14.101-08:002016-01-20T12:46:14.101-08:00Which seems not to have a free license ATM http://...Which seems not to have a free license ATM http://www.radgametools.com/sales.htmNeználekhttps://www.blogger.com/profile/08669325905870605757noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-62884125303387717432016-01-20T09:55:12.553-08:002016-01-20T09:55:12.553-08:00Unless BitKnit is not OpenSource it can not be a r...Unless BitKnit is not OpenSource it can not be a replacement for zlibAnonymoushttps://www.blogger.com/profile/16135798776353424333noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-19164789229775830672016-01-18T20:41:03.633-08:002016-01-18T20:41:03.633-08:00It's now part of Rad's Oodle compression p...It's now part of Rad's Oodle compression product.<br />Rich Geldreichhttps://www.blogger.com/profile/14358203173986928600noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-56358521521179240472016-01-18T20:32:21.122-08:002016-01-18T20:32:21.122-08:00Will BitKnit be made available for general-purpose...Will BitKnit be made available for general-purpose use at some future point? It seems tentatively interesting.i336_https://www.blogger.com/profile/12238396155774119888noreply@blogger.comtag:blogger.com,1999:blog-7395797351911594965.post-15748658508132081062016-01-18T01:34:21.871-08:002016-01-18T01:34:21.871-08:00Unfortunately I don't think that it is possibl...Unfortunately I don't think that it is possible to encrypt without affecting the results. As the compressors here are variations on LZ any encryption would have to preserve the patterns in the data at the byte level, otherwise the compression would be affected. That is just a Caeser cipher which would be completely useless for protecting the data. Even then it would change the relationship between adjacent bytes which could affect the results for adaptive entropy coding if used in a codec.<br /><br />What might be possible would be to use your large dataset to help identify the large gaps in the various corpus. Finding files that fill the gaps and are freely usable would still be a problem, but at least it would help guide whether a file would make a good addition to a test set.Turtlehttps://www.blogger.com/profile/05390529619185061981noreply@blogger.com