Rules and Conditions
Who Can Submit
Developers and owners of new or existing compression software (compressors) may compete, either as individuals or groups.
Competition Rules
General
Description of Test Categories
- Participation is free.
- The submitted compressor may use any data-compression method, provided the decompressor can losslessly (bit-accurately) restore the original information using the compressed output data.
- Each participant may submit several essentially different compressors. At our sole discretion we may decline some of them if they lack any essential differences.
- Once we begin accepting submissions we run preliminary tests and publish interim leaderboards about every month. Participants should use the open part of the test set (40% for each test) and our reported statistics to improve their submissions.
- After the deadline for new and updated submissions, we calculate the final leaderboards and announce the winners eligible for awards.
Description of Test Categories
Submissions
Results and Ranking
Ranking
We believe a good rank should consider at least compression ratio, compression speed, and decompression speed. The dependencies in the ranking formula and the exact coefficients are based on the use case, however. Finding a universal formula is therefore problematic, so we use a weight function for the rapid-compression categories only when speed is especially important.
Thus for the "balanced" and "high compression ratio" categories we rank compressors according to the following metric:
c_full_size = compressed-data size + compressed-decompressor size
First place goes to the compressor with the smallest c_full_size.
We compress decompressors using bzip2 v.1.0.8 with the "-9" setting.
We believe that formulas based on the monetary cost associated with compression and decompression should be linear. In this comparison we assume the decompression speed is twice as important as compression speed because of I/O-operation statistics for typical databases. We arbitrarily selected the weight for the compressed-data size: 1,000,000 bytes of compressed data (0.1% of the input data size) costs as much as 1 second of encoding time or 0.5 seconds of decoding time.
So for the rapid categories we rank according to the function:
f = c_time + 2·d_time + 1/10⁶·c_full_size,
where c_time and d_time are, respectively, the compression and decompression times in seconds, and c_full_size is in bytes.
First place goes to the compressor with the smallest value for f.
Awards
Disqualification
A submission is subject to disqualification for a given category if it exhibits any of the following:
- For each submission, a participant may decide to enter only a subset of the categories—for example, only rapid compression for all four tests or only the three categories for one particular test.
- For each submission category, the participant must provide exactly one set of options under which the compressor meets that category's speed requirement. So, participants may submit a compressor in each category, but with just one set of options apiece. If it fails to meet the speed requirement for a given category, we can help the participant select the appropriate options, but that participant is still responsible for providing acceptable options.
- Participants may update each submission up to six times (for a total of up to seven test runs, including the initial submission), as long as the submissions meet the respective deadlines. Although we'll attempt to test submissions beyond the sixth update, especially for the top competitors of interim leaderboards, we make no guarantees to that effect.
- If a participant provides versions of the same compressor for different operating systems (Linux and Windows), we use the characteristics of the fastest one.
- After the deadline for new and updated submissions, each participant will receive the following information for cross-check before we announce results:
- Binary code of all the compressors in the competition
- All raw statistics (compressed-data size as well as compression and decompression times) for each compressor
- The entire test set
Results and Ranking
- We calculate statistics for the full test set, including both the open and closed portions of the data. Final ranks will be in accordance with the rules and formulas defined in "Ranking".
- For each submission we use the best-ranked variant (i.e., the variant with the highest average rank for all categories) to calculate the leaderboards. Our ranking calculations employ the statistics for this variant only; for example, even if another variant has a decompressor that can bit-accurately decode compressed data faster, we decline to consider it because we don't mix and match compressors and decompressors from different variants.
- To compute the compressed-data full size, we compress the decompressor using bzip2 v.1.0.8 with the "-9" setting. If the submitted compressor is just a single program, we use the total compressor size.
- We may, at our sole discretion, fully or partially delete from the results information about certain compressors, but only in exceptional cases (e.g., critical compressor errors or lossy compression).
- We may limit the number of entries in the leaderboards to show only Top-N compressors.
Ranking
We believe a good rank should consider at least compression ratio, compression speed, and decompression speed. The dependencies in the ranking formula and the exact coefficients are based on the use case, however. Finding a universal formula is therefore problematic, so we use a weight function for the rapid-compression categories only when speed is especially important.
Thus for the "balanced" and "high compression ratio" categories we rank compressors according to the following metric:
c_full_size = compressed-data size + compressed-decompressor size
First place goes to the compressor with the smallest c_full_size.
We compress decompressors using bzip2 v.1.0.8 with the "-9" setting.
We believe that formulas based on the monetary cost associated with compression and decompression should be linear. In this comparison we assume the decompression speed is twice as important as compression speed because of I/O-operation statistics for typical databases. We arbitrarily selected the weight for the compressed-data size: 1,000,000 bytes of compressed data (0.1% of the input data size) costs as much as 1 second of encoding time or 0.5 seconds of decoding time.
So for the rapid categories we rank according to the function:
f = c_time + 2·d_time + 1/10⁶·c_full_size,
where c_time and d_time are, respectively, the compression and decompression times in seconds, and c_full_size is in bytes.
First place goes to the compressor with the smallest value for f.
Awards
- Huawei, our sponsor, will remit prizes directly to the first- and second-place competitors in each category, as the "Scope" section specifies. Depending on the winner's jurisdiction and other circumstances, the company may deduct applicable taxes and fees from the prize sum.
- Participants who place among the top three in any category will receive a formal certificate from Moscow State University and Huawei.
- Members of the competition committee are ineligible for awards.
Disqualification
A submission is subject to disqualification for a given category if it exhibits any of the following:
- Failure to meet the speed requirements of that category.
- Use of GPU(s) or multithreading.
- Failure to decompress all the compressed data without loss, including decompression on a computer different from the one used for compression.
- Failure to pass antivirus tests.
- Attempts to remotely access other computers, or other files on the storage device.
- Failure to meet participant or compressor requirements—in particular, submission of a compressor that's owned by another company or individual or that otherwise violates intellectual-property rights.
Participant Requirements
- Winners in each category agree to grant us permission to publish their binary code.
- All participants shall allow us to share their binary code with other participants.
- Each participant guarantees that the submitted compressor or compressors are that participant's own work or, if they're based on third-party source code, they differ essentially from that code and the participant has received all necessary permissions to modify and submit the code to this competition. Each participant must also guarantee that the submitted compressor or compressors violate no third-party intellectual-property rights.
- By entering this competition, the participant agrees with all applicable rules and conditions.
Compressor Requirements
- To participate in the non-block-compression categories (Tests 1, 2 and 3), the compressor should be a command-line standalone application with batch-processing support: all the necessary options and file names must be assignable from the command line. The software can be two separate programs (a set of files with one executable): the compressor for compression (encoding) only and the decompressor for decompression (decoding) only. We recommend this format because we add the decompressor size to the compressed-data size.
- The total size of the package (the compressor and decompressor and all the files they use) must be less than 20 MiB (20,971,520 bytes).
- To participate in the block-compression test (Test 4), the compressor must be a library ( .dll/ .so) with the following interface. The compressor may not take any parameters for Test 4; they must be hard coded in the library.
- Allowed operating systems are Linux and Windows.
- Multithreading is prohibited: we allow one thread only.
- GPU use is prohibited.
- We offer no guarantee that we'll use exact versions of interpreted-language run-time environments, such as Java, Perl, or Python.
- We don't guarantee the exact amount of available physical memory at run time. Please check the test-hardware description before estimating your software's maximum memory consumption.
- Every compressor must be able to encode and bit-accurately decode (decompress) all four test sets.
Submission Requirements
Send your submission and any updates to globalcompetition@compression.ru.
If your submission is bigger than 10 MB, please use a file-sharing service and provide a link to download it.
Your email should contain the following information in free form:
For subsequent submissions, you may provide only new or modified information in your email.
If your submission is bigger than 10 MB, please use a file-sharing service and provide a link to download it.
Your email should contain the following information in free form:
- Author name
- Preferred contact email address and, optionally, additional contact information
- Compressor name
- List of categories you wish to enter
- URL for the compressor package, or an attachment containing the package
For subsequent submissions, you may provide only new or modified information in your email.
Test Hardware
Test-Set Description
Test 1: Qualitative Data (text)
Test 1 uses a single file containing various appended English texts from Project Gutenberg in UTF-8 characters, so it's essentially ASCII. The total size is 1 GB, or 1,000,000,000 bytes.
Test 2: Quantitative Data (images)
Test 2 contains 100 images in 8-bit PNM format. The total size is 1,000,452,004 bytes, or about 1 GB. We compress data file-to-file.
Types of images:
Test 3: Mixed Data From Executable Files
Test 3 comes from an Ubuntu x64 distribution and x64 shared library files from Python packages for Linux. The test contains filtered content of executable files. Also, we filter the data to ensure the test contains a reasonably balanced mix of quantitative and qualitative data. The total size is 1 GB, or 1,000,000,000 bytes.
Test 4: Block Compression of Mixed Data
Test 4 employs a roughly 30-70 mixture of Test 1 and Test 3 data. The total size is 1,000,013,824 bytes, or approximately 1 GB.
Test 1 uses a single file containing various appended English texts from Project Gutenberg in UTF-8 characters, so it's essentially ASCII. The total size is 1 GB, or 1,000,000,000 bytes.
Test 2: Quantitative Data (images)
Test 2 contains 100 images in 8-bit PNM format. The total size is 1,000,452,004 bytes, or about 1 GB. We compress data file-to-file.
Types of images:
- Photographic. These images come from RAW files and constitute about 75% of the test set by volume. We acquired the source images, with permission, from https://www.dpreview.com/.
- Astronomical. We generated these images from FITS files provided by The Sloan Digital Sky Survey; a few images are from NASA's Photojournal.
Test 3: Mixed Data From Executable Files
Test 3 comes from an Ubuntu x64 distribution and x64 shared library files from Python packages for Linux. The test contains filtered content of executable files. Also, we filter the data to ensure the test contains a reasonably balanced mix of quantitative and qualitative data. The total size is 1 GB, or 1,000,000,000 bytes.
Test 4: Block Compression of Mixed Data
Test 4 employs a roughly 30-70 mixture of Test 1 and Test 3 data. The total size is 1,000,013,824 bytes, or approximately 1 GB.
Rationale
Qualitative-Data Compression
For the qualitative-data test, we chose natural-language text for the following reasons:
Quantitative-Data Compression
Our decision to use images for the quantitative-data test follows reasoning similar to that for the qualitative-data test:
Mixed-Data Compression
For many tasks, the input data is a mixture of unknown types that may necessitate different processing approaches. Efficient compression requires quick and reliable identification of common data types and use of an appropriate algorithm or processing mode for compressing them.
In addition, executable files are an important and widespread data class.
Since we focus on universal approaches, we exclude compressed data because processing it efficiently may require specialized decompression and recompression. Also, we filter data to ensure that the test contains comparable estimated percentages of quantitative and qualitative data.
Test Size and Small-Block-Data Compression
The 1 GB data size allows for sufficient statistics or a dictionary to demonstrate the strength of a compression algorithm, and it permits a speed estimate by ensuring that the compression or decompression process far outweighs initialization. Furthermore, it permits all the input data, all the output data and the compression program fit in the RAM of a typical computer.
For many compression tasks, the amount of data requiring compression is limited. A quick random access to compressed blocks is necessary. A compression algorithm's efficiency and implementations are much different for small blocks (files) than for large blocks of many megabytes or more. Ranking for a 1 GB test set may fail to predict compression characteristics for blocks of tens of kilobytes or less.
Speed Categories
Different applications favor different characteristics in a lossless data compressor. The significance and cost of the data size and processing speed may vary tremendously, but a heavyweight boxer shouldn't be set against a lightweight one. We introduced an arbitrary division of "rapid compression," "balanced compression" and "high compression ratio" to account for these variations.
Not Carved in Stone
No test set or testing approach is ideal when considering a broad spectrum of use cases. The test data and method are subject to change and improvement for subsequent competitions. We are open to suggestions!
For the qualitative-data test, we chose natural-language text for the following reasons:
- It's probably the most common example of qualitative data. In practice, texts occur in many data types requiring compression: databases, executable files, XML files, HTML pages, text messages and so on.
- Historically, natural language was a focus of R&D with regard to universal data compression. Many practical algorithms and implementations were tailored to process texts. Therefore, this area is relatively well researched, and creation of an adequate test is less problematic than for other types of qualitative data.
Quantitative-Data Compression
Our decision to use images for the quantitative-data test follows reasoning similar to that for the qualitative-data test:
- Images are a common quantitative-data type, constituting a large portion of all stored and transferred data. In many applications, images need lossless or near-lossless compression.
- Lossless image compression is a well-covered field of science and software engineering, comprising numerous algorithms and solutions.
Mixed-Data Compression
For many tasks, the input data is a mixture of unknown types that may necessitate different processing approaches. Efficient compression requires quick and reliable identification of common data types and use of an appropriate algorithm or processing mode for compressing them.
In addition, executable files are an important and widespread data class.
Since we focus on universal approaches, we exclude compressed data because processing it efficiently may require specialized decompression and recompression. Also, we filter data to ensure that the test contains comparable estimated percentages of quantitative and qualitative data.
Test Size and Small-Block-Data Compression
The 1 GB data size allows for sufficient statistics or a dictionary to demonstrate the strength of a compression algorithm, and it permits a speed estimate by ensuring that the compression or decompression process far outweighs initialization. Furthermore, it permits all the input data, all the output data and the compression program fit in the RAM of a typical computer.
For many compression tasks, the amount of data requiring compression is limited. A quick random access to compressed blocks is necessary. A compression algorithm's efficiency and implementations are much different for small blocks (files) than for large blocks of many megabytes or more. Ranking for a 1 GB test set may fail to predict compression characteristics for blocks of tens of kilobytes or less.
Speed Categories
Different applications favor different characteristics in a lossless data compressor. The significance and cost of the data size and processing speed may vary tremendously, but a heavyweight boxer shouldn't be set against a lightweight one. We introduced an arbitrary division of "rapid compression," "balanced compression" and "high compression ratio" to account for these variations.
Not Carved in Stone
No test set or testing approach is ideal when considering a broad spectrum of use cases. The test data and method are subject to change and improvement for subsequent competitions. We are open to suggestions!
FAQ
Question1:
May I use explicit or implicit data-preprocessing methods, particularly for texts?
Answer 1:
Yes. You may use WRT, starNT and other transforms.
Question 2:
Is there any restriction on the compiler?
Answer 2:
No.
Question 3:
May I store dictionaries and similar prior-knowledge data in the compressor?
Answer 3:
Yes, as long as your compressor still meets the size limit.
Question 4:
May I send two builds of my program, one for Windows and one for Linux, to see which one is faster?
Answer 4:
Yes: this year we will allow one build per operating system, but we count them as either two update submissions or as one initial and one update submission.
Question 5:
How do you run the image test?
Answer 5:
We run the test file by file, processing each image separately.
Question 6:
How do you check the following condition: "every compressor must be able to encode and bit-accurately decode (decompress) all four test sets"?
Answer 6:
Your compressor should be able to process (encode) and losslessly restore (decode) data from all four test sets. It need not efficiently compress data in all cases or even compress the data at all in some cases. Also, this check processes Test 4 block data as one file.
