Compressibility evaluation

From EMule Wiki
(Difference between revisions)
Jump to: navigation, search
(New page: ''Simple Compressibility Evaluation'' With the current,(46c) standard eMule all data is compressed prior to being uploaded unless it comes from certain archives .rar .zip etc. If the data...)
 
m (Evaluation)
 
Line 5: Line 5:
 
== Evaluation ==
 
== Evaluation ==
  
When a file completes, 3 180Kb blocks [thanks SF],(eMule's block size) are tested from the ¼ ½ ¾ points in the file. If the sum of the compressed size of these blocks equals or exceeds the uncompressed size then 10 more 180Kb blocks are tested from the middle half of the file. If the sum of all of these blocks meets or exceeds the uncompressed size then compression is turned off for that file. If the original test or the second test results in a reduction of less than 3% fast compression is used. Otherwise normal full compression is used. For incomplete files the original list is used with the addition of 7z,gz and lzh. The list is also moved to the partfile constructor where it sets a flag so it is evaluated once, saving CPU cycles. Note: From version 1.3e If the SCE code is unable to evaluate a file due to file read errors etc, on three separate occasions it's compressibility will be set as per partfiles to prevent continuous rehashing of a problem file on startup. Important note: As evaluation occurs on completion, after hashing, on first run this code will rehash and evaluate all completed files. This will reset stats for these files.
+
When a file completes, 3 180Kb blocks [thanks SF],(eMule's block size) are tested from the ¼ ½ ¾ points in the file. If the sum of the compressed size of these blocks equals or exceeds the uncompressed size then 10 more 180Kb blocks are tested from the middle half of the file. If the sum of all of these blocks meets or exceeds the uncompressed size then compression is turned off for that file. If the original test or the second test results in a reduction of less than 3% fast compression is used. Otherwise normal full compression is used. For incomplete files the original list is used with the addition of 7z,gz and lzh. The list is also moved to the partfile constructor where it sets a flag so it is evaluated once, saving CPU cycles.  
 +
 
 +
Note: From version 1.3e If the SCE code is unable to evaluate a file due to file read errors etc, on three separate occasions it's compressibility will be set as per partfiles to prevent continuous rehashing of a problem file on startup. Important note: As evaluation occurs on completion, after hashing, on first run this code will rehash and evaluate all completed files. This will reset stats for these files.
  
 
from tk4 mod: [tk4 (http://emuletk4mod.sourceforge.net/TK4_Mod_Readme.htm#SCE)]  
 
from tk4 mod: [tk4 (http://emuletk4mod.sourceforge.net/TK4_Mod_Readme.htm#SCE)]  
  
 
[[category:features]]
 
[[category:features]]

Latest revision as of 14:47, 2 April 2007

Simple Compressibility Evaluation

With the current,(46c) standard eMule all data is compressed prior to being uploaded unless it comes from certain archives .rar .zip etc. If the data fails to compress it is discarded, wasting the time spent attempting compression. This is compounded by the fact that eMule uses zlib's maximum compression setting, the slowest and most CPU intensive setting. So files that are not in the limited don't compress list and are uncompressible waste CPU cycles and slow your system. Therefore the TK4 Mod tests all completed files to evaluate their compressibility. It also uses two compression levels 1 the fastest and 9 the slowest in addition to no compression. It also has a new compressibility column in shared files list, so you can see each file's setting.

[edit] Evaluation

When a file completes, 3 180Kb blocks [thanks SF],(eMule's block size) are tested from the ¼ ½ ¾ points in the file. If the sum of the compressed size of these blocks equals or exceeds the uncompressed size then 10 more 180Kb blocks are tested from the middle half of the file. If the sum of all of these blocks meets or exceeds the uncompressed size then compression is turned off for that file. If the original test or the second test results in a reduction of less than 3% fast compression is used. Otherwise normal full compression is used. For incomplete files the original list is used with the addition of 7z,gz and lzh. The list is also moved to the partfile constructor where it sets a flag so it is evaluated once, saving CPU cycles.

Note: From version 1.3e If the SCE code is unable to evaluate a file due to file read errors etc, on three separate occasions it's compressibility will be set as per partfiles to prevent continuous rehashing of a problem file on startup. Important note: As evaluation occurs on completion, after hashing, on first run this code will rehash and evaluate all completed files. This will reset stats for these files.

from tk4 mod: [tk4 (http://emuletk4mod.sourceforge.net/TK4_Mod_Readme.htm#SCE)]

Personal tools