Introduce multi thread into nsis decompression(reduce installation time)

So far, we have brought faster compression algorithms, and have brought multi thread compression. Users can feel faster installation speed only when a faster compression algorithm is used, however, we may have to choose slower compression algorithms like lzma for better compression ratio. In this case, the prepare speed is indeed improved while the installation speed is as slow as usual. We should also improve this!

according the 7z sdk, http://www.7-zip.org/sdk.html. the decompression speed is about 20MBps to 30MBps in 2Gbhz CPU, it’s slower than traditional hard disk read/write speed which is about 120MBps(SSD is about 250MBps to 450MBps). At the mean time, 4 threads(Intel i5) and 8 threads(Intel i7) are very popular. According this, we can accelerate the speed to be about 4x or 8x faster!

Due to the scripting nature of nsis, I don’t want to significantly change the logic of installation, so only improve decompression speed of large files(more than one chunk). By default, the filebuf is 4MB. We split files into chunks according compression thread number and filebuf size.

Case 1: we have a 1MB file, it’s compressed in single thread and is stored into a single chunk.
Case 2: if it’s a 500MB file, and we use 8 threads for compression, then we get 8 chunks. the installer will use up to 8 threads to decompress(it use maximum thread of physical cpu, so it could be 2 threads if it’s running on a dual core cpu).
The compression thread number and decompression thread number are both configurable.

Here is part of installation log of 16 threads(still contains 6.9GB data, using lzma):

logging started: [2016/01/02 00:16:55:535]

SetDecompressionThread(0 -> 16)

logging stopped: [2016/01/02 00:17:47:592]

And this is part of installation log of 1 thread

logging started: [2016/01/02 00:24:00:864]

SetDecompressionThread(1 -> 1)

logging stopped: [2016/01/02 00:30:41:533]

52s vs 6min41s (401s), roughly 8x faster!

image (5)

We can use lzma now, enjoy the good compression ratio while prepare fast, install fast.

Advertisements

5 Comments

  1. Hi, I am interested in this opetion, I do not find any documentation about it.
    Could you guide me where to find more information about the setdecompressionthread option ?

    Like

  2. is it possible you provide your modification to the main developper of the NSIS project so they can review it and include it in the future realease so everyone can benefit from what seems a wonderful improvement ?

    there is a requirement that is present for quite some time on the same subject as you have completed : https://sourceforge.net/p/nsis/feature-requests/197/

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s