improve nsis with lz4(frame) and lzham

I found our patch server needs almost one hour to pack a patcher recently, 6.9GB source file, packed into 3.2GB, nsis-lzma. it’s unacceptable slow, so I decided to improve it.

I have made a version of nsis with lz4 compression which stores data in dynamic block size last year, and it’s a standalone build of nsis, can not use with other algorithm. and I’ve knew it has released a frame format. so it’s time to switch to frame format(streaming format).

Beside I checked for many websites, lzham is good alternative of lzma, faster and similar compression ratio. so it also worth a try.

Ok now I have implemented a version of nsis with zlib/bzip2/lzma/lz4/lzham support, have so many choices! Beside, lz4/lzham are implemented as a standard compressor plugin, so we can use a single nsis installation, and switch compressors in script, like how we switch between zlib bzip2 lzma.

Here is the result(taken in a file server equipped with fusion io, so i/o is not the bottleneck, instead, CPU is bottleneck because of the single compression thread of nsis).

Source Size: 6.9GB

LZ4: 32 s, 66.6%, 4.59GB

ZLIB: 1129 s, 57.2%, 3.95GB

ZBIP2: 1746 s, 55.7%, 3.84GB

LZHAM: 1717 s, 53.7%,3.71GB

LZMA: 3475 s, 46.4%,3.2GB

photo_2015-12-24_21-58-52

Because patch’s data has been prepared with VC-DIFF and other delta logic before packing with nsis, so compression ratio is not the critical factor.

According the result, lzma is still the best choice for small installers or when it has very good compression ratio. Secondly, lz4 should be the best choice if the source files are already compressed by zlib(lots of games use .zip to store their resources). Additionally, if you don’t mind slightly worse compression ratio, and faster compression speed and decompression speed, choose lzham.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s