Hi people,

This is a tip for who work with multi-core Linux machines, have a good I/O (fast) and want try improve the ontape time of your backups.

I executed some tests with the PIGZ - parallel implementation of gzip .
http://www.zlib.net/pigz/

This program use the advantage of multi-cores to compacting .

In a test environment, where I have a 8 cores with 2 threads each (16 cpus from linux point of view) running red hat, with emc storage where I got* I/O average of 280MB/s* and an archive without compact of 280GB .

Running just with ontape : 45 minutes
Running with ontape + gzip -1 (fast compact):* 1h15m
Running with ontape + pigz -1 (fast compact),: 35 minutes
(using the 16 cores and blocks of 1MB)

time ontape -s -L 0 -v -t STDIO | pigz -1c -b 1024 > archive.L0.gz
.....
real 35m21.294s

The generated file is 100% gzip , with size of 67 GB .

Regards
Cesar

Great stuff. Have you tested a restore using the same?

I'd be curious how that goes, since the restore will be the most time
critical process if and when it's used.

Nick

Hi Nick,

I just uncompress (with gzip and pigz) the file and checked with md5sum andarchecker , don't execute the complete process of restore, yet.

Check the IIUG forum :
http://www.iiug.org/forums/ids/index...mes/read/21476

I got a feedback from other guy what already use the pigz with Informix.
I'm still testing it .
Regards
Cesar


--- Em qua, 29/9/10, Nick <milesnmiles (AT) gmail (DOT) com> escreveu:

De: Nick <milesnmiles (AT) gmail (DOT) com>
Assunto: Re: Tip: ontape + gzip parallel compress
Para: informix-list (AT) iiug (DOT) org
Data: Quarta-feira, 29 de Setembro de 2010, 18:17

Great stuff.* Have you tested a restore using the same?

I'd be curious how that goes, since the restore will be the most time
critical process if and when it's used.

Nick

_______________________________________________
Informix-list mailing list
Informix-list (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list

On Sep 30, 3:09*pm, Cesar Inacio Martins
<cesar_inacio_mart... (AT) yahoo (DOT) com.br> wrote:
We're using pigz since about a year, we're restoring pigz'ed ontape L0
(replaced /bin/gzip with /bin/pigz to have it system wide) every night
onto a nightly copy instance(the whole 110GB instance is packed in
20minutes into 13GB, default pigz number of threads for 4 CPUs seen by
the OS). I had some problems piping tar -cv | gzip (which was a pigz
actually) so I don't use pigz with piping. I use onconfig variable
(BACKUP_FILTER) to use /bin/gzip for ontape (pigz actually) for all my
5 IFX instances - didn't see any problems ever. Had to restore a
600+GB server once from pigz'd backup, went fine, nothing bothering.