lalexis

updated

13 years ago

L

lalexis

2 Intern

253 Posts

0

6153

March 18th, 2013 04:00

Encryption and Compression


I would like to do both encryption and compression on my backups and was wondering if it were possible and what the diSadvantages are?

I know that Networker offers both and have tried putting them both in a directive with the end result being that the backup got encrypted but compression was next to nothing. So I guess the question is how do I get it to compress first and then encrypt?

If I were able to do both how much longer would it take for a backup of about 200GB on a LTO4 tape drive?

I am running Networker 8.0.1.1 Build 132 on AIX 7.1 with a LTO4 tape library

I am also looking at doing this on the hardware level but if I could do it at the Networker level without too added time to the backups I would rather do that.

Thanks

  • deniscnqd

    68 Posts

    3247

    0

    Posted March 26th, 2013 08:00

    So you don't need to create NFS filesystems.

    Just create FS and say to Networker that this FS is an 'AFTD' or 'file' type, then retry...

  • ble1

    6 Operator

    14354 Posts

    56186 Points

    2740

    0

    Posted March 18th, 2013 07:00

    When you do encryption of the stream, it is already "compressed" - encrypted streams compress as much as zipping zip archive would compress.

  • lalexis

    2 Intern

    253 Posts

    2740

    0

    Posted March 18th, 2013 08:00

    My experience has been quite different...

    This is using Networker to encrypt and nothing else,

    Oracle reserves way more space that is currently used so the below is typical

    Before encryption a 200GB Oracle database would only use 120GB of space on tape

    After encryption the same database used the entire 200GB of tape

    Compression/encryption of windows machines yielded simular results as do the AIX servers although not that extreme

  • deniscnqd

    68 Posts

    2740

    0

    Posted March 20th, 2013 02:00

    Hello,

    expected behaviour.

    Since you want to encrypt data, Networker encrypts the whole file (even blanks/null, because it does not matter that the database file is not full), so every byte is encrypted and cannot be compressed in an effective way, leading to a size almost equal to the original file. (AES have a 'linear transformation' in order to strew bits to have a better 'statistic distribution').

    So, if you want to have both, you have to give Networker compressed files (but still not sure)...

    And also, don't remember that you have hardware drive compression.

  • lalexis

    2 Intern

    253 Posts

    2740

    0

    Posted March 20th, 2013 06:00

    Is it possible to get a compressed backup to a hard drive, from Networker and then encrypt the backup as it is sent to tape?

    I am only worried about the tape being encrypted because it may have to leave the site at some time and all communication between servers is with ssh.

    I have 1 server dedicated to backup/recovery that has more than enough space to cover a few days worth of backups that could be then sent to tape and removed from the server. Only thing is that I have not set it up to do that and do not really know how to do it in Networker.

    Thanks

  • lalexis

    2 Intern

    253 Posts

    2740

    0

    Posted March 20th, 2013 07:00

    Thanks, I am able to run the wizard...

    Looks like I need a new license for that.

    Once I get that the plan will be to compress onto disk and then encrypt to tape

    Looks like I may need to create all new Pools, Groups and so on or nodify the ones I have and creates new ones for the tape backup.

    I will let you know how it goes after I obtain a new license

  • deniscnqd

    68 Posts

    2740

    1

    Posted March 20th, 2013 07:00

    Is it possible to get a compressed backup to a hard drive, from Networker and then encrypt the backup as it is sent to tape?

    Of course !

    You can tell Networker to process a script before a save (e.g : compressing huge data into files) with the savepnpc command, then saving data with compression.

    I am only worried about the tape being encrypted because it may have to leave the site at some time and all communication between servers is with ssh.

    So, what is the problem ?

    You can always recover encrypted data, even on an another Networker server, since you provide the pass-phrase.

    I have 1 server dedicated to backup/recovery that has more than enough space to cover a few days worth of backups that could be then sent to tape and removed from the server. Only thing is that I have not set it up to do that and do not really know how to do it in Networker.

    Try to use 'file' devices or 'adv_file' devices (licensed feature), then stage data on tape after a period.

  • deniscnqd

    68 Posts

    2740

    0

    Posted March 20th, 2013 08:00

    From my memory, a 'file' device is not a licensed feature.

    After its creation, it's seen as a new device, that can be addressed by a pool.

    Once I get that the plan will be to compress onto disk and then encrypt to tape

    Issue a savepnpc against the client, to perform a compression of your files before saving them (on disk or tapes) with an AES directive.

    I don't think that you will be able to save data on the Networker server disks then encrypting when staging these datas to tape, because the save data format is specific.

    It will work if you copy data from the client to the server, then saving to tape.

  • lalexis

    2 Intern

    253 Posts

    470

    0

    Posted March 26th, 2013 04:00

    I am trying to backup 3 directories on another AIX client.

    It is saved to a pool on disk

    client.png

    pool.png

  • lalexis

    2 Intern

    253 Posts

    2739

    0

    Posted March 26th, 2013 04:00

    I have obtained the proper license and most of my backups work (still have some things to learn) but I have run into this error:

    save: Unable to write data into multiple buffers
    for save-set ID '3092349938': File too large (errno=27)

    I have set all the limits on the storage node to -1 for root but still get this error (AIX 7.1 /etc/security/limits)

    Is there something else that can be set, maybe in Networker, that could resolve this?

    Thanks