Usually cloning is not performed during backup window, but rather after.
Can you tell us following: - what is your daily backup / clone amount - how much time does it take to backup - what is the speed you get during backup / clone
I'm not sure why do you think more pools would give you more performance, but I do not share that opinion.
Also, could you explain what exactly are you referring when saying "cloning via the GUI"?
Well, I don't remember any GUI cloning unless you are referring to volume cloning. Also that pool recommendation is nonsense if you ask me - I can't see that happening.
Now back to your data. Backup window is 18-8 meaning you have 10 hours to clone data.
I assume you first backup your data to disks (VTL). As that is disk you should not suffer from multiplexing issue (unless you use real tape for backup).
I will assume that your backup is done by 8am.
I'm not sure about read speed you get from VTL, but I will assume that each drive can give you 40MB/s. SDLT tape on other side should be able to do the same. So, you build relation 12->12. With GUI you want be able to do that so scripting will be required.
40MB/s one drive will give you 144GB per hour. That's one drive. With 10 drives that would 1.5TB. 1 hour. So, if everything is configured to match your environment you should cope with load without any problems.
Things to change: - If you are using volume cloning now it's time to consider to do some scripting to have optimal cloning setup - To drive drives you need power - CPU power (I will skip here details including PCI bus and its speed). 4 CPUs for 12 disk/tapes and 12 tapes won't be enough
Monday thru Thursday I have 14 hours to backup and clone. Friday thru Sunday I have 62 hours as I don¿t backup on Saturday and Sunday.
My VTL is used for the slow clients as I only have 12 TB of total storage space. Most of the clients get push to SDLT (max of 25 mps per drive). So the data is multiplex on the tape then de multiplexed when it clones.
I have a vb program that sorts by client, ssid size, tape location and can be adjusted for number of ssid for each clone job. I am having problems with jobs locking during a nsrclone job. I¿m surprised that Networker doesn¿t have a better interface for cloning like Net Backup, not sure if would be on any slated releases.
Is there another way I should be sort my jobs before I run the clone? Are there any bugs with nrsclone in 7.1.1? Could I readjust my performance settings to get a better backup / cloning performance? How many and what size CPU, size of memory do you suggest for this environment?
You may wish to check release notes and bug fix list for latest versions; those for 7.1.4 will have what has been fixed after 7.1.1 in 7.1.2, 7.1.3 and 7.1.4, same goes for 7.2x and 7.3.x - that will give you the list of dixed issues. Nevertheless I would suggest to move to 7.2.2 or 7.1.4 at least.
I'm not sure what do you think under nsrclone locking other jobs - can you be more specific?
I'm quite sure better sizing and configuration should be achieved for you environment, but that would real job that someone else had to do it. I would start from changing your backup strategy (eg. speading fulls over the week where possible, improving disk performance, sending all backups to disk, saveset size analysis, etc.). EMC (or your support company) probably have individual willing (paid) to do that so I would consider that as an option.
If I run more than 3 concurrent jobs the processes just lock, might be version related. What do you think of the way I have the job sorts setup?
I thought about juggling the fulls during the week, but I got a few client that cause 80% of my problems. I don¿t think I can make it backup / clone before my time windows ends.
Unfortunately getting someone in is not an option at this time. What do you think of my current performance settings (settings - server=48, ATL=11, group=0, client between 4 and 8, device=1)?
I have 4-5 nsrclone or nsrstage sessions running at the same time and can't see that so it could be related to NW version indeed.
Leave those clients that are big/slow to run over the weekend, but other capable running inside backup windows should get full over the week days and thus you spead the load.
I thin parallelism settings are mostly default ones.. except server and client. I usually never put client parallelism above default one (4) as it increases CPU impact and in most cases that would be enough. From past experience I was usually forced to lower that value and never increase it. However if I find box that happy to deliver nice performance I would increase parallelism.
Your setup is such that you will never ever have more than 4 sessions going to tape drive which is fine... actually 4 would be max that I would allow anyway is looks rather good for that point. However, I remember seeing once a message about session limits during cloning and staging which made me believe those sessions get calculated into parallelism too (and not just backup as I thought originally). So, I have rather big parallelism value (96) for server to cope with backup, restore, staging and cloning that may happen at the same time. So far I do not have issues with backup window. I believe every company has or had the same problem as you do - "few clients causing 80% of problems or traffic". Usually they are old, overloaded, on old HW infrastructure and very very important so that any change towards something better gets delayed for years. Usually you fix it either via dedicated storage nodes approach (big volume, but new HW required), filer replacing file systems (obviously new HW), etc.
mdurden1
4 Posts
0
September 26th, 2006 13:00
Backup Windows (18:00 to 8:00):
- Monday Incremental 1.5TB
- Tuesday Differential Level 5 2TB
- Wednesday Incremental .5TB
- Thursday Differential Level 3 .65TB
- Friday Full or Consolidated 11TB
- Saturday Skip backup *
- Sunday Skip backup *
Speed and time vary per point in time (I have this in graph format, no sure how to send in this forum)
Per another forum you can de-multiplex a client¿s stream by directing that stream to a single pool
Cloning using the windows interface
ble1
4 Operator
•
14.4K Posts
0
September 26th, 2006 13:00
Can you tell us following:
- what is your daily backup / clone amount
- how much time does it take to backup
- what is the speed you get during backup / clone
I'm not sure why do you think more pools would give you more performance, but I do not share that opinion.
Also, could you explain what exactly are you referring when saying "cloning via the GUI"?
ble1
4 Operator
•
14.4K Posts
0
September 26th, 2006 14:00
Now back to your data. Backup window is 18-8 meaning you have 10 hours to clone data.
I assume you first backup your data to disks (VTL). As that is disk you should not suffer from multiplexing issue (unless you use real tape for backup).
I will assume that your backup is done by 8am.
I'm not sure about read speed you get from VTL, but I will assume that each drive can give you 40MB/s. SDLT tape on other side should be able to do the same. So, you build relation 12->12. With GUI you want be able to do that so scripting will be required.
40MB/s one drive will give you 144GB per hour. That's one drive. With 10 drives that would 1.5TB. 1 hour. So, if everything is configured to match your environment you should cope with load without any problems.
Things to change:
- If you are using volume cloning now it's time to consider to do some scripting to have optimal cloning setup
- To drive drives you need power - CPU power (I will skip here details including PCI bus and its speed). 4 CPUs for 12 disk/tapes and 12 tapes won't be enough
In short, that would be it.
mdurden1
4 Posts
0
September 27th, 2006 06:00
My VTL is used for the slow clients as I only have 12 TB of total storage space. Most of the clients get push to SDLT (max of 25 mps per drive). So the data is multiplex on the tape then de multiplexed when it clones.
I have a vb program that sorts by client, ssid size, tape location and can be adjusted for number of ssid for each clone job. I am having problems with jobs locking during a nsrclone job. I¿m surprised that Networker doesn¿t have a better interface for cloning like Net Backup, not sure if would be on any slated releases.
Is there another way I should be sort my jobs before I run the clone?
Are there any bugs with nrsclone in 7.1.1?
Could I readjust my performance settings to get a better backup / cloning performance?
How many and what size CPU, size of memory do you suggest for this environment?
ble1
4 Operator
•
14.4K Posts
0
September 27th, 2006 06:00
I'm not sure what do you think under nsrclone locking other jobs - can you be more specific?
I'm quite sure better sizing and configuration should be achieved for you environment, but that would real job that someone else had to do it. I would start from changing your backup strategy (eg. speading fulls over the week where possible, improving disk performance, sending all backups to disk, saveset size analysis, etc.). EMC (or your support company) probably have individual willing (paid) to do that so I would consider that as an option.
mdurden1
4 Posts
0
September 27th, 2006 07:00
If I run more than 3 concurrent jobs the processes just lock, might be version related. What do you think of the way I have the job sorts setup?
I thought about juggling the fulls during the week, but I got a few client that cause 80% of my problems. I don¿t think I can make it backup / clone before my time windows ends.
Unfortunately getting someone in is not an option at this time. What do you think of my current performance settings (settings - server=48, ATL=11, group=0, client between 4 and 8, device=1)?
ble1
4 Operator
•
14.4K Posts
0
September 27th, 2006 08:00
Leave those clients that are big/slow to run over the weekend, but other capable running inside backup windows should get full over the week days and thus you spead the load.
I thin parallelism settings are mostly default ones.. except server and client. I usually never put client parallelism above default one (4) as it increases CPU impact and in most cases that would be enough. From past experience I was usually forced to lower that value and never increase it. However if I find box that happy to deliver nice performance I would increase parallelism.
Your setup is such that you will never ever have more than 4 sessions going to tape drive which is fine... actually 4 would be max that I would allow anyway is looks rather good for that point. However, I remember seeing once a message about session limits during cloning and staging which made me believe those sessions get calculated into parallelism too (and not just backup as I thought originally). So, I have rather big parallelism value (96) for server to cope with backup, restore, staging and cloning that may happen at the same time. So far I do not have issues with backup window. I believe every company has or had the same problem as you do - "few clients causing 80% of problems or traffic". Usually they are old, overloaded, on old HW infrastructure and very very important so that any change towards something better gets delayed for years. Usually you fix it either via dedicated storage nodes approach (big volume, but new HW required), filer replacing file systems (obviously new HW), etc.