We do have folks running a bunch of different technologies but I'll leave that conversation to the customers to chat about here. i am very interested as well to learn how it is all going and how we may make life better for everyone beyond just the Avamar and Data Domain integration.
BTW- since you have been runnign NetWorker for so long, as we have spoken in the past, I am curious to hear your thoughts on how we have been doing with the product over the years. Sure some good and some bad but I'd really like to hear it all.
I'm interested in hearing from anyone with real world experience of NetWorker's Deduplication Node feature / Avamar integration. Also, is there anyone out there with performance stats for NetWorker backups to a DataDomain device?
We did some trialling of Data Domain DD690 devices about a year ago - both as SMB / CIFS shares and as a VTL with Networker as a frontend. The product is very easy to use with a small command set. Benefits from a VTL point of view are that you can define a large number of virtual drives, this reduces the contention for devices that a traditional setup with multiple storage nodes / tape pools and a limited number of physical drives can suffer from. If you have the network bandwidth you could conceivably perforem a full backup every night and the dedupe facility would only save less than an incremental amount of data - at the expense of larger Networker indexes. You can also use the Data Domain as a target for advanced file type dumps - although you do need to consider the Network Bandwidth as it is presented as a network share.
The only reason we considered a Data Domain device as a VTL is that we have more SAN bandwidth than LAN bandwidth and the ease of integration with no change to existing policies. A better fit would be to use the DD as a CIFS / NFS Tier 2/3 target and simply dump multiple copies of your database dumps / log files to it across the network.
However as far as dedupe is considered your mileage will vary considerably depending upon the type of data you are backing up. We found that we were getting nowhere near the expected levels of deduplication due to our extensive use of file compression. With our on-line databases dumping to expensive replicated tier 1 SAN, storage is at a premium and all our databases are compressed either in flight or subsequently. In this case the data is effectively randomised and dedupe can find little commonality. In addition we have an Exchange add-on the compresses any email attachments and PDF documents too are internally compressed. Interestingly Oracle compression fared far better as a de-dupe source than for example Sybase dumps - I believe Oracle does not use the LZH algorithm but simply removes repeated data. There is also a dedupe penalty for parallel database dumps to mutilple disk files even if compression is turned off, the preferred option for data Domain is to dump to a single uncompressed dump file, but this has perfromance impacts on the speed of the database dump.
The solution of course is to stop compressing files on disk and we would indeed start to see the high dedupe ratios quoted, at the expense of Tier 1 storage space and network bandwidth!. We will probably be revisiting the solution when we finally get 10Gb ethernet or looking at Avamar to get source based dedupe without the need to ship all our data across the network only to have the dedupe device throw it away.
Been running dedupe targets for many years now, started off with Datadomain and have run Networker integrated w Avamar, EMC 3D(VTL) and now FalconStor. They all have their benefits and drawbacks. But there are some things that I feel are common when running Deduplication (and diskbased targets)
Performance: VTL will on paper look faster, but in reality I feel that tape mounts(eject sleep) would cripple the time gained by better "raw" troughput. On the other hand, B2D targets would be limited by the way NetWorker handles the cleanup process. (Number of SS it could remove each day.) If you really need fast recoveries, you need to set the environment right to gain performance. I really would like to test parallell NFS do a DD target, looks promising.
Deduplication: A great feature, but suddenly you as a backupadmin have to convince the application owner abut changes. (Filesperset=1 on oracle for example, no compresson as mentioned in the previous post). IF you would like to get maximum storage capacity on your target. Suddenly you have to explain storage and dedupe technology for your DBAs...
Administration: Backup targets tend to fill up, sooner or later. When you reach 95% usage, you get a bit worried, but you actually don't now if the next backup would fit anyway, cause you have dedupe and all new data would be "eliminated". Scary fridays if you know what I mean..
Clenup: Deleting data on the dedupe target face the same issue as mentioned above, you don't know if it would actually freeup any space. AND it's really time consuming. From a day or two, up to week(s). And when you need it the most(reaching +95% usage) the performance go down as it's disk we talk about.
"The dedupe akillies heel"
How we do things: A quite normal situvation is that you have been running NetWorker for a while and to be cost effective you have streched your backup cycles, as you implemented VTL or disk. If you implement dedupe you must review your schema if you have bandwith enough. Also mentioned in prior post. However, Avamar doesn't have this issue. (So NSR+Avamar might be the solution )
My personal opionion is that the way Avamar does things is the "wet dream", but still it doesn't fit everywhere just yet.
I am getting ready to deploy a multi-site. multi deep storage node array into our enviornment using Networker. Some of the things that concern me currently and I am working Enginering and Prof. Services is the tape out functionality of Avamar. Their are 2 types of TO features supported with the current release of Avamar which is ATO and ADT. Both have their merits and both have their disadvantages. To get all the details I would suggest you speak with your rep.
But my pain level with this is that there is a lack of "True" product integration (without giving EMC your first born) between Avamar and Networker. Understood that they are both their own Backup and Recovery products. But most Avamar customers also use Networker or are retiring Networker for Avamar. It would only make sense to me that they would find a cost effective way to integrate the two products and have a true single point of management, that provides for deduplication, tape out and management from the Networker console. IMHO if EMC went down this route and made it cost effective it woould only increase the value of both products and make them a force to be delt with.
Thanks for the post. I read in your other post that you are in the process of an Avamar installation. Can you give a couple specifics on integration that you would like to see between Avamar and NetWorker? I can say that there is additional integration on the roadmaps for 2010 – this is one of the goals of the new Backup Recovery Systems (BRS) Division that brings together NetWorker, Avamar and Data Domain. On the cost side, if you do add Avamar to a NetWorker installation, a new lower priced client model is available to help offset some of the cost. I think if you look across the backup industry, we have made good progress with integrating Avamar capabilities so NetWorker customers can take advantage of source deduplication. Below are some integration points that come to mind.
Single integrated client software with “built-in” Avamar capabilities
Scheduling and management of both traditional and dedupe (Avamar) jobs from NMC
Reporting on dedupe jobs from NMC
Common recovery workflow for dedupe and non-dedupe clients
What would really be nice is if Networker has a Deduplication Storage Node so customers can take advantage of their existing hardware without the forklift expense of datadomain or the need to manage another backup software like Avamar (not to mention it's expense).
Is there anything planned for Networker to deduplicate data? Say, like CommVault?
Thanks for the response and pardon my delay in getting back to you regarding this.
Currently with Networker their is a type of integrated client software. However it requires adding a dedup storage node to your configuration which does work however you are still bound to the Networker licensing schema. Which makes it very unattractive as one of the selling points behind avamar is unlimited clients (within capacity of the node) and unlimited integrations.
Reporting has always IMHO been a downfall of NMC and you need to get an add-on such as DPA. DPA another great product but they need to beef up the reporting functionality in NMC as DPA can be used for so many other things and to limit it to just backup reporting really constrains a great and versatile product.
From what I understand there is now a GA release of a TO from EMC however from what I understand it is a time cosuming process to properly integrate and still requires a traditional backup product to use it. Maybe give a traditional backup product with the unit as a way to do the tape out...
I wonder if you could rig up some sort of frankenstein hybrid storage node utilising Solaris dedupe ZFS as an advance file type device - although it wouldn't allow you to produce deduped tapes like commvault does
When you say "adding a dedup storage node to your configuration" are you referring to a disk target (or VTL) that has dedupe and not to some software piece of Networker that dedupes? I was a little unclear.
I think the expense of NW client licenses balances the "free" clients of Avamar seeing as how expensive it is to add more storage. I've not performed any studies yet but my gut tells me that one way or another, they'll be equal if not Avamar more expensive.
I think the reference here to dedupe storage node refers to the Avamar grid integrated into the NW environment. The NW client has the Avamar binaries inherent to it now (7.4 SP1 and higher). This part of the client becomes useful then if/when you roll in an Avamar server to do process and store the source dedupe jobs.
One the topic of the cost of clients - if you buy and use Avamar in the NW environment, a new lower cost client can be used (not $0 but close...). Ask your reseller or local EMC person for that cost.
The Dedup piece I am talking about is the Avamar Dedupe.
Any other device such as VTL or DEDUP (Clariion Disk Library or the sort) would be considered a storage node.
I have done my Avamar deployment and it's considerably cheaper then doing a Networker deployment as you don't need any licensing other then your capacity licensing with Avamar.
In response to your licensing comments, we are currently investigating an alternative licensing model for NetWorker. Check out the main page and help us by taking the survey over on the right side.
"However it requires adding a dedup storage node to your configuration which does work however you are still bound to the Networker licensing schema. Which makes it very unattractive as one of the selling points behind avamar is unlimited clients (within capacity of the node) and unlimited integrations."
sweller1
78 Posts
0
September 30th, 2009 10:00
We do have folks running a bunch of different technologies but I'll leave that conversation to the customers to chat about here. i am very interested as well to learn how it is all going and how we may make life better for everyone beyond just the Avamar and Data Domain integration.
BTW- since you have been runnign NetWorker for so long, as we have spoken in the past, I am curious to hear your thoughts on how we have been doing with the product over the years. Sure some good and some bad but I'd really like to hear it all.
Thanks!
Steve
Sunman2
17 Posts
0
September 30th, 2009 10:00
We just purchase a Sepaton system and are looking forward to implementing Dedupe.
Is anyone using a Dedupe system with NetWorker with any other product than Avamar? If so how is it going?
luke_edwards
3 Posts
0
October 20th, 2009 08:00
Hi,
I'm interested in hearing from anyone with real world experience of NetWorker's Deduplication Node feature / Avamar integration. Also, is there anyone out there with performance stats for NetWorker backups to a DataDomain device?
Thanks.
Luke
soupdragon2
1 Rookie
•
18 Posts
0
October 21st, 2009 03:00
We did some trialling of Data Domain DD690 devices about a year ago - both as SMB / CIFS shares and as a VTL with Networker as a frontend. The product is very easy to use with a small command set. Benefits from a VTL point of view are that you can define a large number of virtual drives, this reduces the contention for devices that a traditional setup with multiple storage nodes / tape pools and a limited number of physical drives can suffer from. If you have the network bandwidth you could conceivably perforem a full backup every night and the dedupe facility would only save less than an incremental amount of data - at the expense of larger Networker indexes. You can also use the Data Domain as a target for advanced file type dumps - although you do need to consider the Network Bandwidth as it is presented as a network share.
The only reason we considered a Data Domain device as a VTL is that we have more SAN bandwidth than LAN bandwidth and the ease of integration with no change to existing policies. A better fit would be to use the DD as a CIFS / NFS Tier 2/3 target and simply dump multiple copies of your database dumps / log files to it across the network.
However as far as dedupe is considered your mileage will vary considerably depending upon the type of data you are backing up. We found that we were getting nowhere near the expected levels of deduplication due to our extensive use of file compression. With our on-line databases dumping to expensive replicated tier 1 SAN, storage is at a premium and all our databases are compressed either in flight or subsequently. In this case the data is effectively randomised and dedupe can find little commonality. In addition we have an Exchange add-on the compresses any email attachments and PDF documents too are internally compressed. Interestingly Oracle compression fared far better as a de-dupe source than for example Sybase dumps - I believe Oracle does not use the LZH algorithm but simply removes repeated data. There is also a dedupe penalty for parallel database dumps to mutilple disk files even if compression is turned off, the preferred option for data Domain is to dump to a single uncompressed dump file, but this has perfromance impacts on the speed of the database dump.
The solution of course is to stop compressing files on disk and we would indeed start to see the high dedupe ratios quoted, at the expense of Tier 1 storage space and network bandwidth!. We will probably be revisiting the solution when we finally get 10Gb ethernet or looking at Avamar to get source based dedupe without the need to ship all our data across the network only to have the dedupe device throw it away.
Hakan_Engman
57 Posts
0
October 23rd, 2009 01:00
Hi,
Been running dedupe targets for many years now, started off with Datadomain and have run Networker integrated w Avamar, EMC 3D(VTL) and now FalconStor. They all have their benefits and drawbacks. But there are some things that I feel are common when running Deduplication (and diskbased targets)
Performance: VTL will on paper look faster, but in reality I feel that tape mounts(eject sleep) would cripple the time gained by better "raw" troughput. On the other hand, B2D targets would be limited by the way NetWorker handles the cleanup process. (Number of SS it could remove each day.) If you really need fast recoveries, you need to set the environment right to gain performance. I really would like to test parallell NFS do a DD target, looks promising.
Deduplication: A great feature, but suddenly you as a backupadmin have to convince the application owner abut changes. (Filesperset=1 on oracle for example, no compresson as mentioned in the previous post). IF you would like to get maximum storage capacity on your target. Suddenly you have to explain storage and dedupe technology for your DBAs...
Administration: Backup targets tend to fill up, sooner or later. When you reach 95% usage, you get a bit worried, but you actually don't now if the next backup would fit anyway, cause you have dedupe and all new data would be "eliminated". Scary fridays if you know what I mean..
Clenup: Deleting data on the dedupe target face the same issue as mentioned above, you don't know if it would actually freeup any space. AND it's really time consuming. From a day or two, up to week(s). And when you need it the most(reaching +95% usage) the performance go down as it's disk we talk about.
"The dedupe akillies heel"
How we do things: A quite normal situvation is that you have been running NetWorker for a while and to be cost effective you have streched your backup cycles, as you implemented VTL or disk. If you implement dedupe you must review your schema if you have bandwith enough. Also mentioned in prior post. However, Avamar doesn't have this issue. (So NSR+Avamar might be the solution )
My personal opionion is that the way Avamar does things is the "wet dream", but still it doesn't fit everywhere just yet.
regards/Håkan
Al Smith1
45 Posts
0
October 28th, 2009 16:00
I am getting ready to deploy a multi-site. multi deep storage node array into our enviornment using Networker. Some of the things that concern me currently and I am working Enginering and Prof. Services is the tape out functionality of Avamar. Their are 2 types of TO features supported with the current release of Avamar which is ATO and ADT. Both have their merits and both have their disadvantages. To get all the details I would suggest you speak with your rep.
But my pain level with this is that there is a lack of "True" product integration (without giving EMC your first born) between Avamar and Networker. Understood that they are both their own Backup and Recovery products. But most Avamar customers also use Networker or are retiring Networker for Avamar. It would only make sense to me that they would find a cost effective way to integrate the two products and have a true single point of management, that provides for deduplication, tape out and management from the Networker console. IMHO if EMC went down this route and made it cost effective it woould only increase the value of both products and make them a force to be delt with.
Just my 2cents.
AllanW1
334 Posts
0
October 29th, 2009 13:00
Hi Albert,
Thanks for the post. I read in your other post that you are in the process of an Avamar installation. Can you give a couple specifics on integration that you would like to see between Avamar and NetWorker? I can say that there is additional integration on the roadmaps for 2010 – this is one of the goals of the new Backup Recovery Systems (BRS) Division that brings together NetWorker, Avamar and Data Domain. On the cost side, if you do add Avamar to a NetWorker installation, a new lower priced client model is available to help offset some of the cost. I think if you look across the backup industry, we have made good progress with integrating Avamar capabilities so NetWorker customers can take advantage of source deduplication. Below are some integration points that come to mind.
As for the tape functionality, hopefully Steve W. or someone will comment on that.
Thanks again for visiting the Community and let us know your thoughts on integration.
Allan
ps: By the way, saw your Cub Scout note. Lots of fun. Any winter camping planned? :-)
jrinehart1
28 Posts
1
January 6th, 2010 13:00
What would really be nice is if Networker has a Deduplication Storage Node so customers can take advantage of their existing hardware without the forklift expense of datadomain or the need to manage another backup software like Avamar (not to mention it's expense).
Is there anything planned for Networker to deduplicate data? Say, like CommVault?
Al Smith1
45 Posts
0
January 6th, 2010 14:00
Allan,
Thanks for the response and pardon my delay in getting back to you regarding this.
Currently with Networker their is a type of integrated client software. However it requires adding a dedup storage node to your configuration which does work however you are still bound to the Networker licensing schema. Which makes it very unattractive as one of the selling points behind avamar is unlimited clients (within capacity of the node) and unlimited integrations.
Reporting has always IMHO been a downfall of NMC and you need to get an add-on such as DPA. DPA another great product but they need to beef up the reporting functionality in NMC as DPA can be used for so many other things and to limit it to just backup reporting really constrains a great and versatile product.
From what I understand there is now a GA release of a TO from EMC however from what I understand it is a time cosuming process to properly integrate and still requires a traditional backup product to use it. Maybe give a traditional backup product with the unit as a way to do the tape out...
soupdragon2
1 Rookie
•
18 Posts
0
January 6th, 2010 14:00
I wonder if you could rig up some sort of frankenstein hybrid storage node utilising Solaris dedupe ZFS as an advance file type device - although it wouldn't allow you to produce deduped tapes like commvault does
jrinehart1
28 Posts
0
January 6th, 2010 15:00
Albert,
When you say "adding a dedup storage node to your configuration" are you referring to a disk target (or VTL) that has dedupe and not to some software piece of Networker that dedupes? I was a little unclear.
I think the expense of NW client licenses balances the "free" clients of Avamar seeing as how expensive it is to add more storage. I've not performed any studies yet but my gut tells me that one way or another, they'll be equal if not Avamar more expensive.
ecarter11
2 Intern
•
177 Posts
0
January 6th, 2010 15:00
I think the reference here to dedupe storage node refers to the Avamar grid integrated into the NW environment. The NW client has the Avamar binaries inherent to it now (7.4 SP1 and higher). This part of the client becomes useful then if/when you roll in an Avamar server to do process and store the source dedupe jobs.
One the topic of the cost of clients - if you buy and use Avamar in the NW environment, a new lower cost client can be used (not $0 but close...). Ask your reseller or local EMC person for that cost.
Here are a few dedupe references w/ NetWorker.
http://www.emc.com/collateral/software/data-sheet/h3979-nw-dedup-ds.pdf
http://www.youtube.com/watch?v=jEwZUY4p4vU
http://www.emc.com/collateral/analyst-reports/idc-20090519-data-deduplication.pdf
Al Smith1
45 Posts
0
January 6th, 2010 18:00
The Dedup piece I am talking about is the Avamar Dedupe.
Any other device such as VTL or DEDUP (Clariion Disk Library or the sort) would be considered a storage node.
I have done my Avamar deployment and it's considerably cheaper then doing a Networker deployment as you don't need any licensing other then your capacity licensing with Avamar.
jrinehart1
28 Posts
0
January 7th, 2010 11:00
Thank you all for your replies! It has been very helpful in my understanding of Networker and Avamar.
Much appreciated!
AllanW1
334 Posts
0
January 7th, 2010 12:00
Hi Albert,
In response to your licensing comments, we are currently investigating an alternative licensing model for NetWorker. Check out the main page and help us by taking the survey over on the right side.
"However it requires adding a dedup storage node to your configuration which does work however you are still bound to the Networker licensing schema. Which makes it very unattractive as one of the selling points behind avamar is unlimited clients (within capacity of the node) and unlimited integrations."
Thanks,
Allan