UNSOLVED

landog1

updated

15 years ago

L

landog1

1 Rookie

•

96 Posts

0

2180

November 6th, 2011 06:00

celerra deduplication

We have enabled deduplication on a new file system. 

We are copying older checkpoints from another Celerra to it.  We think we will then have a rather substantial dedupliction rate.

We have set these dedupliction parameters: 

Access Time: 0
Modification Time: 0
Minimum Scan Interval: 1 day

It seems that whenever we look at the file system propertires, it says:

Status:

Scanning (95% complete)

   ...even though the date of the last scan changes.

It also shows:

As of last file system scan (Saturday, November 5, 2011 8:55:50 AM EDT)

Files scanned: 359,946

Files deduped: 218,281 (62.0% of total files)

Original data size: 569,828 MB (6% of current file system capacity)

Space saved: 0 MB (0% of original data size)

If 62% of the files are deduped, why does it show that no space is saved?

Thank you for any explanation you may provide.

  • Rainer_EMC

    6 Operator

    •

    8645 Posts

    512

    0

    Posted November 6th, 2011 15:00

    Take a look at the release notes for the latest release

    I think there was an issue that Space Saved wasn’t displayed until the scan was completed.

  • landog1

    1 Rookie

    •

    96 Posts

    512

    0

    Posted November 7th, 2011 06:00

    Whenever we look at the file system propertires, it says:

    Status:  Scanning (95% complete)

    This has been the case for at least two days, now.  Here is the latest view:

    Status: Scanning (95% complete) 

    Statistics: As of last file system scan (Saturday, November 5, 2011 8:55:50 AM EDT)

    Files scanned: 794,812

    Files deduped: 218,281 (62.0% of total files)

    Original data size: 569,828 MB (6% of current file system capacity)

    Space saved: 0 MB (0% of original data size)

    ... does the scan ever complete?

  • Peter_EMC

    674 Posts

    512

    0

    Posted November 7th, 2011 06:00

    check the server_log for error messages regarding dedup.

    Often seen, Savvol NoSpace ....

  • landog1

    1 Rookie

    •

    96 Posts

    512

    0

    Posted November 7th, 2011 07:00

    These all came up empty..

    server_log server_2 | grep NoSpace

    server_log server_2 | grep Savvol

    server_log server_2 | grep error

    server_log server_2 | grep dedupe

    How can we know whether deduplication is working?

    Thanks...

  • Rainer_EMC

    6 Operator

    •

    8645 Posts

    512

    0

    Posted November 7th, 2011 14:00

    Open a service request

    If you arent running a recent software release then I would suggest to upgrade

  • Rainer_EMC

    6 Operator

    •

    8645 Posts

    512

    0

    Posted November 7th, 2011 15:00

    Access Time: 0

    Modification Time: 0

    I would also suggest to start with the default values there.

    Once that works and you understand what they mean you can always change them later.

    IMHO setting them to 0 doesnt make sense and causes more work than what you gain in additional space savings.

    See the deduplication white paper on what it means where a deduplicated file gets changed and think whether it makes sense to deduplicate files that might be temp. anyway.

  • landog1

    1 Rookie

    •

    96 Posts

    512

    0

    Posted November 8th, 2011 04:00

    Rainer_EMC:  "Open a service request"

    I came here after having opened a service request.  The service rep said that nothing would deduplicate until after 15 days.  He couldn't explain his assertion in light of the parameters that were set for Access Time, Modification Time, and Minimum Scan Interval.

    I won't rant here about EMC support, but if he was right, at least he could have explained what he meant, or how my understanding was faulty..

  • landog1

    1 Rookie

    •

    96 Posts

    512

    0

    Posted November 8th, 2011 04:00

    Rainer_EMC: "I would also suggest to start with the default values there.   Once that works and you understand what they mean you can always change them later.   IMHO setting them to 0 doesnt make sense and causes more work than what you gain in additional space savings."

    Here is my logic.  We have older checkpoints that we would like to migrate off of our production array.  We have created a file system on our secondary Celerra for this data.  If I copy several of the monthly checkpoints to our secondary Celerra without dedupliction, they would take a considerable amount of storage space.  As they should contain a very large percentage of identical files, they should deduplicate well.   As of today, I have copied seven of the checkpoints to the secondary Celerra. 

    These files will remain static.  I would like to not wait 15 days for dedupliction, as we want to copy more checkpoints to the file system. 

    If my settings are incorrect, what are the proper settings so that the dedupliction process starts immediately? 

    Thanks for your assistance.

  • Rainer_EMC

    6 Operator

    •

    8645 Posts

    65

    0

    Posted November 8th, 2011 08:00

    That depends on whether your method of copying the data preserves the atime / mtime

    I would first wait for a successful deduplication run with the default params and then decrease them.

    By getting less file deduped in the first run you lower the chance to dedupe stopping because file system or savvol full.

    If you know exactly that you want a complete directory deduped (which really means compressed and single instanced) you can just as well use a Windows client and set the compressed attribute – that doesn’t use the scheduled scan and the atime/mtime options.

    Rainer

  • Rainer_EMC

    6 Operator

    •

    8645 Posts

    512

    0

    Posted November 8th, 2011 08:00

    I would insist on having the log files analyzed by support

    Deduplication doesn’t just stop without leaving a statement in the logs.

    If you are not on the latest release I would strongly suggest to upgrade – no sense in chasing problems that have already been fixed.