Block-based backup defaulting to Full everyday

can you post the debug logs of the current backup or the subsequent backup?

Have you rebooted the server since configuring it for block based backup?  When using BBB, incrementals will default to fulls until reboot occurs.

This may not be a reboot issue since the message snippet says that it had triggered an incremental backup.

I will reboot this weekend and let you know.

Forgot to get back to this. The server is able to do incrementals, sometimes. But many times it goes back to full in the middle of the week. The server is a physical server with a 12TB file volume (~3 million files) that is deduplicated by Windows 2012. When it reverts to Full, it takes about 26 hours to complete, so it runs all the next day and overlaps the window. Its backing up to a Data Domain.

Is there more than 1 device that the BBB can save to, ie multiple DD devices assigned to the save pool? I found that I needed to limit the backup to one Networker DD device only.

I am wondering if you ever found a fix for this?  We recently moved to block based backups and I am seeing the same issue.

Any help you can provide on the problem would be greatly appreciated.

Thanks in advance.

Hello techwom,

• Could you please describe the environment?

          - NetWorker server version and platform, NetWorker Client version and platform, Client OS and Memory size,...

         - Target device for those backups : AFTD, DDBoost ?

• Is the issue consistent or intermittent? If intermittent, is there a known trigger? Not enough or memory problems could be a reason for example.
• Did this work before? If yes , when did the issue first occur (what changed at that time)?
• Could you please run the backup command in debug mode ( -D2) and upload the logs?
• Are there any event viewer errors ?

Theory:

• NetWorker uses its own CBT (Change Block Tracking) mechanism to track changes on file system between backups.
• NetWorker has CBT map of all blocks on disk .The CBT map is set of bits. Every bit corresponds to single block on disk. CBT map is kept by NetWorker in memory.
• During normal work of file system on disk if block on disk is changed, NetWorker also changes corresponding bit in CBT bit map in memory.
• After the backup, all bits in the CBT map are set to „0” . It means all the blocks on disk are clean, not changed since the last backup. If block on a disk is changed than NetWorker knows about it and set the corresponding bit in CBT map to „1” .
• During the full backup NetWorker reads the whole disk block after block.
• During incremental backup NetWorker reads only changed blocks since last backup – those which corresponding value is „1”.
• If the host is rebooted, the first backup must be a full backup. The reboot is required to initialize the write tracker (WT) Driver. The write tracking driver maintains a bitmap to track changes (write I/O requests) for every block based backup volume (Fulls and Incrementals).The drive save in system memory, so when we reboot the client, the backup would become full -> by design
• Up to 38 incrementals can be created. After the 38th incremental a full backup is forced.

Regards,

--Liliana

• Networker Server version is 8.2.0.1 on Windows 2008 64-bit virtual server.
• Networker client version is 8.2.01 running on a a Windows 2008 64-bit virtual Microsft Cluster (shared drives are presented as physical RDM's).  32G of RAM.
• Target device is Data Domain using DDBoost
• Issue is pretty consistent.  Some drives will back up as incremental , but most will default to a full.
• Did not work before, always defaulting to full backups.
• Can you assit me with the command to run in debug mode?
• I am seeing the following in event viewer where Networker is concened:
  • Could not perform a VSS save operation. The volume GUID of current backup instance and the earlier backup instance for the save set 'Q:\' are different
  • Could not perform a VSS save operation. The volume GUID of current backup instance and the earlier backup instance for the save set 'E:\' are different

Hello techwom,

Are you using Virtual Synthetic Full (VSF) with Data Domain ? You can check that in NMC GUI in Group properties ( with Diagnostic View).

In that case regarding the BBB backups to DD, all the BBB incremental backups to DD shall be performed at Virtual Full.

What is the difference between the drives that have a successful incremental backup and the ones that didn't ?

Found that Windows 2008 x64 is not listed on the supported OS ( only 2008 R2) as well RDM devices in a MSCS should be checked if are supported configuration. Seems that VOLUME_GUID between the backups are not preserved . Not sure , but it seems they should be and in that case need to be investigated.

An incremental backup shifts to a full backup when the preceding incremental backup failed ,when you add a volume for the backup of the All save set, when you change the size of the volume,... Is there any of this happen ?

Can you upload the screenshots showing the configuration?

I would open a service request with EMC support for further investigation if the issue need an immediate attention.

Regards,

--Liliana

I'm glad you added that note Jack, because I recently had the same problem and found I had two target volumes which is one too many.

However, even after fixing that I got no INCR BBB until I upgraded the client agent from 8.2.2 to 8.2.3. I had a support case open with EMC, who have just confirmed in their tests that NW 8.2.2.6 won't do incrementals on BBB and 8.2.3.1 will. It's a previously unknown issue, there's a new KB on the way, and the solution is: upgrade or downgrade the client version away from 8.2.2 for BBB INCR.