Not Archiving Messages From Certain Date Range

Hi there,

it's funny that only a certain date range cannot be archived. Can you be more specific on how long that is and how long in the past? I assume you run an archive task in EmailXtract before!

Search in the archive and its results depends a lot on the index. You can have a perfect archive and still not find the info since the index is corrupt or shows errors. - These messages can be resolved by opening a shortcut, although the index search doesn't bring them up.
-> Please verify in the EX Administrator and in HealthCheck.

When you archive the messages (the specific time frame), what does the "message detail" log selected in EmailXtract say? There must be a reason given as to why the message is touched or left alone. The same is true for shortcutting messages!!!
-> Please run an archive task and search in the log for messages you deem archivable that were skipped.

Having shortcutted messages without prior successful archiving wasn't perhaps the best thing to do; those messages are basically lost and can only be restored to Exchange and then archived again (in case multiple users got the same message and so the mails can be archived from other mailboxes, the shortcuts for the problematic user can be resolved later on).

When one runs an archive task, the already archived messages are not checked against SQL and therefore the process is faster. You can create a registry entry in HKLM\Software\OTG\EmailXtract_Exchange or Notes with "DebugEnabled" (DWORD), set this to "1" and restart the EmilXtract service. The archiving tasks shows the tab "Hidden settings" and you'll be able to select "Skip already archived check". This archive task is then slower but checks every message if it was archived previously!
-> If this doesn't solve the problem I wouldn't hesitate to contact support for this.
-> Don't forget to set this to "0" later on again.

Hope this info helps and you'll find the problem!
Best regards, Jochen.

Thank you for your advice...I didn't remember how to activate the Hidden Tab, so that helped me immensely. Yes, shortcutting the mailbox was a bad idea, but it was just for 1 mailbox and spanned only 2 days, so I don't think that will be too devestating. I currently am rerunning the archive task using the hidden tab, so hopefully that will solve it. I will keep you posted.

Hello to all,

Kauanoe, Have you activate Journaling? have you rules (inclusion/exclusion) in your folders in EmailXtender Administrator?

If you have Journaling, or by archive task too, when a e-mail goes by msmq, emailXtender check that rules. This e-mail is marked as processed. If this e-mail were discarted to archive, this e-mail don't to be reprocessed in the future, you need activate Hidden Tab as you said the companion.

Regards.

Which version of EX are you running? When you say that it bypasses those messages and never gets archived, what do you see in the log when it skips the message?

Make sure everything looks good in SQL. If there was a problem with the DB you wouldn't be able to archive anything at all but I'd still check it out. I was having strange problems with one client using Xtract and when I looked at the DB in SQL it was suspect.

Again, it would affect everything, not just a specific date range.

That having been said, if you ran a shortcut task agains everything in this user's mailbox then you should just move on to a different user. All of the actual email is now gone and the archive task won't archive a shortcut. Know what I mean?

Journaling has been initiated from the beginning. After running the archive task once more with the hidden tab exposed, it gathered a little mor email (ran it for the specific date range), but not everything. How I found that out was to re-run the shortcut task for the date range (and only shortcut email already archived), and there were a lot of email that still did not get shortcutted. I also try to look for these email in the archive (using the search client), and nothing.

Anyone have anymore ideas?

Message Center and Indexing were installed on shared disks. SQL is on a completely separate server.

Did EVERYTHING get installed to a shared disk? Is SQL clustered on the same two servers?

Hi there,

is the date range that is apparently not archiving within the dates that you started journaling? Are these older messages?

If you started journaling beforehand, something is not right since the messages should have been archived previously.

It's paramount to understand -when running a manual archiving task- why the messages are not archived. What does the detailed EmailXtract log say??? What's in the OTG event log? Is jouranling really working?

Hope we'll figure this out soon,
Jochen.

Journaling was the first thing that we initiated...archiving of older email came later.

The background is that EX is installed in a cluster. Upon failover, something happened where the services were not completely coming online...and this seemed to have been going on during the dates in question. We finally got the failover to work properly (after reinstalling the program, but NO deletions when uninstalling).

This is an example of what the log indicates when doing an archive task:

11:09:59: Skipping (Archived) Class: IPM.Note Date: 3/11/2008 - 23:24:6 From: "Snowcreek Vacation Rentals" Subject: Snowcreek Resort Confirmation MsgID: 49288ED0D9E2BC3279F73101

11:10:00: Skipping (Archived) Class: IPM.Note Date: 3/11/2008 - 23:23:57 From: "Snowcreek Vacation Rentals" Subject: Snowcreek Resort Confirmation MsgID: 49288ED5BC99DFA38F5D27FC

This is an example of what the log indicates when doing a shortcut task:

11:03:36: Skipping (Not Previously Archived -> Class: IPM.Note Date: 3/11/2008 - 23:24:6 From: "Snowcreek Vacation Rentals" Subject: Snowcreek Resort Confirmation MsgID: 49288ED0D9E2BC3279F73101

11:03:36: Skipping (Not Previously Archived -> Class: IPM.Note Date: 3/11/2008 - 23:23:57 From: "Snowcreek Vacation Rentals" Subject: Snowcreek Resort Confirmation MsgID: 49288ED5BC99DFA38F5D27FC

The above are the same 2 messages, and when I do a search using the EX Search Client it does not show up. I even try to put the message id, and that gains no hit either.

Also, to clarify, there are no rules set up...all email are being gathered.

Thanks.

No...everything was installed on the shared disks...

So, are you saying that during the install ONLY the message Center and Indexing were put on shared disks? I'm hoping that when you say "Message Center" you mean everything BUT indexing.

The reason I bring this up is because in the past two months I have been sent out to repair two cluster installs that weren't done properly. They put the container drive, indexes and msg center on shared disk but forgot or didn't know that the program files location itself has a bunch of temporary holding locations that need to be on shared disk as well.

The install doc is very vague about this but it does mention it.

For example, let say you chose to put the container files, msgCenter and indexes on shared drives during the install. But installed EX itself to the local D: drive. If an archive task was running and the server failed over, the temporary files for the volume would still be in the installdir\EmailVault_Index_archive\StorageDir directory on the D drive of the primary node. Then while on the secondary node it could possible close the volume. This might result in what you are seeing now but I have never seen it myself so I can't say for sure.

At this point, SQL would think that the volumes were in the container file but really they are still sitting in the StorageDir of the primary server. Same goes for other temp directories like the exIndexTemp.

I also wanted to add 1 more thing. When this issue was noticed (meaning a shortcut task was run against 1 mailbox for the specified period and not all messages were shortcutted), the first thing we did was to dispose the indexes for that entire month. I verified that all containers were closed for that month and once it was disposed added it back into EX Admin from Lost & Found. Still those messages were nowhere in the archive when doing a search. (There were no errors in the re-indexing process).

Do you think I should dispose the monthly data again, then run an archive task from scratch? Then, when that process is complete, add the containers back?

Hi Kauanoe,

do you still have the problem?

I would only do what you've mentioned if I knew it would help. But I am not sure if disposing and readding the containers will actually help (and why). I guess the messages were added to SQL but were not added to the EX messages during the fail overs...

In your shoes I would have contacted EMC or the support provider!

Keep well,
Jochen.

I did a simulation of this in our test environment, and it seemed to have addressed the issue. So I did this to production, and it was able to gather almost 3000 more email that were missing from the archive to begin with. When I ran the same archive task without disposing the monthly data, these messages were bypassed, so that worked out well.

I would've contacted support, but wanted to see from the forum if anyone experienced this before.