Start a Conversation

Unsolved

This post is more than 5 years old

1076

January 26th, 2009 12:00

iscsi replication error

We try to replicate an iscsi lun but we got the follwing error:

nas_replicate -create iscsi1 -source -lun 1 -target iqn.1992-05.com.emc:ck2000733006820000-2 -destination -lun 1 -target iqn.1992-05.com.emc:ck2000728002810000-2 -interconnect id=30003

Warning 17455382635: version set out of space

The target lun size is near 1TB

here the destination :

Logical Unit 1 on target mg0301:
(Production) fsid=313 size=921600MB alloc=0MB dense read-only
path=/mg0301/fs313_T2_LUN1_CK200072800281_0000/fs313_T2_LUN1_CK200072800281_0000
replication=available destination
max_extension_size=6825MB
Healthy

here the source:

server_3 :
Logical Unit 1 on target mg0301:
(Production) fsid=427 size=921600MB alloc=61246MB dense read-write
path=/mg0301/fs427_T2_LUN1_CK200073300682_0000/fs427_T2_LUN1_CK200073300682_0000
replication=none
max_extension_size=6825MB
Healthy

storage pool size:

nas_pool -size clar_r5_performance -storage CK200073300682
id = 3
name = clar_r5_performance
used_mb = 1137012
avail_mb = 2160726
total_mb = 3297738
potential_mb = 318178


What can be the reason? Can you help me?

8.6K Posts

January 26th, 2009 13:00

How much space is free in the file systems for the source and the destination LUN ?

Does the src file system have 61.246MB free ?

Looks like it doesnt

17 Posts

January 26th, 2009 23:00

How can i check this?

I think this is real LUN size=921600MB

and

the alloc=61246MB is the used space of the LUN so it seems that there is enough free space.

What space will the snaps use? Can i check this somewhere?

I am a little bit confuse because a normal fs replication use its own savvol for snaps.

8.6K Posts

January 27th, 2009 00:00

correct

since this is a dense LUN the size=921600MB got reserved but currently only the alloc=61246MB blocks are actually used (allocated)

In order to replicate an ISCSI LUN the Celerra has to create a snapshot (two actually on src and dst)

For a dense LUN (non-virtually provisioned) LUN this means it will reserve as much space as needed in case all the blocks were changed, which is the alloc value
Its better worded in the Celerra Manager GUI - if you look at the properties of the LUN there it says "Space required for the next snapshot", which should display the same value as the alloc=.

yes, its very different from normal file system snapshots

we did implement it this way without a savvol so that customers can snap individual ISCSI LUNs and not just the whole file system

So you need to enlarge the src file system - the max_extension_size=6825MB
tells me there isnt enough space

The other option would be to use a virtually provisioned (sparse) LUN - there both the LUN and the snapshots to not reserve any space when created - they will just allocate as they go.
Currently you have to use the CLI to create a VP LUN - this will get added to the GUI in one of the next maintenance releases.

In case you're wondering - you cant "convert" a LUN from full to virtually provisioned - but you can replicate and change it this way.

regards
Rainer
P.S.: just curious - what does server_df for that src file system say?

8.6K Posts

January 29th, 2009 09:00

Did that help ?

Do you need more explanation ?

17 Posts

February 2nd, 2009 07:00

Hi Rainer,

this helps. The important hint was that iscsi do not use SavVol.
Now it works.

Thanks.

Br

Mario

2 Intern

 • 

20.4K Posts

February 2nd, 2009 20:00

so you just had to extend the file system on the source and the target ?

17 Posts

February 2nd, 2009 23:00

Yes, thats right.

Br

Mario

5 Practitioner

 • 

274.2K Posts

February 3rd, 2009 05:00

yes and no...In this scenario - the correct sizing of the file system should be:

2*Lun Size+(Number of Snaps (2)-1)*ChangeRate*Lun Size)+NumberofMounted(aka promoted snaps) 0

With numbers mentioned and the provisioning method ¿ it is ¿ assumpition of 20% change rate...
(1000*2)+200+0 =2200

It is VERY important to note that this is due to the fact that the luns AND the snapshots are fully provisioned. Ranier already went through the vp lun portion. You can also vp the snapshots. This is done by setting a global parameter in the nbs facility with the parameter name sparseTws. This will make ALL snapshots on a datamover sparse (aka VP). This should not be done without consideration..i.e - the need for a full "clone" would be prohibited here. If you did VP on the LUN and the Snapshot it would change the FS size to 468GBs from the full provisioned version of 2.2TBs. This is assuming no other snapshots, 68GB allocated and 20% change rate.

Message was edited by:
kzaw

2 Intern

 • 

20.4K Posts

February 3rd, 2009 05:00

Rainer,

so one would need to expand file system size by Replicated iSCSI LUN * 2 ..in order to replicate it ?

8.6K Posts

February 3rd, 2009 10:00

Hi Kevin,

sorry to amend this, but I think its important to understand the difference

You can also vp the snapshots. This is done by setting a global parameter in the nbs facility with the parameter name sparseTws.
This will make ALL snapshots on a datamover sparse (aka VP).


The nbs.sparseTWS param is *only* used when you promote a readonly ISCSI snapshot to read-write (hence the TemporaryWriteableSnapshot name)
In your formula that's the NumberofMountedSnaps

If you just replicate for DR without application integration or just want to break replication and change the destination LUN to rw you dont need this.

In most customer use scenario's an ISCSI snapshot is only temporarily promoted.
For example if you use EMC Replication Manager with Exchange it will automatically promote the snap and get a mount host to verify the Exchange database using eseutil.exe

Whether or not a (normal) readonly ISCSI snapshot reserves space *purely* depends on the "provisioning guarantee" of its source LUN.

If the source LUN is fully provisioned (aka dense) - which is the default - case then a snapshot will reserve as much space as needed to guarantee a full overwrite of whats used.

If a source LUN is virtually provisioned any snapshots of this will not reserve space up-front - they will allocate when they need it block-by-block

2 Intern

 • 

20.4K Posts

February 3rd, 2009 11:00


If the source LUN is fully provisioned (aka dense) -
which is the default - case then a snapshot will
reserve as much space as needed to guarantee a full
overwrite of whats used.


what do you mean what's used ? % of utilization within iSCSI LUN itself ? So let's say i have a 100G file system, within that file system i have a 90G iSCSI LUN (dense). iSCSI LUN itself is only 50% utilized. So when i snapshot this LUN do i need an extra 45G of space in my file system, bringing the total to 135G or do i need 180G ?


If a source LUN is virtually provisioned any
snapshots of this will not reserve space up-front -
they will allocate when they need it block-by-block


what happens if it keeps expanding and then file system runs out of space ? Does snapshot get suspended ?

Message was edited by:
dynamox

8.6K Posts

February 4th, 2009 03:00


what do you mean what's used ? % of utilization within iSCSI LUN itself ? So let's say i have a 100G
file system, within that file system i have a 90G´ iSCSI LUN (dense).
iSCSI LUN itself is only 50% utilized. So when i snapshot this LUN do i need an
extra 45G of space in my file system, bringing the total to 135G or do i need 180G ?


sorry, thats really hard to explain without a whiteboard

In your example you need 45GB free in the file system in order to create that snap.
After the snap creation 90+45=135GB would be reserved.

Basically what counts is how many blocks are "used" in the PLU or previous snap (the alloc= figure)
For whatever is used we reserve as much space to "backfill" the reserve

In other words: "The storage space needed to create a snap equals to the space
allocated to the PLU prior to the snap creation."

what happens if it keeps expanding and then file system runs out of space ? Does snapshot get suspended ?


no, you will get an error (unless you have configured automatic file system extention)
No Events found!

Top