And then following the -pRP flags for cp from the cp man page
-p Cause cp to preserve the following attributes of each source file in the copy: modification time, access time, file flags, file mode, user ID, and group ID, as allowed by permissions.
If the user ID and group ID cannot be preserved, no error message is displayed and the exit value is not altered.
If the source file has its set-user-ID bit on and the user ID can- not be preserved, the set-user-ID bit is not preserved in the copy's permissions. If the source file has its set-group-ID bit on and the group ID cannot be preserved, the set-group-ID bit is not preserved in the copy's permissions. If the source file has both its set-user-ID and set-group-ID bits on, and either the user ID or group ID cannot be preserved, neither the set-user-ID nor set- group-ID bits are preserved in the copy's permissions.
-P If the -R option is specified, no symbolic links are followed.
This is the default.
-R If source_file designates a directory, cp copies the directory and the entire subtree connected at that point. If the source_file ends in a /, the contents of the directory are copied rather than the directory itself. This option also causes symbolic links to be copied, rather than indirected through, and for cp to create spe- cial files rather than copying them as normal files. Created directories have the same mode as the corresponding source direc- tory, unmodified by the process' umask.
Note that cp copies hard linked files as separate files. If you need to preserve hard links, consider using tar(1), cpio(1), or pax(1) instead.
EMCOPY, on a windows machine with both shares mounted.
This will copy both the files and the ACL's, pending inherited permissions from the upper level directory, and or flags set with EMCOPY
Or
You can also use a local sync to complete this, however there are some caveats to this. As the target directory of the sync must be empty, or any data in the target location will be over written. Also know that the target directory will be Read-Only until the sync is broken.
Not sure of which OneFS version you are currently running the following output is and example of the syncIQ configuration for 7.2.0.1.
isi720x-1# isi sync reports view --policy=Local_Sync --job-id=1 Policy Name: Local_Sync Job ID: 1 Start Time: 2015-06-25T11:35:02 End Time: 2015-06-25T11:35:34 Action: run State: finished ID: 1-Local_Sync Policy ID: 809f78c46b6cc4f4e7ec3437a898dfad Sync Type: initial Duration: 32s Errors: - Source Directories Visited: 1 Source Directories Deleted: 0 Num Retransmitted Files: 0 Retransmitted Files: - Total Files: 0 New Files Replicated: 0 Source Files Deleted: 0 Files Changed: 0 Target Files Deleted: 0 Up To Date Files Skipped: 0 User Conflict Files Skipped: 0 Error Io Files Skipped: 0 Error Net Files Skipped: 0 Error Checksum Files Skipped: 0 Bytes Transferred: 1273 Total Network Bytes: 1273 Total Data Bytes: 703 File Data Bytes: 703 Sparse Data Bytes: 0 Target Snapshots: SIQ-Failover-Local_Sync-2015-06-25_11-35-34 Total Phases: 2 Phases Phase: STF_PHASE_TW Start Time: 2015-06-25T11:35:21 End Time: 2015-06-25T11:35:25
Phase: STF_PHASE_IDMAP_SEND Start Time: 2015-06-25T11:35:25 End Time: 2015-06-25T11:35:30
Subreports: Policy Name Job ID Subreport ID Start Time End Time Action State ------------------------------------------------------------------------------------------- Local_Sync 1 1 2015-06-25T11:35:02 2015-06-25T11:35:34 run finished ------------------------------------------------------------------------------------------- Total: 1
Moving and copying to SyncIQ source should work fine as shown by the steps below:
Showing Sync Source and Target:
Source: isi720x-1# isi sync policies list Name Path Action Enabled Target --------------------------------------------------- Local_Sync /ifs/data/dir sync Yes localhost --------------------------------------------------- Total: 1
Target: isi720x-1# isi sync target list Name Source Target Path Last Job State FOFB State ------------------------------------------------------------------ Local_Sync isi720x /ifs/data/empty finished writes_disabled ------------------------------------------------------------------ Total: 1
Creating a random directory: isi720x-1# mkdir test_move
show path: isi720x-1# pwd /ifs/data
Show file exsists: isi720x-1# ls |grep test_move test_move
Moving test_move to source sync directory: isi720x-1# mv test_move ./dir
Change directory to dir: isi720x-1# cd dir
Show that i'm in /ifs/data/dir (source of sync): isi720x-1# pwd /ifs/data/dir
Show that test_move was move sucessfully into /ifs/data/dir: isi720x-1# ls |grep test_move test_move
What is the error you are seeing when attempting to mv files to the sync source, as there is likely something outside of the SyncIQ restricting this.
if you ever tried failover/failback you'd have protection domain created on the source cluster by the mirror policy
you can check with the command above, and also with
isi sync targer lists
From what I learned from support even if the domain is "SyncIQ,Writable" mv to it is not allowed
Workaround is to go to the target cluster, delete the *_mirror policy - this will delete the "SyncIQ,Writable" protection domain in your source cluster and allow you to mv into that directory.
The last few things I can take a look at before I recommend opening a case, to have this looked at further would be
# isi sync reports list -v
Also grab
# isi sync target list
And attempt to copy the file instead of mv
As stated above the only reasons I can think of at a moments notice inside of SyncIQ that this would be read-only is a partial failover/failback.
Other wise there is something else effecting this directory, out side of SyncIQ
Can you touch a file inside the directory? If it's read-only I would suspect not.
Any chance you could remove the quota and try again? This is sheerly out of curiosity. I tested it locally with and with out a quota on the directory and for both I was able to mv a file into a SyncIQ source without a problem.
One other thing to try would be to mkdir the directory in the file system instead of /root. make a directory in /ifs instead. I did test this as well and was still able to mv the directory from /root to /ifs/data/dir.
ahh darn, /ifs/data/n2isilonpoc/ts is target of my loopback replication and that policy is still in place ..hense read-only, that's what i get for monkeying around with virtual isilon too much . I think i was trying to test SyncIQ where one directory was target of loopback replication and source for another SyncIQ policy. That is not supported but i did not clean up the policy.
Stdekart
104 Posts
0
June 25th, 2015 09:00
chjatwork,
There are a few option for you, ordered from simplest to more complex:
cp should work as shown by the mv man page as an example command (denoted by ***):
As the rename(2) call does not work across file systems, mv uses cp(1) and rm(1) to accomplish the move. The effect is equivalent to:
rm -f destination_path && \
***cp -pRP source_file destination && \
rm -rf source_file
And then following the -pRP flags for cp from the cp man page
-p Cause cp to preserve the following attributes of each source file
in the copy: modification time, access time, file flags, file mode,
user ID, and group ID, as allowed by permissions.
If the user ID and group ID cannot be preserved, no error message
is displayed and the exit value is not altered.
If the source file has its set-user-ID bit on and the user ID can-
not be preserved, the set-user-ID bit is not preserved in the
copy's permissions. If the source file has its set-group-ID bit on
and the group ID cannot be preserved, the set-group-ID bit is not
preserved in the copy's permissions. If the source file has both
its set-user-ID and set-group-ID bits on, and either the user ID or
group ID cannot be preserved, neither the set-user-ID nor set-
group-ID bits are preserved in the copy's permissions.
-P If the -R option is specified, no symbolic links are followed.
This is the default.
-R If source_file designates a directory, cp copies the directory and
the entire subtree connected at that point. If the source_file
ends in a /, the contents of the directory are copied rather than
the directory itself. This option also causes symbolic links to be
copied, rather than indirected through, and for cp to create spe-
cial files rather than copying them as normal files. Created
directories have the same mode as the corresponding source direc-
tory, unmodified by the process' umask.
Note that cp copies hard linked files as separate files. If you
need to preserve hard links, consider using tar(1), cpio(1), or
pax(1) instead.
Example:
touch a file
isi720x-1# touch testfile
Validate file was made
isi720x-1# ls
testfile
Check permissions
isi720x-1# ls -le testfile
-rw-r--r-- 1 root wheel 0 Jun 25 11:49 testfile
OWNER: user:root
GROUP: group:wheel
SYNTHETIC ACL
0: user:root allow file_gen_read,file_gen_write,std_write_dac
1: group:wheel allow file_gen_read
2: everyone allow file_gen_read
Change owner of the file to an AD user
isi720x-1# chown MNELITE\\sdekart testfile
Also manually add in an additional ACL for test purposes
isi720x-1# chmod +a user MNELITE\\sleef allow dir_gen_all testfile
Check permissions
isi720x-1# ls -le testfile
-rw-rwxr-- + 1 MNELITE\sdekart wheel 0 Jun 25 11:55 testfile
OWNER: user:MNELITE\sdekart
GROUP: group:wheel
0: user:MNELITE\sleef allow file_gen_all
1: user:MNELITE\sdekart allow file_gen_read,file_gen_write,std_write_dac
2: group:wheel allow file_gen_read
3: everyone allow file_gen_read
Use cp -pRP to copy the file
isi720x-1# cp -pPR testfile testfile2
Check permissions on the new file.
isi720x-1# ls -le testfile2
-rw-rwxr-- + 1 MNELITE\sdekart wheel 0 Jun 25 11:55 testfile2
OWNER: user:MNELITE\sdekart
GROUP: group:wheel
0: user:MNELITE\sleef allow file_gen_all
1: user:MNELITE\sdekart allow file_gen_read,file_gen_write,std_write_dac
2: group:wheel allow file_gen_read
3: everyone allow file_gen_read
Or
EMCOPY, on a windows machine with both shares mounted.
This will copy both the files and the ACL's, pending inherited permissions from the upper level directory, and or flags set with EMCOPY
Or
You can also use a local sync to complete this, however there are some caveats to this. As the target directory of the sync must be empty, or any data in the target location will be over written. Also know that the target directory will be Read-Only until the sync is broken.
Not sure of which OneFS version you are currently running the following output is and example of the syncIQ configuration for 7.2.0.1.
isi720x-1# isi sync policies view --policy=Local_Sync
ID: 809f78c46b6cc4f4e7ec3437a898dfad
Name: Local_Sync
Path: /ifs/data/dir
Action: sync
Enabled: Yes
Target: localhost
Description: Sync local from one directory to another
Check Integrity: Yes
Source Include Directories: -
Source Exclude Directories: -
Source Subnet: -
Source Pool: -
Source Match Criteria:
Target Path: /ifs/data/empty
Target Snapshot Archive: No
Target Snapshot Pattern: SIQ_%{SrcCluster}_%{PolicyName}_%Y-%m-%d_%H-%M
Target Snapshot Expiration: Never
Target Snapshot Alias: SIQ_%{SrcCluster}_%{PolicyName}
Target Detect Modifications: Yes
Source Snapshot Archive: No
Source Snapshot Pattern:
Source Snapshot Expiration: Never
Schedule: Manually scheduled
Log Level: notice
Log Removed Files: No
Workers Per Node: 3
Report Max Age: 1Y
Report Max Count: 2000
Force Interface: No
Restrict Target Network: No
Target Compare Initial Sync: No
Disable Stf: No
Expected Dataloss: No
Disable Fofb: No
Disable File Split: No
Changelist creation enabled: No
Resolve: -
Last Job State: unknown
Last Started: -
Last Success: -
Password Set: No
Conflicted: No
Has Sync State: No
And out put from the job completing
isi720x-1# isi sync reports view --policy=Local_Sync --job-id=1
Policy Name: Local_Sync
Job ID: 1
Start Time: 2015-06-25T11:35:02
End Time: 2015-06-25T11:35:34
Action: run
State: finished
ID: 1-Local_Sync
Policy ID: 809f78c46b6cc4f4e7ec3437a898dfad
Sync Type: initial
Duration: 32s
Errors: -
Source Directories Visited: 1
Source Directories Deleted: 0
Num Retransmitted Files: 0
Retransmitted Files: -
Total Files: 0
New Files Replicated: 0
Source Files Deleted: 0
Files Changed: 0
Target Files Deleted: 0
Up To Date Files Skipped: 0
User Conflict Files Skipped: 0
Error Io Files Skipped: 0
Error Net Files Skipped: 0
Error Checksum Files Skipped: 0
Bytes Transferred: 1273
Total Network Bytes: 1273
Total Data Bytes: 703
File Data Bytes: 703
Sparse Data Bytes: 0
Target Snapshots: SIQ-Failover-Local_Sync-2015-06-25_11-35-34
Total Phases: 2
Phases
Phase: STF_PHASE_TW
Start Time: 2015-06-25T11:35:21
End Time: 2015-06-25T11:35:25
Phase: STF_PHASE_IDMAP_SEND
Start Time: 2015-06-25T11:35:25
End Time: 2015-06-25T11:35:30
Subreports:
Policy Name Job ID Subreport ID Start Time End Time Action State
-------------------------------------------------------------------------------------------
Local_Sync 1 1 2015-06-25T11:35:02 2015-06-25T11:35:34 run finished
-------------------------------------------------------------------------------------------
Total: 1
dynamox
9 Legend
•
20.4K Posts
0
June 25th, 2015 10:00
i use synciq to make copies, not move data.
Stdekart
104 Posts
0
June 25th, 2015 11:00
chjatwork,
I appear to have read a little more into the question than I should have.
In short, mv will keep windows ACL's and owners. (see first option for more info on what mv effectively does and the flags used)
You can also see the man page for mv, on an Isilon cluster.
dynamox
9 Legend
•
20.4K Posts
0
June 26th, 2015 19:00
not sure if this is a bug or what but i can't mv folder where my target folder is replicated via SyncIQ. Any idea Shane ?
chjatwork
2 Intern
•
356 Posts
0
June 27th, 2015 07:00
Shane,
I will test the move command and let you know.
Stdekart
104 Posts
0
June 29th, 2015 06:00
Dynamox,
Target folders for SyncIQ are read-only. Meaning you can't mv or cp files into the target directory with out first breaking the sync.
Covered in the 7.2 CLI guide. Page 532 under (Replication Policies and Jobs) paragraph 3.
https://support.emc.com/docu56048_OneFS-7.2-CLI-Administration-Guide.pdf?language=en_US
dynamox
9 Legend
•
20.4K Posts
0
June 29th, 2015 12:00
sorry, i meant i can mv into my SyncIQ source directory.
Stdekart
104 Posts
0
June 29th, 2015 12:00
dynamox,
Moving and copying to SyncIQ source should work fine as shown by the steps below:
Showing Sync Source and Target:
Source:
isi720x-1# isi sync policies list
Name Path Action Enabled Target
---------------------------------------------------
Local_Sync /ifs/data/dir sync Yes localhost
---------------------------------------------------
Total: 1
Target:
isi720x-1# isi sync target list
Name Source Target Path Last Job State FOFB State
------------------------------------------------------------------
Local_Sync isi720x /ifs/data/empty finished writes_disabled
------------------------------------------------------------------
Total: 1
Creating a random directory:
isi720x-1# mkdir test_move
show path:
isi720x-1# pwd
/ifs/data
Show file exsists:
isi720x-1# ls |grep test_move
test_move
Moving test_move to source sync directory:
isi720x-1# mv test_move ./dir
Change directory to dir:
isi720x-1# cd dir
Show that i'm in /ifs/data/dir (source of sync):
isi720x-1# pwd
/ifs/data/dir
Show that test_move was move sucessfully into /ifs/data/dir:
isi720x-1# ls |grep test_move
test_move
What is the error you are seeing when attempting to mv files to the sync source, as there is likely something outside of the SyncIQ restricting this.
dynamox
9 Legend
•
20.4K Posts
0
June 29th, 2015 12:00
OneFS 7.1.0.1 (virtual)
Stdekart
104 Posts
0
June 29th, 2015 13:00
dynamox,
So this is a virtual are you by chance testing fail over fail back?
As a partial fail back can leave you in a state where the source is read-only.
The following command might give me a little bit better idea of where the current Sync is sitting.
This was run from the cluster I just mv'd a directory into the source directory for reference.
isi720x-1# isi_classic domain list -lw
ID | Root Path | Type | Override Date | Def. Ret. | Min. Ret. | Max. Ret. | Autocommit Period | Priv. Del.
------+-----------------+--------+---------------+-----------+-----------+-----------+-------------------+-----------
65537 | /ifs/data/empty | SyncIQ | None | None | None | None | None | Off
Shane
bhalilov1
1 Rookie
•
114 Posts
0
June 29th, 2015 13:00
Dynamox,
if you ever tried failover/failback you'd have protection domain created on the source cluster by the mirror policy
you can check with the command above, and also with
isi sync targer lists
From what I learned from support even if the domain is "SyncIQ,Writable" mv to it is not allowed
Workaround is to go to the target cluster, delete the *_mirror policy - this will delete the "SyncIQ,Writable" protection domain in your source cluster and allow you to mv into that directory.
dynamox
9 Legend
•
20.4K Posts
0
June 29th, 2015 13:00
not testing failover/failback
Stdekart
104 Posts
0
June 29th, 2015 14:00
dynamox,
The last few things I can take a look at before I recommend opening a case, to have this looked at further would be
# isi sync reports list -v
Also grab
# isi sync target list
And attempt to copy the file instead of mv
As stated above the only reasons I can think of at a moments notice inside of SyncIQ that this would be read-only is a partial failover/failback.
Other wise there is something else effecting this directory, out side of SyncIQ
Can you touch a file inside the directory? If it's read-only I would suspect not.
Any chance you could remove the quota and try again? This is sheerly out of curiosity. I tested it locally with and with out a quota on the directory and for both I was able to mv a file into a SyncIQ source without a problem.
One other thing to try would be to mkdir the directory in the file system instead of /root.
make a directory in /ifs instead. I did test this as well and was still able to mv the directory from /root to /ifs/data/dir.
/data/dir.
Stdekart
104 Posts
0
June 29th, 2015 14:00
dynamox,
Glad to help!
dynamox
9 Legend
•
20.4K Posts
0
June 29th, 2015 14:00
ahh darn, /ifs/data/n2isilonpoc/ts is target of my loopback replication and that policy is still in place ..hense read-only, that's what i get for monkeying around with virtual isilon too much
. I think i was trying to test SyncIQ where one directory was target of loopback replication and source for another SyncIQ policy. That is not supported but i did not clean up the policy.
Thank you Shane.