Unsolved
This post is more than 5 years old
10 Posts
0
3130
ScaleIO: Set SDC Performance Profile and Set SDC Name Failed
Resolved: I did not set up gatewayUser.properties correctly on initial installation of ScalieIO using CSV file. I had to edit the driver_sync.conf file and then restart the scini service. Below is edited driver_synch.conf file.
###############################################################################
# driver_sync Configuration file
# Everything after a '#' until the end of the line is ignored
###############################################################################
# Repository address, prefixed by protocol
#repo_address = sftp://localhost/path/to/repo/dir
#repo_address = ftp://localhost/path/to/repo/dir
#repo_address = file://local/path/to/repo/dir
repo_address = ftp://QNzgdxXix:Aw3wFAwAq3@ftp.emc.comAw3wFAwAq3@ftp.emc.com
# Repository user (valid for ftp/sftp protocol)
#repo_user = scini
repo_user = admin
# Repository password (valid for ftp protocol)
#repo_password = scini
repo_password = password
# Local directory for modules
local_dir = /bin/emc/scaleio/scini_sync/driver_cache/
# User's RSA private key file (sftp protocol)
user_private_rsa_key = /bin/emc/scaleio/scini_sync/scini_key
# Repository host public key (sftp protocol)
repo_public_rsa_key = /bin/emc/scaleio/scini_sync/scini_repo_key.pub
# Should the fetched modules' signatures be checked [0, 1]
module_sigcheck = 0
# EMC public signature key (needed when module_sigcheck is 1)
#emc_public_gpg_key = /bin/emc/scaleio/scini_sync/emc_key.pub
emc_public_gpg_key = /home/administrator/ScaleIO_2.0.0.2_Complete_Linux_SW_Download/ScaleIO_2.0.0.2_GPG-RPM-KEY_Download/RPM-GPG-KEY-ScaleIO_2.0.7120.0
# Sync pattern (regular expression) for massive retrieve
sync_pattern = .*
-----------------------------------------------------
ScaleIO v2.0.0.2
Process:
Created Complete Installation CSV topology file.
Logged into IM server and ScaleIO Java GUI
Uploaded Installation Packages (MDM, SDSs, SDC, XCACHE, LIA modules)
Uploaded Installation CSV topology file
Initiated Installation
Install Phase complete: 11 commands completed successfully
Configuration Phase Completed: 0 aborted, 2 failed and 25 completed
Commands that failed details: set SDC Performance Profile failed and set SDC name failed
Debug:
- Found in the file /bin/emc/scaleio/scini_sync/drive_sync.log
- § driver_sync.sh START Wed Jan 18 ~
- § /usr/bin/sftp
- Error (sftp): File /bin/emc/scaleio/scini_sync/scinit_key must exist
- § Driver_sync.sh END Wed Jan18 ~
- Found in the file /opt/emc/scaleio/gateway/logs# cat scaleio-trace.log | grep "Error Failed"
- 2017-01-18 18:45:27,242 [executor-1] ERROR c.e.s.s.d.i.commands.SetSdcName - Error Failed to set SDC name to SDC1_S4 due to Could not find the SDC executing command .SetSdcName (retry) : com.emc.s3g.scaleio.im.services.installation.configurators.CommandServiceMdmConnection.setSDCName(CommandServiceMdmConnection.java:6147)/opt/emc/scaleio/lia/logs/emc-scaleio-lia-install.log
I guess the problem is the SDC and XCACHE driver(s) not installed which is the result that Ubuntu 14.04 kernel specific driver can't be found and therefore not installed. I read the section "Deploying on Ubuntu server" but do not understand it very well. Also some of the directories and files mentioned in that section (e.g., driver_sync.sh) are not even present until after installing and configuring the ScaleIO cluster.
I found and downloaded the correct SDC and XCACHE drivers (.tars) for my specific Ubuntu kernel version 4.4.0-53-generic form the EMC ftp link. The two files where scini.tar (scinit.siob, scini,sig) and xcache.tar (xcache.siob, xcache.sig).
Questions:
1. Is there a way I can edit the driver_sync.conf file to use local repository where tar files and or .siob and .sig files are located to successful install the SDC on the Gateway node? Or is there a manual procedure to install SDC on the Gateway manually. Below is my attempt to edit the driver_sync.conf file.
###############################################################################
# driver_sync Configuration file
# Everything after a '#' until the end of the line is ignored
###############################################################################
# Repository address, prefixed by protocol
#repo_address = sftp://localhost/path/to/repo/dir
#repo_address = ftp://localhost/path/to/repo/dir
repo_address = file://home/administrator
# Repository user (valid for ftp/sftp protocol)
repo_user = scini
# Repository password (valid for ftp protocol)
repo_password = scini
# Local directory for modules
local_dir = /bin/emc/scaleio/scini_sync/driver_cache/
# User's RSA private key file (sftp protocol)
user_private_rsa_key = /bin/emc/scaleio/scini_sync/scini_key
# Repository host public key (sftp protocol)
repo_public_rsa_key = /bin/emc/scaleio/scini_sync/scini_repo_key.pub
# Should the fetched modules' signatures be checked [0, 1]
module_sigcheck =0
# EMC public signature key (needed when module_sigcheck is 1)
emc_public_gpg_key = /bin/emc/scaleio/scini_sync/emc_key.pub
# Sync pattern (regular expression) for massive retrieve
sync_pattern = .*
2. At this point do I need to start over on ScaleIO install and configuration? If so how do I clear existing configuration on the three MDM/SDS nodes and one Gateway/SDC node?
---------------------------- Added 1-25-2017 -------------------------------------
Note: scini not running or install
- S4:~# service scini status
- scini driver is not running
- S4:~# service scini restart
- *
- ###############################################################################
- Verifying scini.ko against current system kernel.
- Module kernel version: 4.4.0-31-generic
- Running kernel version: 4.4.0-53-generic
- Current scini.ko seems to be incompatible with your kernel.
- Will attempt to fetch from module repository.
- driver_sync.sh START - Tue Jan 24 15:35:03 MST 2017
- /bin/emc/scaleio/scini_sync/driver_sync.sh: line 120: repo_address: unbound variable
- Error: Unable to fetch scini.ko. Will try to load the existing module...
- insmod: ERROR: could not insert module /bin/emc/scaleio/scini.ko: Invalid module format
- Error (1) loading module binary /bin/emc/scaleio/scini.ko
- rmmod: ERROR: Module scini is not currently loaded
- Error (1) unloading module binary /bin/emc/scaleio/scini.ko
- *
- S4:~# find / -name *sdc*.deb
- /tmp/scaleio/sdc/EMC-ScaleIO-sdc-2.0-7120.0.Ubuntu.14.04.x86_64.deb
- S4:~# find / -name *sdc*.tar
- /home/administrator/ScaleIO_2.0.0.2_Complete_Linux_SW_Download/ScaleIO_2.0.0.2_UBUNTU_14.04_Download/EMC-ScaleIO-sdc-2.0-7120.0.Ubuntu.14.04.x86_64.tar
- /tmp/scaleio/sdc/EMC-ScaleIO-sdc-2.0-7120.0.Ubuntu.14.04.x86_64.tar
- /opt/emc/scaleio/gateway/temp/scaleio/EMC-ScaleIO-sdc-2.0-7120.0.Ubuntu.14.04.x86_64.tar
- S4:~# find / -name scini.tar (downloaded from EMC ftp for Ubuntu kernel 4.4.0-53-generic)
- /home/administrator/scini.tar
- /bin/emc/scaleio/scini_sync/driver_cache/scini.tar
- S4:~# dpkg -l | grep -E '^ii' | grep scini
- S4:~# dpkg -l | grep -E '^ii' | grep sdc
- ii emc-scaleio-sdc 2.0-7120.0.Ubuntu.14.04 amd64 scaleio sdc package
Note: 3-node cluster, 3-SDSs and 24-HDDs configured:
- S2:~# scli --query_cluster
- Cluster:
- Mode: 3_node, State: Normal, Active: 3/3, Replicas: 2/2
- Master MDM:
- Name: MDM2_S_S2, ID: 0x667ac10c74289b21
- IPs: 10.160.32.32, Management IPs: 10.200.72.181, Port: 9011
- Version: 2.0.7120
- Slave MDMs:
- Name: MDM1_M_S1, ID: 0x6d03f16b7e21ade0
- IPs: 10.160.32.31, Management IPs: 10.200.72.180, Port: 9011
- Status: Normal, Version: 2.0.7120
- Tie-Breakers:
- Name: MDM3_T_S3, ID: 0x3679135d2299ceb2
- IPs: 10.160.32.33, Port: 9011
- Status: Normal, Version: 2.0.7120
- S2:~# scli --query_all_sds
- Query-all-SDS returned 3 SDS nodes.
- Protection Domain 85b8147900000000 Name: domain1
- SDS ID: d62b204f00000002 Name: SDS1_S1 State: Connected, Joined IP: 10.160.32.31,10.160.0.31 Port: 7072 Version: 2.0.7120
- SDS ID: d62b204e00000001 Name: SDS3_S3 State: Connected, Joined IP: 10.160.32.33,10.160.0.33 Port: 7072 Version: 2.0.7120
- SDS ID: d62b204d00000000 Name: SDS2_S2 State: Connected, Joined IP: 10.160.32.32,10.160.0.32 Port: 7072 Version: 2.0.7120
- S2:~# scli --query_all_sdc
- MDM restricted SDC mode: Disabled
- Query all SDC returned 0 SDC nodes.
- S2:~# scli --query_sds --sds_name SDS1_S1
- SDS d62b204f00000002 Name: SDS1_S1 Version: 2.0.7120
- Protection Domain: 85b8147900000000, Name: domain1
- DRL mode: Volatile
- Authentication error: None
- IP information (total 2 IPs):
- 1: 10.160.32.31 Role: SDS Only
- 2: 10.160.0.31 Role: SDC Only
- Port: 7072
- RAM Read Cache information:
- 128.0 MB (131072 KB) total size
- Cache is enabled
- RAM Read Cache memory allocation state is PENDING
- Device information (total 8 devices):
- 1: Name: N/A Path: /dev/sdc Original-path: /dev/sdc ID: aded076300020000
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 2: Name: N/A Path: /dev/sdd Original-path: /dev/sdd ID: aded076400020001
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 3: Name: N/A Path: /dev/sde Original-path: /dev/sde ID: aded076500020002
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 4: Name: N/A Path: /dev/sdf Original-path: /dev/sdf ID: aded076600020003
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 5: Name: N/A Path: /dev/sdg Original-path: /dev/sdg ID: aded076700020004
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 6: Name: N/A Path: /dev/sdh Original-path: /dev/sdh ID: aded076800020005
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 7: Name: N/A Path: /dev/sdi Original-path: /dev/sdi ID: aded076900020006
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 8: Name: N/A Path: /dev/sdj Original-path: /dev/sdj ID: aded076a00020007
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- Rfcache device information (total 0 devices):
- Membership-state: Joined; Connection-state: Connected; SDS-state: Normal; 0 devices with error; 0 Rfcache devices with errors
- 2.2 TB (2227 GB) total capacity
- 2.2 TB (2219 GB) unused capacity
- 0 Bytes snapshots capacity
- 0 Bytes in-use capacity
- 0 Bytes thin capacity
- 0 Bytes unreachable-unused capacity
- 0 active moving-in forward-rebuild jobs
- 0 active moving-in backward-rebuild jobs
- 0 active moving-in rebalance jobs
- 0 active moving-out forward-rebuild jobs
- 0 active moving-out backward-rebuild jobs
- 0 active moving-out rebalance jobs
- 0 pending moving-in forward-rebuild jobs
- 0 pending moving-in backward-rebuild jobs
- 0 pending moving-in rebalance jobs
- 0 pending moving-out forward-rebuild jobs
- 0 pending moving-out backward-rebuild jobs
- 0 pending moving-out rebalance jobs
- 0 Bytes decreased capacity
- Primary-reads 0 IOPS 0 Bytes per-second
- Primary-writes 0 IOPS 0 Bytes per-second
- Secondary-reads 0 IOPS 0 Bytes per-second
- Secondary-writes 0 IOPS 0 Bytes per-second
- Backward-rebuild-reads 0 IOPS 0 Bytes per-second
- Backward-rebuild-writes 0 IOPS 0 Bytes per-second
- Forward-rebuild-reads 0 IOPS 0 Bytes per-second
- Forward-rebuild-writes 0 IOPS 0 Bytes per-second
- Rebalance-reads 0 IOPS 0 Bytes per-second
- Rebalance-writes 0 IOPS 0 Bytes per-second
- S2:~# scli --query_sds --sds_name SDS2_S2
- SDS d62b204d00000000 Name: SDS2_S2 Version: 2.0.7120
- Protection Domain: 85b8147900000000, Name: domain1
- DRL mode: Volatile
- Authentication error: None
- IP information (total 2 IPs):
- 1: 10.160.32.32 Role: SDS Only
- 2: 10.160.0.32 Role: SDC Only
- Port: 7072
- RAM Read Cache information:
- 128.0 MB (131072 KB) total size
- Cache is enabled
- RAM Read Cache memory allocation state is PENDING
- Device information (total 8 devices):
- 1: Name: N/A Path: /dev/sdc Original-path: /dev/sdc ID: aded075300000000
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 2: Name: N/A Path: /dev/sdd Original-path: /dev/sdd ID: aded075400000001
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 3: Name: N/A Path: /dev/sde Original-path: /dev/sde ID: aded075500000002
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 4: Name: N/A Path: /dev/sdf Original-path: /dev/sdf ID: aded075600000003
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 5: Name: N/A Path: /dev/sdg Original-path: /dev/sdg ID: aded075700000004
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 6: Name: N/A Path: /dev/sdh Original-path: /dev/sdh ID: aded075800000005
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 7: Name: N/A Path: /dev/sdi Original-path: /dev/sdi ID: aded075900000006
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 8: Name: N/A Path: /dev/sdj Original-path: /dev/sdj ID: aded075a00000007
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- Rfcache device information (total 0 devices):
- Membership-state: Joined; Connection-state: Connected; SDS-state: Normal; 0 devices with error; 0 Rfcache devices with errors
- 2.2 TB (2227 GB) total capacity
- 2.2 TB (2219 GB) unused capacity
- 0 Bytes snapshots capacity
- 0 Bytes in-use capacity
- 0 Bytes thin capacity
- 0 Bytes unreachable-unused capacity
- 0 active moving-in forward-rebuild jobs
- 0 active moving-in backward-rebuild jobs
- 0 active moving-in rebalance jobs
- 0 active moving-out forward-rebuild jobs
- 0 active moving-out backward-rebuild jobs
- 0 active moving-out rebalance jobs
- 0 pending moving-in forward-rebuild jobs
- 0 pending moving-in backward-rebuild jobs
- 0 pending moving-in rebalance jobs
- 0 pending moving-out forward-rebuild jobs
- 0 pending moving-out backward-rebuild jobs
- 0 pending moving-out rebalance jobs
- 0 Bytes decreased capacity
- Primary-reads 0 IOPS 0 Bytes per-second
- Primary-writes 0 IOPS 0 Bytes per-second
- Secondary-reads 0 IOPS 0 Bytes per-second
- Secondary-writes 0 IOPS 0 Bytes per-second
- Backward-rebuild-reads 0 IOPS 0 Bytes per-second
- Backward-rebuild-writes 0 IOPS 0 Bytes per-second
- Forward-rebuild-reads 0 IOPS 0 Bytes per-second
- Forward-rebuild-writes 0 IOPS 0 Bytes per-second
- Rebalance-reads 0 IOPS 0 Bytes per-second
- Rebalance-writes 0 IOPS 0 Bytes per-second
- S2:~# scli --query_sds --sds_name SDS3_S3
- SDS d62b204e00000001 Name: SDS3_S3 Version: 2.0.7120
- Protection Domain: 85b8147900000000, Name: domain1
- DRL mode: Volatile
- Authentication error: None
- IP information (total 2 IPs):
- 1: 10.160.32.33 Role: SDS Only
- 2: 10.160.0.33 Role: SDC Only
- Port: 7072
- RAM Read Cache information:
- 128.0 MB (131072 KB) total size
- Cache is enabled
- RAM Read Cache memory allocation state is PENDING
- Device information (total 8 devices):
- 1: Name: N/A Path: /dev/sdc Original-path: /dev/sdc ID: aded075b00010000
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 2: Name: N/A Path: /dev/sdd Original-path: /dev/sdd ID: aded075c00010001
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 3: Name: N/A Path: /dev/sde Original-path: /dev/sde ID: aded075d00010002
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 4: Name: N/A Path: /dev/sdf Original-path: /dev/sdf ID: aded075e00010003
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 5: Name: N/A Path: /dev/sdg Original-path: /dev/sdg ID: aded075f00010004
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 6: Name: N/A Path: /dev/sdh Original-path: /dev/sdh ID: aded076000010005
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 7: Name: N/A Path: /dev/sdi Original-path: /dev/sdi ID: aded076100010006
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- 8: Name: N/A Path: /dev/sdj Original-path: /dev/sdj ID: aded076200010007
- Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
- Rfcache device information (total 0 devices):
- Membership-state: Joined; Connection-state: Connected; SDS-state: Normal; 0 devices with error; 0 Rfcache devices with errors
- 2.2 TB (2227 GB) total capacity
- 2.2 TB (2219 GB) unused capacity
- 0 Bytes snapshots capacity
- 0 Bytes in-use capacity
- 0 Bytes thin capacity
- 0 Bytes unreachable-unused capacity
- 0 active moving-in forward-rebuild jobs
- 0 active moving-in backward-rebuild jobs
- 0 active moving-in rebalance jobs
- 0 active moving-out forward-rebuild jobs
- 0 active moving-out backward-rebuild jobs
- 0 active moving-out rebalance jobs
- 0 pending moving-in forward-rebuild jobs
- 0 pending moving-in backward-rebuild jobs
- 0 pending moving-in rebalance jobs
- 0 pending moving-out forward-rebuild jobs
- 0 pending moving-out backward-rebuild jobs
- 0 pending moving-out rebalance jobs
- 0 Bytes decreased capacity
- Primary-reads 0 IOPS 0 Bytes per-second
- Primary-writes 0 IOPS 0 Bytes per-second
- Secondary-reads 0 IOPS 0 Bytes per-second
- Secondary-writes 0 IOPS 0 Bytes per-second
- Backward-rebuild-reads 0 IOPS 0 Bytes per-second
- Backward-rebuild-writes 0 IOPS 0 Bytes per-second
- Forward-rebuild-reads 0 IOPS 0 Bytes per-second
- Forward-rebuild-writes 0 IOPS 0 Bytes per-second
- Rebalance-reads 0 IOPS 0 Bytes per-second
- Rebalance-writes 0 IOPS 0 Bytes per-second
---------------------------------------------------------------------------------------
Hope all this makes sense given I still don't understand everything yet.
CMCC
pawelw1
306 Posts
0
January 25th, 2017 06:00
Hi CCMC,
It looks like there is a space in the "repo_address" and that's why the synch script is complaining - can you please edit it, remove the space after "file:// " and try to synch again?
Thank you,
Pawel