Skip to main content
Start a Conversation

Unsolved

This post is more than 5 years old

CCMC_870db8

10 Posts

3128

January 19th, 2017 21:00

ScaleIO: Set SDC Performance Profile and Set SDC Name Failed

Resolved: I did not set up gatewayUser.properties correctly on initial installation of ScalieIO using CSV file.   I had to edit the driver_sync.conf file and then restart the scini service.  Below is edited driver_synch.conf file.

###############################################################################

# driver_sync Configuration file

# Everything after a '#' until the end of the line is ignored

###############################################################################

# Repository address, prefixed by protocol

#repo_address = sftp://localhost/path/to/repo/dir

#repo_address = ftp://localhost/path/to/repo/dir

#repo_address = file://local/path/to/repo/dir

repo_address = ftp://QNzgdxXix:Aw3wFAwAq3@ftp.emc.comAw3wFAwAq3@ftp.emc.com

# Repository user (valid for ftp/sftp protocol)

#repo_user = scini

repo_user = admin

# Repository password (valid for ftp protocol)

#repo_password = scini

repo_password = password

# Local directory for modules

local_dir = /bin/emc/scaleio/scini_sync/driver_cache/

# User's RSA private key file (sftp protocol)

user_private_rsa_key = /bin/emc/scaleio/scini_sync/scini_key

# Repository host public key (sftp protocol)

repo_public_rsa_key = /bin/emc/scaleio/scini_sync/scini_repo_key.pub

# Should the fetched modules' signatures be checked [0, 1]

module_sigcheck = 0

# EMC public signature key (needed when module_sigcheck is 1)

#emc_public_gpg_key = /bin/emc/scaleio/scini_sync/emc_key.pub

emc_public_gpg_key = /home/administrator/ScaleIO_2.0.0.2_Complete_Linux_SW_Download/ScaleIO_2.0.0.2_GPG-RPM-KEY_Download/RPM-GPG-KEY-ScaleIO_2.0.7120.0

# Sync pattern (regular expression) for massive retrieve

sync_pattern = .*

-----------------------------------------------------

ScaleIO v2.0.0.2

Process:

Created Complete Installation CSV topology file.

scaleiocsv.png

Logged into IM server and ScaleIO Java GUI

Uploaded Installation Packages (MDM, SDSs, SDC, XCACHE, LIA modules)

Uploaded Installation CSV topology file

Initiated Installation

Install Phase complete: 11 commands completed successfully

Configuration Phase Completed: 0 aborted, 2 failed and 25 completed

Commands that failed details: set SDC  Performance Profile failed and set SDC name failed

Debug:

  • Found in the file /bin/emc/scaleio/scini_sync/drive_sync.log
    • § driver_sync.sh START Wed Jan 18 ~
    • § /usr/bin/sftp
    • Error (sftp): File /bin/emc/scaleio/scini_sync/scinit_key must exist
    • § Driver_sync.sh END Wed Jan18 ~
  • Found in the file /opt/emc/scaleio/gateway/logs# cat scaleio-trace.log | grep "Error Failed"
    • 2017-01-18 18:45:27,242 [executor-1] ERROR c.e.s.s.d.i.commands.SetSdcName - Error Failed to set SDC name to SDC1_S4 due to Could not find the SDC executing command .SetSdcName (retry) : com.emc.s3g.scaleio.im.services.installation.configurators.CommandServiceMdmConnection.setSDCName(CommandServiceMdmConnection.java:6147)/opt/emc/scaleio/lia/logs/emc-scaleio-lia-install.log

I guess the problem is the SDC and XCACHE driver(s) not installed which is the result that Ubuntu 14.04 kernel specific driver can't be found and therefore not installed. I read the section "Deploying on Ubuntu server" but do not understand it very well.  Also some of the directories and files mentioned in that section (e.g., driver_sync.sh) are not even present until after installing and configuring the ScaleIO cluster.

I found and downloaded the correct SDC and XCACHE drivers (.tars) for my specific Ubuntu kernel version 4.4.0-53-generic form the EMC ftp link. The two files where scini.tar (scinit.siob, scini,sig) and  xcache.tar (xcache.siob, xcache.sig).

Questions:


1. Is there a way I can edit the driver_sync.conf file to use local repository where tar files and or .siob and .sig files are located to successful install the SDC on the Gateway node?  Or is there a manual procedure to install SDC on the Gateway manually.  Below is my attempt to edit the driver_sync.conf file. 


###############################################################################

# driver_sync Configuration file

# Everything after a '#' until the end of the line is ignored

###############################################################################

# Repository address, prefixed by protocol

#repo_address = sftp://localhost/path/to/repo/dir

#repo_address = ftp://localhost/path/to/repo/dir

repo_address = file://home/administrator

# Repository user (valid for ftp/sftp protocol)

repo_user = scini

# Repository password (valid for ftp protocol)

repo_password = scini

# Local directory for modules

local_dir = /bin/emc/scaleio/scini_sync/driver_cache/

# User's RSA private key file (sftp protocol)

user_private_rsa_key = /bin/emc/scaleio/scini_sync/scini_key

# Repository host public key (sftp protocol)

repo_public_rsa_key = /bin/emc/scaleio/scini_sync/scini_repo_key.pub

# Should the fetched modules' signatures be checked [0, 1]

module_sigcheck =0

# EMC public signature key (needed when module_sigcheck is 1)

emc_public_gpg_key = /bin/emc/scaleio/scini_sync/emc_key.pub

# Sync pattern (regular expression) for massive retrieve

sync_pattern = .*

2. At this point do I need to start over on ScaleIO install and configuration? If so how do I clear existing configuration on the three MDM/SDS nodes and one Gateway/SDC node?

---------------------------- Added 1-25-2017 -------------------------------------

Note: scini not running or install

  • S4:~# service scini status
    • scini driver is not running
  • S4:~# service scini restart
    • *
    • ###############################################################################
    • Verifying scini.ko against current system kernel.
    • Module kernel version: 4.4.0-31-generic
    • Running kernel version: 4.4.0-53-generic
    • Current scini.ko seems to be incompatible with your kernel.
    • Will attempt to fetch from module repository.
    • driver_sync.sh START - Tue Jan 24 15:35:03 MST 2017
    • /bin/emc/scaleio/scini_sync/driver_sync.sh: line 120: repo_address: unbound variable
    • Error: Unable to fetch scini.ko. Will try to load the existing module...
    • insmod: ERROR: could not insert module /bin/emc/scaleio/scini.ko: Invalid module format
    • Error (1) loading module binary /bin/emc/scaleio/scini.ko
    • rmmod: ERROR: Module scini is not currently loaded
    • Error (1) unloading module binary /bin/emc/scaleio/scini.ko
    • *
  • S4:~# find / -name *sdc*.deb
    • /tmp/scaleio/sdc/EMC-ScaleIO-sdc-2.0-7120.0.Ubuntu.14.04.x86_64.deb
  • S4:~# find / -name *sdc*.tar
    • /home/administrator/ScaleIO_2.0.0.2_Complete_Linux_SW_Download/ScaleIO_2.0.0.2_UBUNTU_14.04_Download/EMC-ScaleIO-sdc-2.0-7120.0.Ubuntu.14.04.x86_64.tar
    • /tmp/scaleio/sdc/EMC-ScaleIO-sdc-2.0-7120.0.Ubuntu.14.04.x86_64.tar
    • /opt/emc/scaleio/gateway/temp/scaleio/EMC-ScaleIO-sdc-2.0-7120.0.Ubuntu.14.04.x86_64.tar
  • S4:~# find / -name scini.tar (downloaded from EMC ftp for Ubuntu kernel 4.4.0-53-generic)
    • /home/administrator/scini.tar
    • /bin/emc/scaleio/scini_sync/driver_cache/scini.tar
  • S4:~# dpkg -l | grep -E '^ii' | grep scini
  • S4:~# dpkg -l | grep -E '^ii' | grep sdc
    • ii emc-scaleio-sdc                       2.0-7120.0.Ubuntu.14.04               amd64        scaleio sdc package

Note: 3-node cluster, 3-SDSs and 24-HDDs configured:

  • S2:~# scli --query_cluster
    • Cluster:
    • Mode: 3_node, State: Normal, Active: 3/3, Replicas: 2/2
    • Master MDM:
    • Name: MDM2_S_S2, ID: 0x667ac10c74289b21
    • IPs: 10.160.32.32, Management IPs: 10.200.72.181, Port: 9011
    • Version: 2.0.7120
    • Slave MDMs:
    • Name: MDM1_M_S1, ID: 0x6d03f16b7e21ade0
    • IPs: 10.160.32.31, Management IPs: 10.200.72.180, Port: 9011
    • Status: Normal, Version: 2.0.7120
    • Tie-Breakers:
    • Name: MDM3_T_S3, ID: 0x3679135d2299ceb2
    • IPs: 10.160.32.33, Port: 9011
    • Status: Normal, Version: 2.0.7120

  • S2:~# scli --query_all_sds
    • Query-all-SDS returned 3 SDS nodes.
    • Protection Domain 85b8147900000000 Name: domain1
    • SDS ID: d62b204f00000002 Name: SDS1_S1 State: Connected, Joined IP: 10.160.32.31,10.160.0.31 Port: 7072 Version: 2.0.7120
    • SDS ID: d62b204e00000001 Name: SDS3_S3 State: Connected, Joined IP: 10.160.32.33,10.160.0.33 Port: 7072 Version: 2.0.7120
    • SDS ID: d62b204d00000000 Name: SDS2_S2 State: Connected, Joined IP: 10.160.32.32,10.160.0.32 Port: 7072 Version: 2.0.7120
  • S2:~# scli --query_all_sdc
    • MDM restricted SDC mode: Disabled
    • Query all SDC returned 0 SDC nodes.

  • S2:~# scli --query_sds --sds_name SDS1_S1
  • SDS d62b204f00000002 Name: SDS1_S1 Version: 2.0.7120
  • Protection Domain: 85b8147900000000, Name: domain1
  • DRL mode: Volatile
  • Authentication error: None
  • IP information (total 2 IPs):
  • 1: 10.160.32.31     Role: SDS Only
  • 2: 10.160.0.31      Role: SDC Only
  • Port: 7072
  • RAM Read Cache information:
  • 128.0 MB (131072 KB) total size
  • Cache is enabled
  • RAM Read Cache memory allocation state is PENDING
  • Device information (total 8 devices):
  • 1: Name: N/A  Path: /dev/sdc  Original-path: /dev/sdc  ID: aded076300020000
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 2: Name: N/A  Path: /dev/sdd  Original-path: /dev/sdd  ID: aded076400020001
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 3: Name: N/A  Path: /dev/sde  Original-path: /dev/sde  ID: aded076500020002
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 4: Name: N/A  Path: /dev/sdf  Original-path: /dev/sdf  ID: aded076600020003
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 5: Name: N/A  Path: /dev/sdg  Original-path: /dev/sdg  ID: aded076700020004
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 6: Name: N/A  Path: /dev/sdh  Original-path: /dev/sdh  ID: aded076800020005
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 7: Name: N/A  Path: /dev/sdi  Original-path: /dev/sdi  ID: aded076900020006
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 8: Name: N/A  Path: /dev/sdj  Original-path: /dev/sdj  ID: aded076a00020007
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • Rfcache device information (total 0 devices):
  • Membership-state: Joined; Connection-state: Connected; SDS-state: Normal; 0 devices with error; 0 Rfcache devices with errors
  • 2.2 TB (2227 GB) total capacity
  • 2.2 TB (2219 GB) unused capacity
  • 0 Bytes snapshots capacity
  • 0 Bytes in-use capacity
  • 0 Bytes thin capacity
  • 0 Bytes unreachable-unused capacity
  • 0 active moving-in forward-rebuild jobs
  • 0 active moving-in backward-rebuild jobs
  • 0 active moving-in rebalance jobs
  • 0 active moving-out forward-rebuild jobs
  • 0 active moving-out backward-rebuild jobs
  • 0 active moving-out rebalance jobs
  • 0 pending moving-in forward-rebuild jobs
  • 0 pending moving-in backward-rebuild jobs
  • 0 pending moving-in rebalance jobs
  • 0 pending moving-out forward-rebuild jobs
  • 0 pending moving-out backward-rebuild jobs
  • 0 pending moving-out rebalance jobs
  • 0 Bytes decreased capacity
  • Primary-reads                            0 IOPS 0 Bytes per-second
  • Primary-writes                           0 IOPS 0 Bytes per-second
  • Secondary-reads                          0 IOPS 0 Bytes per-second
  • Secondary-writes                         0 IOPS 0 Bytes per-second
  • Backward-rebuild-reads                   0 IOPS 0 Bytes per-second
  • Backward-rebuild-writes                  0 IOPS 0 Bytes per-second
  • Forward-rebuild-reads                    0 IOPS 0 Bytes per-second
  • Forward-rebuild-writes                   0 IOPS 0 Bytes per-second
  • Rebalance-reads                          0 IOPS 0 Bytes per-second
  • Rebalance-writes                         0 IOPS 0 Bytes per-second
  • S2:~# scli --query_sds --sds_name SDS2_S2
  • SDS d62b204d00000000 Name: SDS2_S2 Version: 2.0.7120
  • Protection Domain: 85b8147900000000, Name: domain1
  • DRL mode: Volatile
  • Authentication error: None
  • IP information (total 2 IPs):
  • 1: 10.160.32.32     Role: SDS Only
  • 2: 10.160.0.32      Role: SDC Only
  • Port: 7072
  • RAM Read Cache information:
  • 128.0 MB (131072 KB) total size
  • Cache is enabled
  • RAM Read Cache memory allocation state is PENDING
  • Device information (total 8 devices):
  • 1: Name: N/A  Path: /dev/sdc  Original-path: /dev/sdc  ID: aded075300000000
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 2: Name: N/A  Path: /dev/sdd  Original-path: /dev/sdd  ID: aded075400000001
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 3: Name: N/A  Path: /dev/sde  Original-path: /dev/sde  ID: aded075500000002
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 4: Name: N/A  Path: /dev/sdf  Original-path: /dev/sdf  ID: aded075600000003
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 5: Name: N/A  Path: /dev/sdg  Original-path: /dev/sdg  ID: aded075700000004
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 6: Name: N/A  Path: /dev/sdh  Original-path: /dev/sdh  ID: aded075800000005
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 7: Name: N/A  Path: /dev/sdi  Original-path: /dev/sdi  ID: aded075900000006
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 8: Name: N/A  Path: /dev/sdj  Original-path: /dev/sdj  ID: aded075a00000007
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • Rfcache device information (total 0 devices):
  • Membership-state: Joined; Connection-state: Connected; SDS-state: Normal; 0 devices with error; 0 Rfcache devices with errors
  • 2.2 TB (2227 GB) total capacity
  • 2.2 TB (2219 GB) unused capacity
  • 0 Bytes snapshots capacity
  • 0 Bytes in-use capacity
  • 0 Bytes thin capacity
  • 0 Bytes unreachable-unused capacity
  • 0 active moving-in forward-rebuild jobs
  • 0 active moving-in backward-rebuild jobs
  • 0 active moving-in rebalance jobs
  • 0 active moving-out forward-rebuild jobs
  • 0 active moving-out backward-rebuild jobs
  • 0 active moving-out rebalance jobs
  • 0 pending moving-in forward-rebuild jobs
  • 0 pending moving-in backward-rebuild jobs
  • 0 pending moving-in rebalance jobs
  • 0 pending moving-out forward-rebuild jobs
  • 0 pending moving-out backward-rebuild jobs
  • 0 pending moving-out rebalance jobs
  • 0 Bytes decreased capacity
  • Primary-reads                            0 IOPS 0 Bytes per-second
  • Primary-writes                           0 IOPS 0 Bytes per-second
  • Secondary-reads                          0 IOPS 0 Bytes per-second
  • Secondary-writes                         0 IOPS 0 Bytes per-second
  • Backward-rebuild-reads                   0 IOPS 0 Bytes per-second
  • Backward-rebuild-writes                  0 IOPS 0 Bytes per-second
  • Forward-rebuild-reads                    0 IOPS 0 Bytes per-second
  • Forward-rebuild-writes                   0 IOPS 0 Bytes per-second
  • Rebalance-reads                          0 IOPS 0 Bytes per-second
  • Rebalance-writes                         0 IOPS 0 Bytes per-second
  • S2:~# scli --query_sds --sds_name SDS3_S3
  • SDS d62b204e00000001 Name: SDS3_S3 Version: 2.0.7120
  • Protection Domain: 85b8147900000000, Name: domain1
  • DRL mode: Volatile
  • Authentication error: None
  • IP information (total 2 IPs):
  • 1: 10.160.32.33     Role: SDS Only
  • 2: 10.160.0.33      Role: SDC Only
  • Port: 7072
  • RAM Read Cache information:
  • 128.0 MB (131072 KB) total size
  • Cache is enabled
  • RAM Read Cache memory allocation state is PENDING
  • Device information (total 8 devices):
  • 1: Name: N/A  Path: /dev/sdc  Original-path: /dev/sdc  ID: aded075b00010000
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 2: Name: N/A  Path: /dev/sdd  Original-path: /dev/sdd  ID: aded075c00010001
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 3: Name: N/A  Path: /dev/sde  Original-path: /dev/sde  ID: aded075d00010002
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 4: Name: N/A  Path: /dev/sdf  Original-path: /dev/sdf  ID: aded075e00010003
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 5: Name: N/A  Path: /dev/sdg  Original-path: /dev/sdg  ID: aded075f00010004
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 6: Name: N/A  Path: /dev/sdh  Original-path: /dev/sdh  ID: aded076000010005
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 7: Name: N/A  Path: /dev/sdi  Original-path: /dev/sdi  ID: aded076100010006
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • 8: Name: N/A  Path: /dev/sdj  Original-path: /dev/sdj  ID: aded076200010007
  • Storage Pool: pool1, Capacity: 278 GB Error-fixes: 0 scanned 0 MB, Compare errors: 0 State: Normal
  • Rfcache device information (total 0 devices):
  • Membership-state: Joined; Connection-state: Connected; SDS-state: Normal; 0 devices with error; 0 Rfcache devices with errors
  • 2.2 TB (2227 GB) total capacity
  • 2.2 TB (2219 GB) unused capacity
  • 0 Bytes snapshots capacity
  • 0 Bytes in-use capacity
  • 0 Bytes thin capacity
  • 0 Bytes unreachable-unused capacity
  • 0 active moving-in forward-rebuild jobs
  • 0 active moving-in backward-rebuild jobs
  • 0 active moving-in rebalance jobs
  • 0 active moving-out forward-rebuild jobs
  • 0 active moving-out backward-rebuild jobs
  • 0 active moving-out rebalance jobs
  • 0 pending moving-in forward-rebuild jobs
  • 0 pending moving-in backward-rebuild jobs
  • 0 pending moving-in rebalance jobs
  • 0 pending moving-out forward-rebuild jobs
  • 0 pending moving-out backward-rebuild jobs
  • 0 pending moving-out rebalance jobs
  • 0 Bytes decreased capacity
  • Primary-reads                            0 IOPS 0 Bytes per-second
  • Primary-writes                           0 IOPS 0 Bytes per-second
  • Secondary-reads                          0 IOPS 0 Bytes per-second
  • Secondary-writes                         0 IOPS 0 Bytes per-second
  • Backward-rebuild-reads                   0 IOPS 0 Bytes per-second
  • Backward-rebuild-writes                  0 IOPS 0 Bytes per-second
  • Forward-rebuild-reads                    0 IOPS 0 Bytes per-second
  • Forward-rebuild-writes                   0 IOPS 0 Bytes per-second
  • Rebalance-reads                          0 IOPS 0 Bytes per-second
  • Rebalance-writes                         0 IOPS 0 Bytes per-second

---------------------------------------------------------------------------------------

Hope all this makes sense given I still don't understand everything yet.

CMCC

pawelw1

306 Posts

January 25th, 2017 06:00

Hi CCMC,

It looks like there is a space in the "repo_address" and that's why the synch script is complaining - can you please edit it, remove the space after "file:// " and try to synch again?

Thank you,

Pawel

View More

View All

No Events found!

Top