UNSOLVED

lamsarr2

updated

6 years ago

L

lamsarr2

1 Rookie

10 Posts

0

9181

July 7th, 2020 07:00

iSilon H400 Error when adding a node in the cluster

I have a iSilon H400 with 4 nodes, 2 Switch Dell S4112–ON. In each node IntA and IntB is connected to the both backend switch (IntA siwtch1 and IntB switch2).

Ext-1 of each node is connected a the backbone switch by 1G.

Ext-2 of each node is connected a the backbone switch by 10G.

Cluster is created succesffully in the Node1.

Now When I join node2 the process goes to the end with status completed
Isilon OneFS/amd64 (XXX-ISILON-2) (ttyu0)
login: root
Password:

But after login by root/password when i try to excute command of on one side i have this message error in both 

"Failed to connect to REST server: [61] Connection refused". And when trying to connect by GUI by admin/password i have "server-side failure with status code 500".

  • lamsarr2

    1 Rookie

    10 Posts

    6351

    0

    Posted July 7th, 2020 07:00

    Node1 Configs:

    Please select an option.

    Node build: Isilon OneFS v8.1.2.0 B_8_1_2_016(RELEASE)
    Node serial number: XXXXXXXXXXXXXXXX

    Select an option:
    [ 1] Create a new cluster
    [ 2] Join an existing cluster
    [ 3] Exit wizard and configure manually
    [ 4] Reboot into SmartLock Compliance mode
    Wizard >>> 1
    Do you accept the EULA? [no] yes
    Please change the root password from the default.
    Please enter new password for root:
    Please re-enter password for root:

    Password changed.

    Please change the UI admin password from the default.
    Please enter new password for admin:
    Please re-enter password for admin:

    Password changed.


    Enter a new name for the cluster:
    Configure name >>> XXX-ISILON

    Cluster name set to XXX-ISILON

    Cluster encoding:
    [ 1] Windows-SJIS
    [ 2] Windows-949
    [ 3] Windows-1252
    [ 4] EUC-KR
    [ 5] EUC-JP
    [ 6] EUC-JP-MS
    [ 7] UTF-8-MAC
    [ 8] UTF-8
    [ 9] ISO-8859-1 (Latin-1)
    [10] ISO-8859-2 (Latin-2)
    [11] ISO-8859-3 (Latin-3)
    [12] ISO-8859-4 (Latin-4)
    [13] ISO-8859-5 (Cyrillic)
    [14] ISO-8859-6 (Arabic)
    [15] ISO-8859-7 (Greek)
    [16] ISO-8859-8 (Hebrew)
    [17] ISO-8859-9 (Latin-5)
    [18] ISO-8859-10 (Latin-6)
    [19] ISO-8859-13 (Latin-7)
    [20] ISO-8859-14 (Latin-8)
    [21] ISO-8859-15 (Latin-9)
    [22] ISO-8859-16 (Latin-10)
    [Enter] Use current encoding: utf-8
    Configure encoding >>>
    Encoding set to utf-8

    Configure interface int-a:
    [ 1] Configure netmask
    [ 2] Configure int-a IP ranges
    [Enter] Keep the current configuration:
    Netmask: (not set)
    IP ranges: (not set)
    Configure interface int-a >>> 1

    Enter a new netmask
    Configure int-a netmask >>> 255.255.255.0

    Configure interface int-a:
    [ 1] Configure netmask
    [ 2] Configure int-a IP ranges
    [Enter] Keep the current configuration:
    Netmask: 255.255.255.0
    IP ranges: (not set)
    Configure interface int-a >>> 2

    Configure int-a IP ranges
    [ 1] Add an IP range
    [ 2] Delete an IP range
    [Enter] Keep the current IP ranges:
    IP ranges: (not set)
    Configure int-a IP ranges >>> 1

    Enter the low IP address of the range to add:
    Low IP address (add) >>> 10.1.1.10

    Enter the high IP address of the range:
    High IP address (add) >>> 10.1.1.15

    Configure int-a IP ranges
    [ 1] Add an IP range
    [ 2] Delete an IP range
    [Enter] Keep the current IP ranges:
    IP ranges: 10.1.1.10-10.1.1.15
    Configure int-a IP ranges >>>

    Configure interface int-a:
    [ 1] Configure netmask
    [ 2] Configure int-a IP ranges
    [Enter] Keep the current configuration:
    Netmask: 255.255.255.0
    IP ranges: 10.1.1.10-10.1.1.15
    Configure interface int-a >>>

    Select the internal interfaces to configure
    [ 1] int-a - primary internal interface
    [ 2] int-b - secondary internal interface (failover)
    [Enter] Exit configuring internal interfaces
    Configure internal interfaces >>> 2

    Configure interface int-b (failover):
    [ 1] Configure netmask
    [ 2] Configure int-b IP ranges
    [ 3] Configure failover IP ranges
    [Enter] Keep the current configuration:
    Netmask: (not set)
    IP ranges: (not set)
    Failover IP ranges: (not set)
    Failover: disabled
    Configure interface int-b (failover) >>> 1

    Type a new netmask
    Configure int-b netmask >>> 255.255.255.0

    Configure interface int-b (failover):
    [ 1] Configure netmask
    [ 2] Configure int-b IP ranges
    [ 3] Configure failover IP ranges
    [Enter] Keep the current configuration:
    Netmask: 255.255.255.0
    IP ranges: (not set)
    Failover IP ranges: (not set)
    Failover: disabled
    Configure interface int-b (failover) >>> 2

    Configure int-b IP ranges
    [ 1] Add an IP range
    [ 2] Delete an IP range
    [Enter] Keep the current IP ranges:
    IP ranges: (not set)
    Configure int-b IP ranges >>> 1

    Enter the low IP address of the range to add:
    Low IP address (add) >>> 10.1.2.10

    Enter the high IP address of the range:
    High IP address (add) >>> 10.1.2.15

    Configure int-b IP ranges
    [ 1] Add an IP range
    [ 2] Delete an IP range
    [Enter] Keep the current IP ranges:
    IP ranges: 10.1.2.10-10.1.2.15
    Configure int-b IP ranges >>>

    Configure interface int-b (failover):
    [ 1] Configure netmask
    [ 2] Configure int-b IP ranges
    [ 3] Configure failover IP ranges
    [Enter] Keep the current configuration:
    Netmask: 255.255.255.0
    IP ranges: 10.1.2.10-10.1.2.15
    Failover IP ranges: (not set)
    Failover: disabled
    Configure interface int-b (failover) >>> 3

    Configure failover IP ranges
    [ 1] Add an IP range
    [ 2] Delete an IP range
    [Enter] Keep the current IP ranges:
    Failover IP ranges: (not set)
    Configure failover IP ranges >>> 1

    Enter the low IP address of the range to add:
    Low IP address (add) >>> 10.1.3.10

    Enter the high IP address of the range:
    High IP address (add) >>> 10.1.3.15

    Configure failover IP ranges
    [ 1] Add an IP range
    [ 2] Delete an IP range
    [Enter] Keep the current IP ranges:
    Failover IP ranges: 10.1.3.10-10.1.3.15
    Configure failover IP ranges >>>

    Configure interface int-b (failover):
    [ 1] Configure netmask
    [ 2] Configure int-b IP ranges
    [ 3] Configure failover IP ranges
    [ 4] Disable failover
    [Enter] Keep the current configuration:
    Netmask: 255.255.255.0
    IP ranges: 10.1.2.10-10.1.2.15
    Failover IP ranges: 10.1.3.10-10.1.3.15
    Failover: enabled
    Configure interface int-b (failover) >>>

    Select the internal interfaces to configure
    [ 1] int-a - primary internal interface
    [ 2] int-b - secondary internal interface (failover)
    [Enter] Exit configuring internal interfaces
    Configure internal interfaces >>>

    Configure external subnet
    [ 1] mgmt-1 - External interface
    [ 2] 10gige-1 - External interface
    [Enter] Exit configuring external network.
    Configure external subnet >>> 1

    Configure interface mgmt-1:
    [ 1] Configure netmask
    [ 2] Configure MTU
    [ 3] Configure mgmt-1 IP ranges
    [Enter] Keep the current configuration:
    Netmask: (not set)
    MTU: 1500
    IP ranges: (not set)
    Configure interface mgmt-1 >>> 1

    Enter a new netmask
    Configure mgmt-1 netmask >>> 255.255.255.192

    Configure interface mgmt-1:
    [ 1] Configure netmask
    [ 2] Configure MTU
    [ 3] Configure mgmt-1 IP ranges
    [Enter] Keep the current configuration:
    Netmask: 255.255.255.192
    MTU: 1500
    IP ranges: (not set)
    Configure interface mgmt-1 >>> 3

    Configure external IP ranges
    [ 1] Add an IP range
    [ 2] Delete an IP range
    [Enter] Keep the current IP ranges:
    IP ranges: (not set)
    Configure mgmt-1 IP ranges >>> 1

    Enter the low IP address of the range to add:
    Low IP address (add) >>> 10.149.195.51

    Enter the high IP address of the range:
    High IP address (add) >>> 10.149.195.54

    Configure external IP ranges
    [ 1] Add an IP range
    [ 2] Delete an IP range
    [Enter] Keep the current IP ranges:
    IP ranges: 10.149.195.51-10.149.195.54
    Configure mgmt-1 IP ranges >>>

    Configure interface mgmt-1:
    [ 1] Configure netmask
    [ 2] Configure MTU
    [ 3] Configure mgmt-1 IP ranges
    [Enter] Keep the current configuration:
    Netmask: 255.255.255.192
    MTU: 1500
    IP ranges: 10.149.195.51-10.149.195.54
    Configure interface mgmt-1 >>>

    Enter default gateway:
    Configure default gateway >>> 10.149.195.61

    Configure SmartConnect settings
    [ 1] SmartConnect zone name
    [ 2] SmartConnect service IP
    [Enter] Keep the current SmartConnect settings:
    SmartConnect zone name: (not set)
    SmartConnect service IP: (not set)
    Configure SmartConnect settings >>> 1

    Enter SmartConnect zone name
    SmartConnect zone name >>> XXX-ISILON-SIEGE

    Configure SmartConnect settings
    [ 1] SmartConnect zone name
    [ 2] SmartConnect service IP
    [Enter] Keep the current SmartConnect settings:
    SmartConnect zone name: XXX-ISILON-SIEGE
    SmartConnect service IP: (not set)
    Configure SmartConnect settings >>> 2

    Enter SmartConnect service IP
    SmartConnect service IP >>> 10.149.195.35

    Configure SmartConnect settings
    [ 1] SmartConnect zone name
    [ 2] SmartConnect service IP
    [Enter] Keep the current SmartConnect settings:
    SmartConnect zone name: XXX-ISILON-SIEGE
    SmartConnect service IP: 10.149.195.35
    Configure SmartConnect settings >>>

    Configure DNS settings
    [ 1] DNS servers
    [ 2] Search domains
    [Enter] Keep current DNS settings:
    DNS servers: (not set)
    Search domains: (not set)
    Configure DNS settings >>> 1

    Enter a comma-separated list of DNS servers:
    Configure DNS >>> 10.149.3.250,10.149.201.30

    Configure DNS settings
    [ 1] DNS servers
    [ 2] Search domains
    [Enter] Keep current DNS settings:
    DNS servers: 10.149.3.250, 10.149.201.30
    Search domains: (not set)
    Configure DNS settings >>> 2

    Enter a comma-separated list of search domains:
    Search domains >>> XXXlocal.sn

    Configure DNS settings
    [ 1] DNS servers
    [ 2] Search domains
    [Enter] Keep current DNS settings:
    DNS servers: 10.149.3.250, 10.149.201.30
    Search domains: XXXlocal.sn
    Configure DNS settings >>>

    Configure external subnet
    [ 1] mgmt-1 - External interface
    [ 2] 10gige-1 - External interface
    [Enter] Exit configuring external network.
    Configure external subnet >>>

    Configure cluster date and time
    [ 1] Configure time zone
    [ 2] Configure day and time
    [Enter] Keep the current date and time: 2020/07/06 13:58:30 UTC.
    Configure date >>>

    Configure cluster join mode
    [ 1] Manual
    [ 2] Secure
    [Enter] Keep the current join mode: Manual
    Configure join mode >>>
    Join mode set to Manual.

    You have made the following configuration changes:

    Cluster name : (not set) -> XXX-ISILON
    Encoding : (not set) -> utf-8
    int-a : enabled -> enabled
    int-a netmask : (not set) -> 255.255.255.0
    int-a IP ranges : (not set) -> { 10.1.1.10-10.1.1.15 }
    int-b : disabled -> enabled
    int-b netmask : (not set) -> 255.255.255.0
    int-b IP ranges : (not set) -> { 10.1.2.10-10.1.2.15 }
    lpbk : disabled -> enabled
    lpbk netmask : (not set) -> 255.255.255.0
    lpbk IP ranges : (not set) -> { 10.1.3.10-10.1.3.15 }
    mgmt-1 netmask : (not set) -> 255.255.255.192
    mgmt-1 IP range : (not set) -> { 10.149.195.51-10.149.195.54 }
    mgmt-1 gateway : (not set) -> 10.149.195.61
    SmartConnect zone name : (not set) -> XXX-ISILON-SIEGE
    SmartConnect service IP: (not set) -> 10.149.195.35
    DNS servers : (not set) -> { 10.149.3.250, 10.149.201.30 }
    Search domains : (not set) -> { XXXlocal.sn }

    Do you wish to commit these changes? [yes]
    Commit changes? >>> yes
    arp: writing to routing socket: Device busy
    arp: writing to routing socket: Device busy
    No matching processes were found
    Stopping syslogd.
    Waiting for PIDS: 1915.
    Starting syslogd.
    Executing script isi_led
    Executing script isi_post_network
    Pre-IFS daemons: isi_watchdog_d isi_hangdump
    Ethernet Back-End. isi_eth_mixer_d service DISABLED
    isi_flexnet_d.
    Executing script mountcritremote
    Executing script isi_mcp
    Starting isi_mcp.
    Executing script virecover
    Executing script pwcheck
    Executing script tmp
    Executing script dmesg
    Executing script accounting
    Executing script devfs
    Executing script archdep
    Executing script motd
    Executing script hostapd
    Executing script gptboot
    Executing script nfsclient
    Executing script isi_mount_ifs
    Updating motd:Mounting IFS filesystem...
    .
    Executing script cleartmp
    Executing script abi
    Executing script statd
    Clearing /tmp (X related).
    Executing script lockd
    Executing script localpkg
    Executing script SERVERS
    Executing script DAEMON
    Executing script apm
    Executing script ntpd
    Executing script lpd
    Executing script rarpd
    Executing script sdpd
    Executing script powerd
    Executing script nscd
    Executing script moused
    Executing script mountlate
    Executing script utx
    Executing script local
    Executing script rtadvd
    Mounting late file systems:.
    Executing script bootparams
    Executing script apmd
    Executing script swaplate
    Executing script LOGIN
    Executing script cron
    Executing script othermta
    Executing script sendmail
    Executing script inetd
    Starting cron.
    Executing script msgs
    Executing script jail
    Executing script syscons
    Executing script bgfsck
    Executing script securelevel
    -->/dev/da16s1e
    -->/dev/da15s1e
    -->/dev/da14s1e
    -->/dev/da13s1e
    -->/dev/da12s1e
    -->/dev/da11s1e
    -->/dev/da10s1e
    -->/dev/da9s1e
    -->/dev/da8s1e
    -->/dev/da7s1e
    -->/dev/da6s1e
    -->/dev/da5s1e
    -->/dev/da4s1e
    -->/dev/da3s1e
    -->/dev/da2s1e

    Formatting drives...

    N/A N/A Bay A0/da9 100% Bay A1/da2 100%
    Bay A2/da10 100% Bay B0/da3 100% Bay B1/da11 100% Bay B2/da4 100%
    Bay C0/da12 100% Bay C1/da5 100% Bay C2/da13 100% Bay D0/da6 100%
    Bay D1/da14 100% Bay D2/da7 100% Bay E0/da15 100% Bay E1/da8 100%
    Bay E2/da16 100%
    Executing script isi_post_ifs
    a 244b5494.0
    a 2c543cd1.0
    a 38ae8eda.0
    a 415660c1.0
    a 653b494a.0
    a 6a909d98.0
    a BaltimoreCyberTrustRoot.pem
    a DigiCertHighAssuranceEV-RootCA.pem
    a DigiCertSHA2HighAssuranceServerCA.pem
    a GeoTrust_Global_CA.pemx 244b5494.0
    a GoogleInternetAuthority_G3.pem
    a VerisignClass3PublicPrimaryCertificationAuthority.pem

    x 2c543cd1.0
    x 38ae8eda.0
    x 415660c1.0
    x 653b494a.0
    x 6a909d98.0
    x BaltimoreCyberTrustRoot.pem
    x DigiCertHighAssuranceEV-RootCA.pem
    x DigiCertSHA2HighAssuranceServerCA.pem
    x GeoTrust_Global_CA.pem
    x GoogleInternetAuthority_G3.pem
    x VerisignClass3PublicPrimaryCertificationAuthority.pem
    kern.cam.clear_wce_on_resets: 0 -> 1
    kern.cam.lost_cache_downs_drive: 0 -> 1
    Upgrading Source Records...
    Upgrading Target Records...
    Upgrading policies...
    Upgrading siq global config...
    + BASEDIR=/ifs/.ifsvar/upgrade
    + BASESCRIPT=/ifs/.ifsvar/upgrade/isi_database_delete.sh
    + LOCKFILE=/ifs/.ifsvar/upgrade/isi_database_delete.sh.lock
    + FLAGFILE=/ifs/.ifsvar/upgrade/isi_database_delete.sh.flag
    + ULOG=/usr/sbin/isi_upgrade_ulog
    + DBFILE=/ifs/.ifsvar/modules/stats/persistence.db
    + [ -e /ifs/.ifsvar/upgrade/isi_database_delete.sh.flag ]
    + [ 0 -eq 0 ]
    + exec lockf -k /ifs/.ifsvar/upgrade/isi_database_delete.sh.lock /usr/share/upgrade/event-actions/post-commit/isi_database_delete.sh under-lock
    + BASEDIR=/ifs/.ifsvar/upgrade
    + BASESCRIPT=/ifs/.ifsvar/upgrade/isi_database_delete.sh
    + LOCKFILE=/ifs/.ifsvar/upgrade/isi_database_delete.sh.lock
    + FLAGFILE=/ifs/.ifsvar/upgrade/isi_database_delete.sh.flag
    + ULOG=/usr/sbin/isi_upgrade_ulog
    + DBFILE=/ifs/.ifsvar/modules/stats/persistence.db
    + [ -e /ifs/.ifsvar/upgrade/isi_database_delete.sh.flag ]
    + [ 1 -eq 0 ]
    + /bin/rm -f /ifs/.ifsvar/modules/stats/persistence.db
    + /usr/sbin/isi_upgrade_ulog -l 42 'Deleted DB file'
    + /usr/bin/isi_gconfig -t stats-config -R history
    + /usr/sbin/isi_upgrade_ulog -l 48 'Reset stats_config history'
    + /usr/bin/touch /ifs/.ifsvar/upgrade/isi_database_delete.sh.flag
    + /usr/sbin/isi_upgrade_ulog -l 52 'Touched flagfile'
    + /usr/bin/killall isi_stats_hist_d
    + /usr/bin/killall isi_stats_d
    + /usr/sbin/isi_upgrade_ulog -l 58 'Sent TERM to isi_stats_hist_d and isi_stats_d'
    Please run 'isi upgrade patches --help' for patch system usage
    Please run 'isi upgrade patches --help' for patch system usage
    Please run 'isi upgrade patches --help' for patch system usage
    Please run 'isi upgrade patches --help' for patch system usage
    Cleared upgrade config
    This platform is not supported by this script
    request: STATUS, task:STATUS_SYNC, hash: (null)
    Executing script isi_firmware_versions
    Save firmware status
    completed

    Isilon OneFS/amd64 (XXX-ISILON-1) (ttyu0)

    login:

    -----

    Node2 Configs:


    Please select an option.

    Node build: Isilon OneFS v8.1.2.0 B_8_1_2_016(RELEASE)
    Node serial number: XXXXXXXXXXX

    Select an option:
    [ 1] Create a new cluster
    [ 2] Join an existing cluster
    [ 3] Exit wizard and configure manually
    [ 4] Reboot into SmartLock Compliance mode
    Wizard >>> 2

    Select the cluster you want to join
    Index Name Version Status
    ----- ---------- ------------ ------------------------------------------------
    1 XXX-ISILON B_8_1_2_016R available
    [Enter] Refresh the list
    Join cluster >>> 1
    Joining cluster: 'XXX-ISILON'
    Joining status: Beginning join to cluster XXX-ISILON
    Joining status: Beginning join to cluster XXX-ISILON
    Joining status: Checking version compatibility with cluster XXX-ISILON
    Joining status: Versions compatible
    Joining status: Joined cluster XXX-ISILON as devid 2
    arp: writing to routing socket: Device busy
    arp: writing to routing socket: Device busy
    No matching processes were found
    Stopping syslogd.
    Waiting for PIDS: 1878.
    Starting syslogd.
    Executing script isi_led
    Executing script isi_post_network
    Pre-IFS daemons: isi_watchdog_d isi_hangdump
    Ethernet Back-End. isi_eth_mixer_d service DISABLED
    isi_flexnet_d.
    Executing script mountcritremote
    Executing script isi_mcp
    Starting isi_mcp.
    Executing script virecover
    Executing script pwcheck
    Executing script tmp
    Executing script dmesg
    Executing script accounting
    Executing script devfs
    Executing script archdep
    Executing script motd
    Executing script hostapd
    Executing script gptboot
    Executing script nfsclient
    Executing script isi_mount_ifs
    Mounting IFS filesystem...
    Updating motd:.
    Executing script cleartmp
    Executing script abi
    Executing script statd
    Clearing /tmp (X related).
    Executing script lockd
    Executing script localpkg
    Executing script SERVERS
    Executing script DAEMON
    Executing script apm
    Executing script ntpd
    Executing script lpd
    Executing script rarpd
    Executing script sdpd
    Executing script powerd
    Executing script nscd
    Executing script moused
    Executing script mountlate
    Executing script utx
    Executing script local
    Executing script rtadvd
    Mounting late file systems: Executing script bootparams
    .
    Executing script apmd
    Executing script swaplate
    Executing script LOGIN
    Executing script cron
    Executing script othermta
    Executing script sendmail
    Executing script inetd
    Starting cron.
    Executing script msgs
    Executing script jail
    Executing script syscons
    Executing script bgfsck
    Executing script securelevel
    -->/dev/da16s1e
    -->/dev/da15s1e
    -->/dev/da14s1e
    -->/dev/da13s1e
    -->/dev/da12s1e
    -->/dev/da11s1e
    -->/dev/da10s1e
    -->/dev/da9s1e
    -->/dev/da8s1e
    -->/dev/da7s1e
    -->/dev/da6s1e
    -->/dev/da5s1e
    -->/dev/da4s1e
    -->/dev/da3s1e
    -->/dev/da2s1e

    Formatting drives...

    N/A N/A Bay A0/da9 100% Bay A1/da2 100%
    Bay A2/da10 100% Bay B0/da3 100% Bay B1/da11 100% Bay B2/da4 100%
    Bay C0/da12 100% Bay C1/da5 100% Bay C2/da13 100% Bay D0/da6 100%
    Bay D1/da14 100% Bay D2/da7 100% Bay E0/da15 100% Bay E1/da8 100%
    Bay E2/da16 100%
    mnv0: HW link up event
    mnv0: transport link up
    mnv0: peer up
    Executing script isi_post_ifs
    request: STATUS, task:STATUS_SYNC, hash: (null)
    Executing script isi_firmware_versions
    Save firmware status
    completed

    Isilon OneFS/amd64 (XXX-ISILON-2) (ttyu0)

    login:

  • Phil.Lam

    4 Apprentice

    641 Posts

    12 Points

    6342

    0

    Posted July 7th, 2020 14:00

    @lamsarr2,

    Have you resolved the REST API issue?  If not, try rebooting node 2?

  • DELL-Josh Cr

    Community Manager

    9698 Posts

    43691 Points

    6340

    0

    Posted July 7th, 2020 14:00

    1. check if the isi_papi_d service is enabled : 

    # isi services -a isi_papi_d


    2. if the isi_papi_d service is disabled, enable it by running the following command : 

    # isi services -a isi_papi_d
    Service 'isi_papi_d' is enabled.


    3. if the isi_papi_d service was already enabled, check the /var/log/apache2/webui_httpd_error.log for errors such as this : 

    2018-09-24T11:47:56Z <18.3> PDC-NAS-ISN01-20 httpd[80131]: [:error] [pid 80131:tid 34676253696] (20)Not a directory: [client 192.168.10.10:37994] FastCGI: failed to connect to server "/usr/sbin/isi_papi_d": socket file descriptor (1712) is larger than FD_SETSIZE (1024), you probably need to rebuild Apache with a larger FD_SETSIZE
    2018-09-24T11:47:56Z <18.3> PDC-NAS-ISN01-20 httpd[80131]: [:error] [pid 80131:tid 34676253696] [client 10.0.2.63:37994] FastCGI: incomplete headers (0 bytes) received from server "/usr/sbin/isi_papi_d"
    2018-09-24T11:47:56Z <18.3> PDC-NAS-ISN01-20 httpd[80131]: [:error] [pid 80131:tid 34676253696] [client 10.0.2.63:37994] FastCGI: do_work() failed with ret 500 for request GET /platform/1/cluster/config HTTP/1.1


    if you see the above error, you should restart the isi_papi_d service by running the below commands : 

    # isi services -a isi_papi_d disable
    
    # isi services -a isi_papi_d enable

     

  • lamsarr2

    1 Rookie

    10 Posts

    6317

    0

    Posted July 8th, 2020 06:00

    Hi PhilLam

    I also restarted Node2, but no change. 

  • lamsarr2

    1 Rookie

    10 Posts

    6318

    0

    Posted July 8th, 2020 06:00

    Hi Josh,

    I executed the commands and checked the service Service 'isi_papi_d' is enabled.
    And in "/var/log/apache2/webui_httpd_error.log" we have below the Output of the executed command. Then we have disabled and reactivated the "isi_papi_d" service and the problem still remains. We still have errors in the console ("UVS-ISILON-1# isi status
    Failed to connect to REST server: [61] Connection refused")
    and in the WebUI "erver-side failure with status code 500".

    I also restarted Node 2, but no change.

    Jul 6 13:15:34 newsyslog[866]: logfile first created
    2020-07-06T14:07:32Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678856704] (61)Connection refused: [client 10.149.196.118:51291] FastCGI: failed to connect to server "/usr/sbin/isi_papi_d": connect() failed, referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:07:32Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678856704] [client 10.149.196.118:51291] FastCGI: incomplete headers (0 bytes) received from server "/usr/sbin/isi_papi_d", referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:07:32Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678856704] [client 10.149.196.118:51291] FastCGI: do_work() failed with ret 500 for request GET /platform/3/event/eventgroup-occurrences?_dc=1594044434091&ignore=false&resolved=false&sort=severity&dir=ASC&limit=1 HTTP/1.1, referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:07:48Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678864896] (61)Connection refused: [client 10.149.196.118:51299] FastCGI: failed to connect to server "/usr/sbin/isi_papi_d": connect() failed, referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:07:48Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678864896] [client 10.149.196.118:51299] FastCGI: incomplete headers (0 bytes) received from server "/usr/sbin/isi_papi_d", referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:07:48Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678864896] [client 10.149.196.118:51299] FastCGI: do_work() failed with ret 500 for request GET /platform/3/event/eventgroup-occurrences?_dc=1594044450093&ignore=false&resolved=false&sort=severity&dir=ASC&limit=1 HTTP/1.1, referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:08:05Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678867968] (61)Connection refused: [client 10.149.196.118:51279] FastCGI: failed to connect to server "/usr/sbin/isi_papi_d": connect() failed, referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:08:05Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678867968] [client 10.149.196.118:51279] FastCGI: incomplete headers (0 bytes) received from server "/usr/sbin/isi_papi_d", referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:08:05Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678867968] [client 10.149.196.118:51279] FastCGI: do_work() failed with ret 500 for request GET /platform/3/event/eventgroup-occurrences?_dc=1594044467089&ignore=false&resolved=false&sort=severity&dir=ASC&limit=1 HTTP/1.1, referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:08:22Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678858752] (61)Connection refused: [client 10.149.196.118:51311] FastCGI: failed to connect to server "/usr/sbin/isi_papi_d": connect() failed, referer: https://10.149.195.51:8080/Status?loadState=1
    2020-07-06T14:08:22Z <18.3> XXX-ISILON-1 httpd[5406]: [:error] [pid 5406:tid 34678858752] [client 10.149.196.118:51311] FastCGI: incomplete headers (0 bytes) received from server "/usr/sbin/isi_papi_d", referer: https://10.149.195.51:8080/Status?loadState=1

  • Phil.Lam

    4 Apprentice

    641 Posts

    12 Points

    6268

    0

    Posted July 12th, 2020 23:00

    @lamsarr2 ,

    Have you engaged support?

  • lamsarr2

    1 Rookie

    10 Posts

    6257

    0

    Posted July 13th, 2020 01:00

    Yes, I opened a SR and we worked with dell/EMC support for two weeks on this issue but they couldn't solve the problem.

    As far as troubleshooting the Dell/EMC support has done all the possible troubleshooting that he could and he had reached the end of it. After all nodes was re-formatted multiple time and tried with different of OneFS 8.1.2.0 and 8.2.0.0, configured with or without DNS and confirmed front and backend connectivity but finaly the new deployment issue could not be resolved.

  • Phil.Lam

    4 Apprentice

    641 Posts

    12 Points

    6246

    0

    Posted July 13th, 2020 08:00

    @lamsarr2,

    Can you try commands with "-t 300" option? it means "timeout 300 seconds".  The default is 60 seconds I think.

     

    Example:

    isi -t 300 status 

  • 5162

    0

    Posted March 8th, 2021 17:00

    were you able to solve the issue? I am seeing the same problem.

  • Phil.Lam

    4 Apprentice

    641 Posts

    12 Points

    2298

    1

    Posted March 12th, 2021 12:00

    @khayamanjam 

    {ISILON} 8.0+ Cannot use the OneFS Web Interface due to " server-side failure with status code 500" error
    https://www.dell.com/support/kbdoc/000167823/cannot-use-the-onefs-webui-server-side-failure-with-status-code-500?lang=en