PowerFlex 3.6：Red Hat OpenStackコンポーネント導入の一時停止

シナリオ

新しいPowerFlex 3.6.x SDCをインストールするAnsibleスクリプトを使用した新しいRHOSP環境の導入。

neutron-cleanup デーモンの起動に失敗します。障害の時期は、ポート フラッピング機能イベントと相関しています。

現象

ansible.log - neutron-cleanup デーモンの起動に失敗しました。

2022-02-25 12:51:20,251 p=784155 u=mistral n=ansible | host: clouddeva-compute-std-9, task: None
2022-02-25 12:51:20,253 p=784155 u=mistral n=ansible | 2022-02-25 12:51:20.253296 | 5254002b-b028-26d5-f46a-000000005317 |      FATAL | Enabling the cleanup service | clouddeva-compute-std-3 | error={"changed": false, "msg": "Unable to enable service neutron-cleanup: Created symlink /etc/systemd/system/multi-user.target.wants/neutron-cleanup.service → /usr/lib/systemd/system/neutron-cleanup.service.\nFailed to reload daemon: Connection timed out\n"}
2022-02-25 12:51:20,315 p=784155 u=mistral n=ansible | host: clouddeva-controller-0, task: None

SDC /var/log/messages - デーモンの起動に失敗しました

Feb 25 07:48:15 clouddeva-compute-std-3 kernel: PowerFlex netPortFlap_PrepareUpdateMessage:164 :FLAP: Prepared report with 9 (out of 9) disqualified and 7 (out of 13) weak violating portals (unstable)
Feb 25 07:48:15 clouddeva-compuFeb 25 07:48:16 clouddeva-compute-std-3 ansible-stat[24479]: Invoked with path=/etc/modules-load.d/openvswitch.conf follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True
Feb 25 07:48:17 clouddeva-compute-ste-std-3 kernel: [4295286250] Connected to SDS with ID 37bb9ae500000009
Feb 25 07:48:15 clouddeva-compute-std-3 kernel: [4295286255] Disconnected from SDS with ID 37bb9ae600000008
Feb 25 07:48:15 clouddeva-compute-std-3 kernel: [4295286261] Disconnected from SDS with ID 37bb9adf00000003
Feb 25 07:48:15 clouddeva-compute-std-3 kernel: [4295286328] Connected to SDS with ID 37bb9ae000000004
Feb 25 07:48:15 clouddeva-compute-std-3 ansible-stat[24380]: Invoked with path=/etc/modules follow=False get_md5=False get_checksum=True get_mime=True get_attributes=True checksum_algorithm=sha1
Feb 25 07:48:15 clouddeva-compute-std-3 kernel: [4295286356] Connected to SDS with ID 37bb9ae600000008
Feb 25 07:48:16 clouddeva-compute-std-3 ansible-modprobe[24424]: Invoked with name=openvswitch state=present params=
Feb 25 07:48:16 clouddeva-compute-std-3 kernel: [4295286563] Connected to SDS with ID 37bb9adf00000003
Feb 25 07:48:16 clouddeva-compute-std-3 kernel: [4295286850] Connected to SDS with ID 37bb9add00000001
td-3 ansible-copy[24523]: Invoked with dest=/etc/modules-load.d/openvswitch.conf src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1645793296.6471314-801998-190517710088290/source mode=None follow=False _original_basename=module-load.conf.j2 checksum=aca7c8bb7013e03e6cf531689fb6220d9e469d63 backup=False force=True unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None
Feb 25 07:48:18 clouddeva-compute-std-3 ansible-stat[24619]: Invoked with path=/usr/libexec/neutron-cleanup follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True
Feb 25 07:48:18 clouddeva-compute-std-3 ansible-copy[24693]: Invoked with dest=/usr/libexec/neutron-cleanup force=True mode=0755 src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1645793298.1814697-802218-228907330848454/source _original_basename=tmpmhvsxu9b follow=False checksum=a9feff7c7567b3ae8baa2006455b8cc8b370d239 backup=False unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None
Feb 25 07:48:19 clouddeva-compute-std-3 ansible-stat[24759]: Invoked with path=/usr/lib/systemd/system/neutron-cleanup.service follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True
Feb 25 07:48:19 clouddeva-compute-std-3 kernel: PowerFlex netCon_IsKaNeeded:4787 :CON 0000000058dfd327 peer 10.232.248.5 didn't receive message for 30 iterations.  Marking as down
Feb 25 07:48:19 clouddeva-compute-std-3 ansible-copy[24803]: Invoked with dest=/usr/lib/systemd/system/neutron-cleanup.service force=True src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1645793299.0068667-802347-201781290304692/source _original_basename=tmpx3w9vtsi follow=False checksum=6b6cd9f074903a28d054eb530a10c7235d0c39fc backup=False unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None mode=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None
Feb 25 07:48:20 clouddeva-compute-std-3 ansible-systemd[24867]: Invoked with enabled=True name=neutron-cleanup daemon_reload=False daemon_reexec=False no_block=False state=None force=None masked=None user=None scope=None
Feb 25 07:48:20 clouddeva-compute-std-3 systemd[1]: Reloading.

SDC /var/log/messages - デーモンの正常な起動

Mar  2 12:25:43 clouddeva-compute-std-0 ansible-modprobe[36359]: Invoked with name=openvswitch state=present params=
Mar  2 12:25:43 clouddeva-compute-std-0 ansible-stat[36411]: Invoked with path=/etc/modules-load.d/openvswitch.conf follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True
Mar  2 12:25:44 clouddeva-compute-std-0 ansible-copy[36472]: Invoked with dest=/etc/modules-load.d/openvswitch.conf src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1646241943.6915667-45654-245764505198253/source mode=None follow=False _original_basename=module-load.conf.j2 checksum=aca7c8bb7013e03e6cf531689fb6220d9e469d63 backup=False force=True unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None
Mar  2 12:25:45 clouddeva-compute-std-0 ansible-stat[36582]: Invoked with path=/usr/libexec/neutron-cleanup follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True
Mar  2 12:25:45 clouddeva-compute-std-0 ansible-copy[36628]: Invoked with dest=/usr/libexec/neutron-cleanup force=True mode=0755 src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1646241945.256886-45861-233404909922568/source _original_basename=tmpce1pezaw follow=False checksum=a9feff7c7567b3ae8baa2006455b8cc8b370d239 backup=False unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None
Mar  2 12:25:46 clouddeva-compute-std-0 ansible-stat[36723]: Invoked with path=/usr/lib/systemd/system/neutron-cleanup.service follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True
Mar  2 12:25:46 clouddeva-compute-std-0 ansible-copy[36767]: Invoked with dest=/usr/lib/systemd/system/neutron-cleanup.service force=True src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1646241946.3689635-46015-253217383764482/source _original_basename=tmphwqutgs0 follow=False checksum=6b6cd9f074903a28d054eb530a10c7235d0c39fc backup=False unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None mode=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None
Mar  2 12:25:47 clouddeva-compute-std-0 ansible-systemd[36808]: Invoked with enabled=True name=neutron-cleanup daemon_reload=False daemon_reexec=False no_block=False state=None force=None masked=None user=None scope=None
Mar  2 12:25:47 clouddeva-compute-std-0 systemd[1]: Reloading.

問題

RHOSP導入が一時停止しました。続行するには再試行する必要があります。

不明

次のいずれかの回避策を使用して、一時停止状態を解決できます。

1. 再試行を続行します。
2. 次のコマンドを実行して、SDCがPowerFlexに登録されたら、ポート フラッピングを(自動化プロセスの一部として)無効にします。

特定のSDCのポート フラッピングを無効にします。

 scli --set_performance_parameters --sdc_id  --tech --sdc_port_flap_feature_enabled 0

すべてのSDCに対してポート フラッピングを無効にします。

 scli --set_performance_parameters --all_sdc --tech --sdc_port_flap_feature_enabled 0

問題が発生するバージョン

PowerFlex 3.6以降