PowerFlex 3.6: Implementatie Red Hat OpenStack-component onderbroken
Summary: De implementatie van Red Hat OpenStack Platform (RHOSP) is onderbroken op een SDC-knooppunt vanwege een fout bij het inschakelen van de service. Als het opnieuw wordt geprobeerd, is het succesvol. Aangenomen wordt dit het geval wanneer poortkleppen is ingeschakeld ...
Symptoms
Scenario
Implementatie van een nieuwe RHOSP-omgeving met behulp van Ansible script waarmee een nieuwe PowerFlex 3.6.x SDC wordt geïnstalleerd.
De neutronen-opruim-daemon start niet. Het tijdstip van de storing is gecorreleerd met gebeurtenissen waarbij de poort flapt.
Symptomen
ansible.log - Kan de neutron-cleanup daemon niet starten:
2022-02-25 12:51:20,251 p=784155 u=mistral n=ansible | host: clouddeva-compute-std-9, task: None
2022-02-25 12:51:20,253 p=784155 u=mistral n=ansible | 2022-02-25 12:51:20.253296 | 5254002b-b028-26d5-f46a-000000005317 | FATAL | Enabling the cleanup service | clouddeva-compute-std-3 | error={"changed": false, "msg": "Unable to enable service neutron-cleanup: Created symlink /etc/systemd/system/multi-user.target.wants/neutron-cleanup.service → /usr/lib/systemd/system/neutron-cleanup.service.\nFailed to reload daemon: Connection timed out\n"}
2022-02-25 12:51:20,315 p=784155 u=mistral n=ansible | host: clouddeva-controller-0, task: None
SDC /var/log/messages - failed daemon start
Feb 25 07:48:15 clouddeva-compute-std-3 kernel: PowerFlex netPortFlap_PrepareUpdateMessage:164 :FLAP: Prepared report with 9 (out of 9) disqualified and 7 (out of 13) weak violating portals (unstable) Feb 25 07:48:15 clouddeva-compuFeb 25 07:48:16 clouddeva-compute-std-3 ansible-stat[24479]: Invoked with path=/etc/modules-load.d/openvswitch.conf follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True Feb 25 07:48:17 clouddeva-compute-ste-std-3 kernel: [4295286250] Connected to SDS with ID 37bb9ae500000009 Feb 25 07:48:15 clouddeva-compute-std-3 kernel: [4295286255] Disconnected from SDS with ID 37bb9ae600000008 Feb 25 07:48:15 clouddeva-compute-std-3 kernel: [4295286261] Disconnected from SDS with ID 37bb9adf00000003 Feb 25 07:48:15 clouddeva-compute-std-3 kernel: [4295286328] Connected to SDS with ID 37bb9ae000000004 Feb 25 07:48:15 clouddeva-compute-std-3 ansible-stat[24380]: Invoked with path=/etc/modules follow=False get_md5=False get_checksum=True get_mime=True get_attributes=True checksum_algorithm=sha1 Feb 25 07:48:15 clouddeva-compute-std-3 kernel: [4295286356] Connected to SDS with ID 37bb9ae600000008 Feb 25 07:48:16 clouddeva-compute-std-3 ansible-modprobe[24424]: Invoked with name=openvswitch state=present params= Feb 25 07:48:16 clouddeva-compute-std-3 kernel: [4295286563] Connected to SDS with ID 37bb9adf00000003 Feb 25 07:48:16 clouddeva-compute-std-3 kernel: [4295286850] Connected to SDS with ID 37bb9add00000001 td-3 ansible-copy[24523]: Invoked with dest=/etc/modules-load.d/openvswitch.conf src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1645793296.6471314-801998-190517710088290/source mode=None follow=False _original_basename=module-load.conf.j2 checksum=aca7c8bb7013e03e6cf531689fb6220d9e469d63 backup=False force=True unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None Feb 25 07:48:18 clouddeva-compute-std-3 ansible-stat[24619]: Invoked with path=/usr/libexec/neutron-cleanup follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True Feb 25 07:48:18 clouddeva-compute-std-3 ansible-copy[24693]: Invoked with dest=/usr/libexec/neutron-cleanup force=True mode=0755 src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1645793298.1814697-802218-228907330848454/source _original_basename=tmpmhvsxu9b follow=False checksum=a9feff7c7567b3ae8baa2006455b8cc8b370d239 backup=False unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None Feb 25 07:48:19 clouddeva-compute-std-3 ansible-stat[24759]: Invoked with path=/usr/lib/systemd/system/neutron-cleanup.service follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True Feb 25 07:48:19 clouddeva-compute-std-3 kernel: PowerFlex netCon_IsKaNeeded:4787 :CON 0000000058dfd327 peer 10.232.248.5 didn't receive message for 30 iterations. Marking as down Feb 25 07:48:19 clouddeva-compute-std-3 ansible-copy[24803]: Invoked with dest=/usr/lib/systemd/system/neutron-cleanup.service force=True src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1645793299.0068667-802347-201781290304692/source _original_basename=tmpx3w9vtsi follow=False checksum=6b6cd9f074903a28d054eb530a10c7235d0c39fc backup=False unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None mode=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None Feb 25 07:48:20 clouddeva-compute-std-3 ansible-systemd[24867]: Invoked with enabled=True name=neutron-cleanup daemon_reload=False daemon_reexec=False no_block=False state=None force=None masked=None user=None scope=None Feb 25 07:48:20 clouddeva-compute-std-3 systemd[1]: Reloading.
SDC /var/log/messages - successful daemon start
Mar 2 12:25:43 clouddeva-compute-std-0 ansible-modprobe[36359]: Invoked with name=openvswitch state=present params= Mar 2 12:25:43 clouddeva-compute-std-0 ansible-stat[36411]: Invoked with path=/etc/modules-load.d/openvswitch.conf follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True Mar 2 12:25:44 clouddeva-compute-std-0 ansible-copy[36472]: Invoked with dest=/etc/modules-load.d/openvswitch.conf src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1646241943.6915667-45654-245764505198253/source mode=None follow=False _original_basename=module-load.conf.j2 checksum=aca7c8bb7013e03e6cf531689fb6220d9e469d63 backup=False force=True unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None Mar 2 12:25:45 clouddeva-compute-std-0 ansible-stat[36582]: Invoked with path=/usr/libexec/neutron-cleanup follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True Mar 2 12:25:45 clouddeva-compute-std-0 ansible-copy[36628]: Invoked with dest=/usr/libexec/neutron-cleanup force=True mode=0755 src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1646241945.256886-45861-233404909922568/source _original_basename=tmpce1pezaw follow=False checksum=a9feff7c7567b3ae8baa2006455b8cc8b370d239 backup=False unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None Mar 2 12:25:46 clouddeva-compute-std-0 ansible-stat[36723]: Invoked with path=/usr/lib/systemd/system/neutron-cleanup.service follow=False get_checksum=True checksum_algorithm=sha1 get_md5=False get_mime=True get_attributes=True Mar 2 12:25:46 clouddeva-compute-std-0 ansible-copy[36767]: Invoked with dest=/usr/lib/systemd/system/neutron-cleanup.service force=True src=/home/tripleo-admin/.ansible/tmp/ansible-tmp-1646241946.3689635-46015-253217383764482/source _original_basename=tmphwqutgs0 follow=False checksum=6b6cd9f074903a28d054eb530a10c7235d0c39fc backup=False unsafe_writes=False content=NOT_LOGGING_PARAMETER validate=None directory_mode=None remote_src=None local_follow=None mode=None owner=None group=None seuser=None serole=None selevel=None setype=None attributes=None regexp=None delimiter=None Mar 2 12:25:47 clouddeva-compute-std-0 ansible-systemd[36808]: Invoked with enabled=True name=neutron-cleanup daemon_reload=False daemon_reexec=False no_block=False state=None force=None masked=None user=None scope=None Mar 2 12:25:47 clouddeva-compute-std-0 systemd[1]: Reloading.
Impact
RHOSP-implementatie is onderbroken en moet opnieuw worden geprobeerd om door te gaan.
Cause
Unknown
Resolution
U kunt een van de volgende tijdelijke oplossingen gebruiken om de onderbroken status op te lossen:
- Ga verder met het opnieuw proberen.
- Schakel poortflappen uit (als onderdeel van het geautomatiseerde proces) zodra de SDC/s zijn geregistreerd in PowerFlex door de onderstaande opdrachten uit te voeren.
Schakel poortflapping uit voor een specifieke SDC:
scli --set_performance_parameters --sdc_id --tech --sdc_port_flap_feature_enabled 0
Schakel poortflapping uit voor alle SDC-exemplaren:
scli --set_performance_parameters --all_sdc --tech --sdc_port_flap_feature_enabled 0
Versies waarop dit van toepassing is
PowerFlex 3.6 en hoger