APEX Cloud Platform for Red Hat OpenShift: Pod container waiting longer than 1 hour in the namespace openshift-marketplace
Summary: Dail home case with the event "Pod container waiting longer than 1 hour" in namespace openshift-marketplace
Symptoms
A dial-home case is created for OpenShift Event Code: 1038CLST0005, the event mentions pod container waiting longer than 1 hour in namespace openshift-marketplace
Cause
The cause is a known bug in OpenShift Container Platform 4.10-4.13
Resolution
Per Red Hat KCS 7016065, the current solution is to upgrade to OpenShift Container Platform 4.14+.
NOTE: An older Red Hat KCS 7011359 had a workaround, but caused problems and has been removed - it is now forwarding to KCS 7016065.
For a customer that can't upgrade to 4.14, follow this workaround:
1) Ignore the event 1038CLST0005 for namespace openshift-marketplace if the OCP version is 4.10 to 4.13.
2) We can silence the event in the namespace openshift-marketplace.
- Go to Observe -> Alerting -> Silences
- Click Create silence
- Under Duration, in "For..." select "-" and then edit "Until..." (change to a future time so there will not be any KubeContainerWaiting alerts for the container registry-server)
- Add labels "alertname=KubeContainerWaiting", "namespace=openshift-marketplace", "container=registry-server" (if you we do not add container, it silences the alert for all the pods in namespace “openshift-marketplace”)

Additional Information
This KB is only for events in the namespace openshift-marketplace , if the event and dial home is for pods in other namespace, don't silence the alert and troubleshoot as necessary.