PowerFlex "ERROR: Failed to fork timer" when Updating License
Summary: The customer is unable to update their PowerFlex license.
Symptoms
Scenario
The customer may see the following message when attempting to update their license "ERROR: Failed to fork timer"
Symptoms
From the MDM trace log
29/09 15:19:38.957139 0x7f91e2645db8:mosEventLog_PostInternal:00608: New event added. Message: "Command set_license received, User: 'admin'. [774]". Additional info: "" Severity: Info 29/09 15:19:38.957208 (nil):mosIO_close:00137: Closing FD(299) 29/09 15:19:38.964892 (nil):mos_CreateProcessWithTimeout:00050: ERROR: Failed to fork timer 29/09 15:19:38.968880 (nil):mosIO_close:00137: Closing FD(299) 29/09 15:19:38.968906 0x7f91e2645db8:mdmObj_SetLicense:01918: Decode rc: ILLEGAL 29/09 15:19:38.968912 0x7f91e2645db8:mdmObj_SetLicense:01921: Bad license. RC: ILLEGAL 29/09 15:19:38.968946 0x7f91e2645db8:mosEventLog_PostInternal:00608: New event added. Message: "Command set_license was not successful. Error code: The license key is invalid or does not match this version. Please contact Customer Support. [774]". Additional info: "" Severity: Warning
Impact
The customer is unable to update their license until their resource issue is resolved.
Cause
When the scli --set_license command is ran to apply the license, the MDM process is cloned (forked) twice.
The first step succeeded but the command was unable to clone the MDM process. This may occur if the system has insufficient resources to clone the process. In this case, the SVM's memory overcommit configuration limited the system to only half the available memory:
grep 'vm.overcommit' ./server/sysctl.txt -H server/sysctl.txt:vm.overcommit_kbytes = 0 server/sysctl.txt:vm.overcommit_memory = 2 server/sysctl.txt:vm.overcommit_ratio = 50
Resolution
Confirm that the system has sufficient resources available.
Check the sysctl kernel parameters for overcommit of memory:
# sysctl -a |grep commit vm.overcommit_memory = 2 (default is 0) vm.overcommit_ratio = 50 (default is 50)
In this case, having "vm.overcommit_memory" set to two means do not overcommit memory. This fails any memory allocation that exceeds the overcommit limit. The total address space commit for the system is not permitted to exceed swap + a configurable amount (default is 50%) of physical RAM.
To fix this issue, add/edit one of the following in /etc/sysctl.conf:
- Change "vm.overcommit_ratio" to 100, so the OS can commit the total address space available and reboot.