PowerProtect: vmware-service is unable to start.

Broadcom vCenter/vSphere related operations are impacted due to service named vcenter-service is failing to start on PowerProtect Data Manager server. 
 

# vmware-service start 
Starting... 
Failed to start Please check log file: /var/log/brs/vmware-service/vmware-service.out

Log file for vmware-service on PowerProtect Data Manager service has event similar to the following reported in it:
 

HH:MM:SS.mss [main] INFO c.e.b.v.s.treeview.DatabaseService - Backing up database ... Standard Commons Logging discovery in action with spring-jcl: please remove commons-logging.jar from classpath in order to avoid potential conflicts ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console... ERROR SpringApplication Application run failed java.lang.IllegalStateException: java.io.FileNotFoundException: config/application.yml (Permission denied) at org.springframework.beans.factory.config.YamlProcessor.handleProcessError(YamlProcessor.java:222) at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:214) at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:166) at org.springframework.boot.env.OriginTrackedYamlLoader.load(OriginTrackedYamlLoader.java:84) at org.springframework.boot.env.YamlPropertySourceLoader.load(YamlPropertySourceLoader.java:50) at org.springframework.boot.context.config.StandardConfigDataLoader.load(StandardConfigDataLoader.java:54) at org.springframework.boot.context.config.StandardConfigDataLoader.load(StandardConfigDataLoader.java:36) at org.springframework.boot.context.config.ConfigDataLoaders.load(ConfigDataLoaders.java:96) at org.springframework.boot.context.config.ConfigDataImporter.load(ConfigDataImporter.java:132) at org.springframework.boot.context.config.ConfigDataImporter.resolveAndLoad(ConfigDataImporter.java:87) at org.springframework.boot.context.config.ConfigDataEnvironmentContributors.withProcessedImports(ConfigDataEnvironmentContributors.java:115) at org.springframework.boot.context.config.ConfigDataEnvironment.processInitial(ConfigDataEnvironment.java:242) at org.springframework.boot.context.config.ConfigDataEnvironment.processAndApply(ConfigDataEnvironment.java:229) at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:96) at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:89) at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEnvironmentPreparedEvent(EnvironmentPostProcessorApplicationListener.java:109) at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEvent(EnvironmentPostProcessorApplicationListener.java:94) at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:174) at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:167) at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:145) at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:133) at org.springframework.boot.context.event.EventPublishingRunListener.multicastInitialEvent(EventPublishingRunListener.java:136) at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:81) at org.springframework.boot.SpringApplicationRunListeners.lambda$environmentPrepared$2(SpringApplicationRunListeners.java:64) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118) at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112) at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:63) at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:361) at org.springframework.boot.SpringApplication.run(SpringApplication.java:320) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1327) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1316) at com.emc.brs.vmwareservice.application.VmwareServiceApp.main(VmwareServiceApp.java:58) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) at org.springframework.boot.loader.Launcher.launch(Launcher.java:107) at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) at org.springframework.boot.loader.thin.ThinJarLauncher.launch(ThinJarLauncher.java:195) at org.springframework.boot.loader.thin.ThinJarLauncher.main(ThinJarLauncher.java:141) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at org.springframework.boot.loader.wrapper.ThinJarWrapper.launch(ThinJarWrapper.java:139) at org.springframework.boot.loader.wrapper.ThinJarWrapper.main(ThinJarWrapper.java:107) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:569) at com.emc.brs.ThinJarLauncher.directlyRun(ThinJarLauncher.java:33) at com.emc.brs.ThinJarLauncher.main(ThinJarLauncher.java:19) Caused by: java.io.FileNotFoundException: config/application.yml (Permission denied) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:216) at java.base/java.io.FileInputStream.(FileInputStream.java:157) at java.base/java.io.FileInputStream.(FileInputStream.java:111) at java.base/sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:86) at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:189) at org.springframework.core.io.UrlResource.getInputStream(UrlResource.java:231) at org.springframework.boot.origin.OriginTrackedResource.getInputStream(OriginTrackedResource.java:61) at org.springframework.beans.factory.config.YamlProcessor.process(YamlProcessor.java:198) ... 52 more

Permissions on the "/usr/local/brs/lib/vmware-service/config/application.config" file are not correct. By default this file does not exist. When the file is created to make custom configuration changes, permissions and ownership of the file is not set properly.

The following steps can be followed to resolve the issue:

1. Log in to the PowerProtect Data Manager server using admin account.
2. Change directory to "/usr/local/brs/lib/vmware-service/config/"

   cd /usr/local/brs/lib/vmware-service/config/

3. Check permissions on the application.config file.
4. If the application.cofig file is not needed for any custom configuration, it can be moved out from this folder. 
5. If this file is required for certain custom configuration setting, use following commands to set expected permissions and ownership.
    

   sudo chown admin:app application.yml 
   sudo chmod 750 application.yml

    
6. Start vmware-service.

   vmware-service start