ViPR SRM: Watch4Net: APG Backend Service Does Not Start Up
Summary: Because of Either No Free TMP Disk Space or Corrupted Index File.
Symptoms
Backend APG3 Service is Stopped and attempts to start it Fail.
The Backend Service Logs Show:
(\opt\APG\Backends\APG-Backend\apg3\logs)
SEVERE -- [2017-07-20 13:23:47 IST] -- Cache::main(): Unable to start APG Backend! com.watch4net.apg.v2.common.config.InvalidConfigurationException: Can't initialize group conf - - Caused by: com.watch4net.apg.v2.common.io.InOutException: Unable to initialize variables for group 'conf'! - - Caused by: java.sql.SQLException: Incorrect key file for table './apg3/data_variable.MYI'; try to repair it
Cause
- MySQL tmp Directory is out of Space (Default is /tmp)*
- The Index File ".MYI" is Corrupted and Must be Regenerated
*Check "tmpdir" Setting in (/opt/APG/Databases/MySQL/Default/my.cnf) for the path of MySQL TMP.
Resolution
-
Attempt to Repair the Database
/opt/AGP/bin/mysql-database-check.sh Enter username: [apg] root Enter database: [apg] apg3 Enter password: watch4net
-
Check Available Space on the Disk where the TMP Directory of MySQL is Located. Using:
df -h
If the Disk is out of Space:- Delete Old Data if any
- Increase the Space of that Disk
- Change the tmpdir setting to point to a different Path with Enough Disk Space
-
If No Space Issues found, Regenerate Index file .MYI
- Rename Current Index file
mv data_variable.MYI data_variable.MYI.bkp - Log in to MySQL Client
/opt/AGP/bin/mysql-client.sh Enter username: [apg] root Enter database: [apg] apg3 Enter password: watch4net
- Regenerate the Index file for Table data_variable
REPAIR TABLE data_variable USE_FRM;
- Rename Current Index file
After it Completes Successfully, you should find a Newly Created "data_variable.MYI" in the data Path of the Database. And the Service should start again Successfully.
Additional Information
Every Table in a mysql Database has three Files (.MYD - .MYI - .frm)
Check /opt/APG/Databases/MySQL/Default/data/apg3
.frm is the Table Format file (How Many Columns - Data Types - Unique - Primary Key - ...)
.MYD is the Actual data in the Table.
.MYI is the Index of Table. To speed up the Search and Querying Process.
The Error seen is for ".MYI" file for "Data_Variable" Table of "APG3" Database. Since it is the index that is corrupted, it can be generated.
If the Corrupted file is ".MYD" The Best Solution is to restore from a Backup.