InsightIQ "HTTP request timed out" or blank when generating FSA reports

In InsightIQ 4.0 to 4.2 and OneFS 8.0.x (or later versions), the OneFS RESTful API collects FSA report data. If InsightIQ is unable to collect this data, the FSA report may not load, may partially load, or may not allow users to navigate the directories.

The issue occurs when the InsightIQ server times out the API call to OneFS too soon.

To validate this cause, run the following command (replacing CLUSTER_GUID with the proper directory):

      $ grep "HTTP request timed out" /var/log/insightiq_clusters/CLUSTER_GUID/api_connection.log

Output similar to the following appears:

    2017-06-12 13:07:50,114,114 ERROR [api.py: 187] HTTP request timed out: https://172.22.33.27:8080/platform/3/fsa/results/4505/directories/4321902598
    2017-06-12 13:08:13,782,782 ERROR [api.py: 187] HTTP request timed out: https://172.22.33.27:8080/platform/3/fsa/results/4505/directories/4321902598

The defined timeout value is seen by running this command:

$ grep 'read timeout=' /var/log/insightiq.log

The output looks like this. In this example output, the timeout is 20 seconds:

2018-04-12 04:18:30,203 WARNI [insightiq.lib.cluster.errors] APITimedOutError: HTTPSConnectionPool(host='10.104.6.138', port=8080): Read timed out. (read timeout=20) URI: https://10.104.6.138:8080/platform/3/statistics/history

To resolve the issue, increase the default timeout for API calls to OneFS by editing the api.py file.

Run this command to edit the InsightIQ api.py file:

    $ sudo vi /usr/share/isilon/lib/python2.7/site-packages/insightiq/lib/api_connection/api.py

On line 35, modify TIMEOUT = 20 to TIMEOUT = 300.

Then restart the InsightIQ server:

    $ sudo service insightiq restart

For InsightIQ 4.2, the path is /usr/share/isilon/lib/python3.10/site-packages/insightiq/lib/api_connection/api.py.