Thank you for taking the time to respond, and you were right, that is what fixed the problem. What i ended up doing was running the following command:

isi_for_array killall isi_webui_d

this did a sighup and restarted the webui_d across all the nodes in the cluster and all is good.

Thanks again!

Usually this happens because the software webui exists and doesn't lean up the pid file.  (Which is localted at /usr/local/www/webkit/appserver.pid) and that just needs to be cleaned up so the appserver can restart.

I'm not sure if it has an auto restart feature or not,  but it would be nice... but this doesn't happen to often so i'm not overly annoyed..

Hello, when i restart the webui service i got this error:

IOError: [Errno 28] No space left on device

/usr/local/sbin/webui_webware_ctl restart: webware failed to start

can some help with this?

thank you

Hopefully it's not the cluster filesystem /ifs that is full...

Troubleshoot a full pool or cluster

Probably it's only a local disk overrunning, which is easier to fix:

Troubleshoot Capacity Alerts on Node Operating System Partitions

hth

-- Peter

Hi Peter, i did everything from pdf but without success

this command newsyslog -R -a /var/log/ dont even work

this is my status:

Filesystem                  Size    Used  Avail Capacity  Mounted on

/dev/mirror/root1          989M    492M    418M    54%      /

devfs                          1.0K    1.0K      0B        100%    /dev

/dev/mirror/var1            989M    989M    -79M    109%    /var

/dev/mirror/var-crash    1.9G    22K    1.8G      0%      /var/crash

/dev/md0                      62M    650K    56M      1%      /tmp/ufp

OneFS                        125T    60T    63T      49%    /ifs

Hey v.adam,

If you ran the following command "find -x /var -type f -size +1M -print0 | xargs -0 ls -lh " then you should have a list of files that you can prune out of your /var directory. your /var/crash directory looks pretty full as well and there is usually dump files and core files that can be removed there as well to help free up space. Just be sure you dont remove the following system directories out of /var/crash

/SupportIQ 

/connectemc 

/cto

But you can remove core files and old log files all you want. otherwise I would open a chat with support over this issue full system partitions can cause node reboots and impacted workflows if not resolved.

Regards,

In addition to D_Tracy's suggestions, we can try to rule out the case

that already removed files which are kept in open status by some process

still consume disk space. 

du -sxh /var

summarizes the disk space taken by all visible files in /var (excluding /var/crash).

If this value is much lower than the number reported by df,

then you have such 'dark matter', and if the related process(es)

cannot be identified, the node would need to be rebooted.

-- Peter

Hi, i started to delete old logs (i think) in /var/log, all files that end with .gz. i think they are old files. after that i have successful start of web service. if this is ok, i will continue with other files. i just need your confirmation that this is ok.

Thank you.

yes this are old logfiles, which can be removed (if not needed for Administration purposes)