Start a Conversation


Go to Solution

1 Rookie


2 Posts


July 9th, 2024 20:46

gsan error stripedns::validateandupdatediskstats

One of my clients is using Avamar as VDP 6.1 (gsan version 7.2.80-167) and was careless enough to reboot the VM without stopping gsan correctly. As a result gsan fails to start with error [gsan] ERROR: <1237> stripedns::validateandupdatediskstats bool check name=device number value=/data06/cur is 0x4101
I did an xfs check of the disk with /data06 and found no errors. 
Unfortunately I could not find in gsan --help, EMC® Avamar® 7.3 Technical Addendum & etc any commands to check and fix gsan
Support for this product ended a long time ago.
The data in this avamar instance is no longer interesting as it has been duplicated.
I am writing here a question to decide whether to deal with Avamar anywhere else in the future, or to give up as an unreliable fragile thing without a public knowledge base for repair. 

tail -f /data01/cur/err.log
2024/07/08-15:19:29.01740 {P0.0} [gsan] <0642> gsan version 7.2.80-167 built Oct 22 2018 09:19:26 msg format 13-10
2024/07/08-15:19:29.01751 {P0.0} [gsan] <1299> Using Intel implementation of SHA1
2024/07/08-15:19:31.09736 {P0.0} [gsan] ERROR: <1237> stripedns::validateandupdatediskstats bool check name=device number value=/data06/cur is 0x4101

tail -f /data01/cur/gsan.log
2024/07/08-15:19:29.01728 {P0.0} [gsan] <0017> Node restarted
2024/07/08-15:19:29.01736 {P0.0} [gsan] Main gsan pid = 23548
2024/07/08-15:19:29.01740 {P0.0} [gsan] <0642> gsan version 7.2.80-167 built Oct 22 2018 09:19:26 msg format 13-10
2024/07/08-15:19:29.01745 {P0.0} [gsan] gsan::run version=7.2.80-167 date=Oct 22 2018 09:19:26 msg=13-10 SSL=TLSv1 OpenSSL 1.0.2a-fips 19 Mar 2015 Zlib=1.2.7 LZO=1.08 Jul 12 2002 platform=Linux OS=SLES-64 Processor=x86_64
2024/07/08-15:19:29.01748 {P0.0} [gsan] gsan::run mtex::hastimedlocks = 1
2024/07/08-15:19:29.01751 {P0.0} [gsan] <1299> Using Intel implementation of SHA1
2024/07/08-15:19:29.01754 {P0.0} [gsan] gsan::run checking clockthread timeout=0
2024/07/08-15:19:29.01757 {P0.0} [gsan] gsan::run exceptions may be thrown in common components
2024/07/08-15:19:29.01760 {P0.0} [gsan] debug::addsrcfileexception added file=bitvect.cpp msgerr=19
2024/07/08-15:19:29.01762 {P0.0} [gsan] debug::addsrcfileexception added file=boolvect.cpp msgerr=19
2024/07/08-15:19:29.01765 {P0.0} [gsan] debug::addsrcfileexception added file=compiterator.cpp msgerr=19
2024/07/08-15:19:29.01768 {P0.0} [gsan] debug::addsrcfileexception added file=direlem.cpp msgerr=19
2024/07/08-15:19:29.01771 {P0.0} [gsan] debug::addsrcfileexception added file=direlemimpl.cpp msgerr=19
2024/07/08-15:19:29.01773 {P0.0} [gsan] debug::addsrcfileexception added file=filecat.cpp msgerr=19
2024/07/08-15:19:29.01776 {P0.0} [gsan] debug::addsrcfileexception added file=filesys.cpp msgerr=19
2024/07/08-15:19:29.01779 {P0.0} [gsan] debug::addsrcfileexception added file=iogroup.cpp msgerr=19
2024/07/08-15:19:29.01781 {P0.0} [gsan] debug::addsrcfileexception added file=ipaddr.cpp msgerr=19
2024/07/08-15:19:29.01784 {P0.0} [gsan] debug::addsrcfileexception added file=ipaddr4.cpp msgerr=19
2024/07/08-15:19:29.01787 {P0.0} [gsan] debug::addsrcfileexception added file=persistentobj.cpp msgerr=19
2024/07/08-15:19:29.01789 {P0.0} [gsan] debug::addsrcfileexception added file=redblacktree.cpp msgerr=19
2024/07/08-15:19:29.01793 {P0.0} [gsan] debug::addsrcfileexception added file=sslcontext.cpp msgerr=19
2024/07/08-15:19:29.01796 {P0.0} [gsan] debug::addsrcfileexception added file=stripeinfo.cpp msgerr=19
2024/07/08-15:19:29.01799 {P0.0} [gsan] debug::addsrcfileexception added file=tchunkinfolist.cpp msgerr=19
2024/07/08-15:19:29.01802 {P0.0} [gsan] debug::addsrcfileexception added file=tchunktype.cpp msgerr=19
2024/07/08-15:19:29.01805 {P0.0} [gsan] debug::addsrcfileexception added file=textfile.cpp msgerr=19
2024/07/08-15:19:29.01807 {P0.0} [gsan] debug::addsrcfileexception added file=tiooffset.cpp msgerr=19
2024/07/08-15:19:29.01810 {P0.0} [gsan] debug::addsrcfileexception added file=upath_base.cpp msgerr=19
2024/07/08-15:19:29.01813 {P0.0} [gsan] tcpsockimpl::setmonitor monitor=0 enable=1
2024/07/08-15:19:29.01818 {P0.0} [gsan] dbstatthread::dbstatthread monotonic start nowm=20899018176799 nowp=1720451969
2024/07/08-15:19:29.01831 {P0.0} [dbstat] Startup, pid=23549 tid=32 stack=0x7fc214962ccf td=0x7fc212618368
2024/07/08-15:19:29.01875 {P0.0} [gsan] gsan::run Setting curraddr to
2024/07/08-15:19:29.01920 {P0.0} [gsan] gsan::run all 12 disks are writeable
2024/07/08-15:19:29.01926 {P0.0} [gsan] gsan::run max open files 800000 / 800000
2024/07/08-15:19:29.01931 {P0.0} [gsan] tstatedoc::restore++ fname=/data01/cur/statedoc-eventmanager.xml
2024/07/08-15:19:29.01986 {P0.0} [gsan] tstatedoc::restore--
2024/07/08-15:19:29.02062 {P0.0} [gsan] stripetab::stripetab sizeof(stripelistitem) = 160
2024/07/08-15:19:29.02067 {P0.0} [gsan] stripetab::stripetab sizeof(stripeitemtype) = 168
2024/07/08-15:19:29.02071 {P0.0} [gsan] nodetab::nodetab sizeof(nodelistitem) = 496
2024/07/08-15:19:29.02073 {P0.0} [gsan] nodetab::nodetab sizeof(nodeitemtype) = 512
2024/07/08-15:19:29.02303 {P0.0} [gsan] tstatedoc::restore++ fname=/data01/cur/statedoc-perfstatus.xml
2024/07/08-15:19:29.61519 {P0.0} [gsan] tstatedoc::restore--
2024/07/08-15:19:30.04683 {P0.0} [gsan] stripedns::stripedns bootendtime=2024-Jul-08 09:41:09
2024/07/08-15:19:30.04693 {P0.0} [gsan] tabfile::open filename=/data01/cur/ size=15642880 accessmode=2
2024/07/08-15:19:30.04699 {P0.0} [gsan] filemap::_init fixmsyncbug fsize=15642880 msize=15646976
2024/07/08-15:19:30.04714 {P0.0} [sync] Startup, pid=23550 tid=33 stack=0x7fc214951ccf td=0x7fc21185d568
2024/07/08-15:19:30.04721 {P0.0} [gsan] stripedns::stripedns reloading disksyncdone=1 shutdowntime=1720415937
2024/07/08-15:19:30.04746 {P0.0} [gsan] tstripetabheader::startup cpseqno=5112
2024/07/08-15:19:30.04751 {P0.0} [gsan] stripedns::stripedns nodelist setbase
2024/07/08-15:19:30.04756 {P0.0} [gsan] stripedns::stripedns datacenterlist setbase
2024/07/08-15:19:30.04760 {P0.0} [gsan] stripedns::stripedns tunnels setbase
2024/07/08-15:19:30.04764 {P0.0} [gsan] stripedns::stripedns stripelist setbase
2024/07/08-15:19:31.09736 {P0.0} [gsan] ERROR: <1237> stripedns::validateandupdatediskstats bool check name=device number value=/data06/cur is 0x4101
2024/07/08-15:19:31.44056 {P0.0} [gsan] tstatedoc::save++ fname=/data01/cur/statedoc-perfstatus.xml
2024/07/08-15:19:33.06560 {P0.0} [gsan] tstatedoc::save--
2024/07/08-15:19:33.20064 {P0.0} [dbstat] Thread finished dbstat
2024/07/08-15:19:33.20077 {P0.0} [gsan] gsan::~gsan, deleted dpnman
2024/07/08-15:19:33.20080 {P0.0} [gsan] gsan::~gsan, deleted dispatcher
2024/07/08-15:19:33.20084 {P0.0} [gsan] gsan::~gsan, deleted strman
2024/07/08-15:19:33.20087 {P0.0} [gsan] gsan::~gsan, deleted recv
2024/07/08-15:19:33.20089 {P0.0} [gsan] gsan::~gsan, deleted exec
2024/07/08-15:19:33.20094 {P0.0} [gsan] gsan::~gsan, deleted sam
2024/07/08-15:19:33.20097 {P0.0} [gsan] gsan::~gsan, deleted threadpool
2024/07/08-15:19:33.20100 {P0.0} [gsan] gsan::~gsan, deleted filepool
2024/07/08-15:19:33.20104 {P0.0} [gsan] gsan::~gsan, deleted dns
2024/07/08-15:19:33.20171 {P0.0} [gsan] tstatedoc::save++ fname=/data01/cur/statedoc-eventmanager.xml
2024/07/08-15:19:33.20550 {P0.0} [gsan] tstatedoc::save--
2024/07/08-15:19:33.20566 {P0.0} [gsan] gsan::~gsan, deleted eventman
2024/07/08-15:19:33.20604 {P0.0} [gsan] Thread finished gsan

1 Rookie


2 Posts

July 10th, 2024 11:31

The reason was that VDP was rebooted during backup jobs execution, and the snapshot disks of VMs being backed up were still attached to it. they prevented the gsan from starting correctly.

Self fixed


No Events found!