This post is more than 5 years old
76 Posts
0
3912
DD prompting for password
My Data domain started prompting for passwords and flat out giving permission denied to a few SQL agent accounts running backup jobs.
The share is r/w to the hosts and I even made the SQL account owner of the share through windows domain admin account.
I can login to the SQL server as the SQL agent account and browse to the share but yet the SQL job still fails.
It's very strange and we have been looking at this for days. It seems to be a problem upstream to the share because tcpdump shows the SMB request never gets past the initial SMB Setup andX Request. Responds with STATUS_LOGON_FAILURE.
account has full read write access. I have made the share completely wide open but it's still getting denied.
2008 Domain Controllers, 208-2012 SQL Server running transact SQL job to dump to the database.
SQL agent is running as an AD service account. Worked for weeks then just stopped at 4AM with no changes.
Tried:
- rebooted DD
- re-joined DD to the domain.
- used cifs troubleshoot to validate the sqlaccount could be looked up from the dd
- used net dns to verify client host and domain controllers resolve.
- Changed ownership of the share to the sql agent.
- updated to ddos 5.2.5 from 5.1
- scratched head vigorously
ScotN
76 Posts
1
June 14th, 2014 12:00
Solution appears to be cifs signing.
Disabling signing on 2008 R2 does NOT work. I disabled this rebooted and it still randomly tried to use a signed SMB connection. Yes, randomly I have SQL agent logs that show hit or miss success with no changes until I set smb signing to auto on the data domain.
#> cifs option set "server signing" auto
Restart cifs.
HarshG
8 Posts
0
June 14th, 2014 08:00
Hi Scot, have u checked your file server on data domain. The problem is with file server on DD . I'll illustrate u better give me a day. Mean while you check your file server on dd. Is it enabled or not
HarshG
8 Posts
1
June 14th, 2014 08:00
Check for
cifs option set "server signing"
This option needs to be enabled on the DD side for SMB1 servers
HarshG
8 Posts
0
June 14th, 2014 09:00
So its working earlier and now it stopped in mid night.
And is it working on new share?
ScotN
76 Posts
0
June 14th, 2014 09:00
I went the other way and disabled signing on the client side. That did not work but I will try dd setting. The thing that I don't understand is how the job could start failing in the middle of the night and never work again.
If I setup a new share with the se perms it works. So what is changing that makes the job fail on multiple windows servers ? My instinct is telling me it would be related to a move in as domain controllers to 2008 a few weeks ago and is just now present a problem some how.
ScotN
76 Posts
0
June 14th, 2014 10:00
Yep, strangest thing.
Same host acls, * for user acl new one works old doesn't.
AND if I delete the old share and create a new one with the same name it still doesn't work.
It's like windows 2008 has remembered something about the share and is using that property incorrectly but I can't see what in tcpdump or in messages with smb logging at 9.
HarshG
8 Posts
0
June 14th, 2014 12:00
Good to know that finally it worker its the cifs server signing problem..
HarshG
8 Posts
0
June 14th, 2014 12:00
Mark my comments as helpful. If I had helped u some way
ScotN
76 Posts
0
June 26th, 2014 07:00
This problem is 2 fold. Enabling signed SMB requires fwd and reverse DNS to resolve to a single interface.
So users who have multiple aliases and possible server issues where a deployment system only updates fwd records signed SMB will stop working. I also had a case where someone re-provisioned a server and it had 2 reverse names so SMB would intermittently fail.
Issue is still ongoing with Signed SMB performance. Looking at disabling it all together.
ScotN
76 Posts
0
July 7th, 2014 12:00
After weeks of troubleshooting CIFS performance issues I have completely disabled signed cifs support on the data domain and performance has returned to normal. Change was made based on documented performance issues comparing signed SMB1 vs SMB2.
Performance symptoms were evident from SQL backup jobs, and standard UNC copies via the guest OS.
https://www.google.com/#q=signed+smb+performance
Support tested iperf and lmdd tools which proved there were no network or SMB1 limitations but neither tool benchmarked signed SMB.