My colleagues and I recently encountered an intermittent problem with the CSCP reporting “Authorization Failed” when you tried to sign in.

This was a Skype for Business 2015 Enterprise Edition pool, and so it didn’t take long to localise the issue to just one Front-End server.

Each failed attempt dropped an instance of event 35009 into the “Lync Server” log:

Remote PowerShell cannot create InitialSessionState for user S-1-5-20. Cause of failure: Cannot find user in Active Directory with the following SAM account name: "S-1-5-20"

That SID belongs to the NT Authority / Network Service – and when we succeed on another machine, we see the traces* correctly trying to sign in as the SID of my admin account.

(* The reference to PowerShell in the log gave us cause to review the Centralised Logging for the “PowershellWebService” scenario, but it didn’t seem to add a great deal of value.)

A successful sign-in also drops the more friendly 35008 into the log:

We collectively tried a range of things – some of them probably several times – without success.

We figured it’s a web-related issue, so we tweaked, tinkered and repaired WebServices and IIS, repaired the address re-writes, and later re-applied the latest cumulative update and bootstrapped it senseless to no avail.

AD Group memberships were checked and found all good.

The one revelation that we thought was an early fix was that the Kerberos password was broken on that machine:

Test-CsKerberosAccountAssignment -identity <YourTopoSite> -verbose
*Broken*
Set-CsKerberosAccountPassword -FromComputer <WorkingFeFqdn> -ToComputer <ThisFeFqdn>
Test-CsKerberosAccountAssignment -identity <YourTopoSite> -verbose
*Kerb is happy now*

Nup.

This was a new FE, not yet in production and the fix was dragging on, so we were seriously considering a blow-away rebuild when one last trick did the job: a total uninstall and reinstall of IIS:

• Stop-CsWindowsService
• Make sure the services don’t bounce back after a reboot:

  Get-CsWindowsService | foreach {set-Service $_.Name -StartupType Disabled }

• Uninstall SfB WebComponents.msi. (I did this from the Control Panel / Programs and Features)
• Uninstall-WindowsFeature -Name Web-Server
• shutdown -r -t 00
• I reinstalled IIS using the code blob stolen from Set-CsFeatures (thank you Pat):

  Add-WindowsFeature -Source <WinSource> -Name NET-Framework-Core,RSAT-ADDS,Windows-Identity-Foundation,Web-Server,Web-Static-Content,Web-Default-Doc,Web-Http-Errors,Web-Dir-Browsing,Web-Asp-Net,Web-Net-Ext,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Http-Logging,Web-Log-Libraries,Web-Request-Monitor,Web-Http-Tracing,Web-Basic-Auth,Web-Windows-Auth,Web-Client-Auth,Web-Filtering,Web-Stat-Compression,Web-Dyn-Compression,NET-WCF-HTTP-Activation45,Web-Asp-Net45,Web-Mgmt-Tools,Web-Scripting-Tools,Web-Mgmt-Compat,Server-Media-Foundation,BITS,Desktop-Experience

  (NB: This is the string to install SfBS 2015 on Win 2012R2.)

• shutdown -r -t 00 (A reboot wasn’t prompted, but I like doing that after significant steps regardless)
• Re-enable the SfB services:
  • Get-CsWindowsService | foreach {invoke-expression “sc.exe config $($_.Name) start= delayed-auto”}
  • sc.exe config w3svc start= auto
• Re-run Bootstrapper.exe:

  PS C:\Program Files\Skype for Business Server 2015\Deployment> .\Bootstrapper.exe

• Reapply the latest CU
• Start-CsWindowsService
• TADA!

Yes, it’s a bit ugly, but it did the job and spared us even more pain. We’ve had no problems with it since.

Revision History

29th August: this is the initial publication.
 

– G.