401 Unauthorized access error when accessing web service (Which uses SharePoint-80 application pool and domain user identity) from client machine

PROBLEM Description:

1. I have deployed web service, on the production server, web service that utilizes the MOSS objects / namespaces. In order that the service should be able to access the information i have deployed the service in the same application pool as SharePoint - 80 and provided the website folder with same permissions as the SharePoint site. The end result is that the information is accessible through the service on the local machine.

When the same methods are invoked from any other machine on the network we get the unauthorized 401 error.

If I change the application pool to the default application pool then the methods are accessible but not the information from MOSS as it is not within the same application pool.

The differences between the two application pools are the identities that are used for the services. Default utilizes the network service account while MOSS uses a custom service account.

We need to identify the error for the first scenario and be able to deploy without error the web service in the SharePoint application pool.

Environment:

Product: Visual Studio 2008, MOSS 2007

OS: Windows Server 2003

RESOLUTION:

I am providing you here the troubleshooting steps which I performed to resolve the issue.

[INFORMATIVE AND IMPORTANT]

• We are having a web service which is consuming services from SharePoint also and running with application pool of SharePoint and identity of the domain user.
• Everything works fine on the server.
• But from client machine when we are trying to access the service, it is throwing 401 unauthorized errors.
• We have set the identity of domain user (say ‘devportal’ user).
• Now to get the actual issue, we enabled the auditing on the application server.
• We enabled “Audit account logon events” and “Audit logon events” for failure.
• Then ran the application again from client machine.
• After that we saw failure audit in security event log and with event ID of 529,533.
• This gave me an idea that this is a failure due to Kerberos.
• So I started troubleshooting from that perspective.
• I created a sample website (just an html page) and ran it same application pool and identity.
• Here also I am able to reproduce the issue.
• Then I made two changes on Active Directory machine:

a. For the server, set the “Trust this computer for delegation to any service (Kerberos only).”

b. For the user, set the “All computers”

• I also set the NTLM authentication on IIS with the following command:

cscript adsutil.vbs set w3svc/NTAuthenticationProviders "NTLM"

• After that everything worked fine.
• But it was popping up the dialog box for authentication.
• I ran the command:

cscript adsutil.vbs set w3svc/NTAuthenticationProviders "Negotiate,NTLM"

Also I configured the SPN with the following commands:

Setspn –A HTTP/NETBIOS_NAME_OF_IIS_SERVER domain\username

Setspn –A HTTP/FQDN_OF_IIS_SERVER domain\username

Now when I browse the web service from client machine then it works fine, problem is resolved -J

But problem still persist on the application server which deployed an application in IIS and that application is consuming this web service

Problem description:

Till now when I browse the web service from server machine or client machine then it works fine

1. I have an application [say DocSytems] deployed on another server [say APPSERVER]

2. DocSytems consumes the web service which is deployed on share point server

3. Now if I access the web service from DocSytems from APPSERVER server machine then it works fine

4. If I call the web service from DocSytems from other machine in the network and login with a domain user name then it doesn’t work

5. But interestingly if I impersonate the domain user (devportal) name which is set in the Identity of the SharePoint-80 app pool and DocSytems application then it works fine

Solution:

• I found that application server’s IIS was not set to Kerberos, i.e. with “Negotiate, NTLM”

• I set the IIS for “Negotiate, NTLM”

To set:

cscript adsutil.vbs set w3svc/WebSite/root/NTAuthenticationProviders "Negotiate,NTLM" ( In yellow color, you need the change it with site Id, in our case it was ‘1’).

To get:

cscript adsutil.vbs get w3svc/WebSite/root/NTAuthenticationProviders

• Then I also found that IE browser (of client) was not able to recognize the windows integrated authentication setting.

• In Internet explorer, I enabled the settings from Toolsà Internet Options à Advanced à Security à “Enabled Integrated Windows Authentication”

• Once I enabled the setting on client machine, everything works as expected.

So the problem is resolved

References on the issue which we resolved:

http://technet.microsoft.com/en-us/library/cc739740(WS.10).aspx

http://support.microsoft.com/kb/832769

http://support.microsoft.com/kb/215383

http://support.microsoft.com/kb/929650

http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/d3df4bc9-0954-459a-b5e6-7a8bc462960c.mspx?mfr=true

http://support.microsoft.com/kb/909887

http://technet.microsoft.com/hi-in/library/bb742516(en-us).aspx

Additional references:

For your future reference, I am mentioning here few articles related to IIS. Please note that these articles may or may not be related to the current case, but good pointers for keeping them in record book and future study.

1. This article talks about "How to configure applications in IIS 6.0" - http://support.microsoft.com/default.aspx?id=814867

2. This article talks about "How to configure security by using IIS 6.0" - http://support.microsoft.com/default.aspx?id=814874

3. This article talks about "How to performance-tune your applications in IIS 6.0" - http://support.microsoft.com/default.aspx?id=814876

4. This link talks about troubleshooting IIS - http://technet2.microsoft.com/windowsserver/en/technologies/featured/iis/default.mspx

5. IIS Authentication - http://msdn.microsoft.com/en-us/library/aa292114.aspx