Application Authentication via https using NTLM

We at Roadmap take security very seriously. Everything we do and everything we touch or make, we think about security as it is part of how we work. In fact: it is in our DNA.

Recently we have been working on a way to generate monthly reports for our customers in a sweet automatic way. To be able to download these reports the report server need to verify the given NTLM authentication credentials which are provided by the application. This was a bit of a challenge as it did not work in the way we would assume.

When you want to download a file, the .NET Framework provides you and easy and simple code snippet, which works straight away:


        public void DownloadAwesomeness()
        {
            var url = new Uri("https://DomainNameHere");
            var location = @"C:\Temp\Destination.txt";
            
            using (var client = new WebClient())
            {
                client.DownloadFile(url, location);

                Console.WriteLine("We are done!");
                Console.Read();
            }
        }

This works perfectly. How hard can it be to add some extra credentials to the WebClient? As the WebClient has a Credentials property, we will set these.


        public void DownloadAwesomeness()
        {
            var url = new Uri("https://DomainNameHere");
            var location = @"C:\Temp\Destination.txt";

            using (var client = new WebClient())
            {
                client.Credentials = new NetworkCredential("username", "password", "domain");
                client.DownloadFile(url, location);

                Console.WriteLine("We are done!");
                Console.Read();
            }
        }

When you will run this you will notice that the client.DownloadFile(url, location); will throw an ‘403 – Access Denied’. Before you start changing any usernames and/or password because you are doubting yourself if you did something wrong, let me help you out here: You didn’t do anything wrong. More interesting is; what is actually going on? What is happening with the credentials?

As you probably know, under the hood of the WebClient() the call of Url is actually done via a HttpWebRequest. I came to the idea to rewrite the WebClient() and client.DownloadFile(url, location) just for the understanding what is happening with the credentials. This resulted in:


        public void DownloadAwesomeness()
        {
            HttpWebRequest authenticationRequest = Request("https://DomainNameHere");
            HttpWebResponse authenticationResponse = (HttpWebResponse)authenticationRequest.GetResponse();
            authenticationResponse.Close();

            using (var s = downloadResponse.GetResponseStream())
            using (var rs = new StreamReader(s, Encoding.Default))
            using (var sw = new StreamWriter(@"C:\Temp\Destination.txt", false, Encoding.Default))
            {
                sw.Write(rs.ReadToEnd());
                sw.Flush();
                sw.Close();
            }

            Console.WriteLine("We are done!");
            Console.Read();
        }

        public HttpWebRequest Request(string url)
        {
            HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
            request.Method = "GET";
            request.UseDefaultCredentials = false;
            request.PreAuthenticate = true;
            request.Credentials = new NetworkCredential("username", "password", "domain");

            return request;
        }

While debugging the code I noticed the authenticationResponse was different from the one I would expect. And this is also the reason why the credentials didn’t work. The moment you call the Url for the first time, you won’t get access at all. In fact, you will get a challenge back which needs to be authenticated. As the HttpWebRequest by default contains an Autoredirect, it will automagically Autoredirect to the new Url. As this request is automagically done it will call the new Url without the earlier given credentials.

bad

The solution to this is basically disabling the AutoRedirect. After disabling we need to capture the challenge Url from the response and call it with the credentials attached to it. You will end up with this big and horrible but working piece of code:


        public void DownloadAwesomeness()
        {
            HttpWebRequest authenticationRequest = Request("https://DomainNameHere");
            HttpWebResponse authenticationResponse = authenticationResponse.Close();

            HttpWebRequest downloadRequest = Request(authenticationResponse.Headers.Get("Location"));
            HttpWebResponse downloadResponse = (HttpWebResponse)downloadRequest.GetResponse();

            using (var s = downloadResponse.GetResponseStream())
            using (var rs = new StreamReader(s, Encoding.Default))
            using (var sw = new StreamWriter(@"C:\Temp\Destination.txt", false, Encoding.Default))
            {
                sw.Write(rs.ReadToEnd());
                sw.Flush();
                sw.Close();
            }

            Console.WriteLine("We are done!");
            Console.Read();
        }

        public static HttpWebRequest Request(string url)
        {
            HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
            request.Method = "GET";
            request.UseDefaultCredentials = false;
            request.PreAuthenticate = true;
            request.Credentials = new NetworkCredential("username", "password", "domain");
            request.AllowAutoRedirect = false;

            return request;
        }


Even though I finally had a working piece of code, I still wasn’t satisfied by this big junk of code. A double call doesn’t make any sense. And with the WebClient() the code looks a lot more cleaner, so there has to be something which also handles the NTLM credentials in a proper and cleaner way.

Remember the client.Credentials = new NetworkCredential(“username”, “password”, “domain”);? Further investigation shows me you can set the Credentials property of the WebClient() in different ways. The one I needed was the CredentialCache. First though; It is just a cache of credentials, what is the need of it for a single call.

Answer; it handles the AutoRedirect FOR you. Applications that need to access multiple resources can store the credentials for those resources in a CredentialCache instance that then provides the proper set of credentials to the Internet resource when required. When the GetCredential method is called, it compares the Uniform Resource Identifier (URI) and authentication type provided with those stored in the cache and returns the first set of credentials that match.

result

This made a lot more clear and easy to fix. The end result:


        public void DownloadAwesomeness()
        {
            var url = new Uri("https://DomainNameHere");
            var location = @"C:\Temp\destination.txt";

            // When calling for the url a NTLM challenge takes place
            // Once this challenge takes place the GetCredentials will automagically be 
            // called via de CredentialCache
            // This will resolve the 2 step authentication
            // Requirement: the uri for the cache must be the Scheme + Host of the domain
            var cc = new CredentialCache();
            cc.Add(new Uri(string.Format("{0}://{1}", url.Scheme, url.Host)), 
                                         "NTLM", new NetworkCredential("username", "password", "domain"));

            using (var client = new WebClient())
            {
                client.Credentials = cc;
                client.DownloadFile(url, location);

                Console.WriteLine("We are done!");
                Console.Read();
            }
        }

This makes me a lot happier as the code is a lot more readable and makes more sense. Although it took me a lot more time just to figure out what is actually happening under the hood of the WebClient() (and eventually I don’t even need the code of the HttpWebRequest) it gave me a lot more insight in how security works. In fact, it makes me even more curious how far I can actually go to get security at even a higher level 🙂

One thought on “Application Authentication via https using NTLM

Leave a Reply

Your email address will not be published. Required fields are marked *