Testing Microsoft Security Essentials and the Hosts file

Computers on the Internet address each other with numbers. What appears as computerworld.com to a human being is 65.221.110.98 to a computer. The system that translates between names and the underlying numbers (really IP addresses) is called DNS and it works very well. Too well, for some bad guys.

Many years ago, before the Internet, the translation between computer names and numbers was done by a file on each computer called the "hosts" file. Needless to say, as the number of computers got large, maintaining a hosts file on every computer became unrealistic. Now, when a computer is called on to reference another computer by name, it first makes a call into the DNS system to retrieve the underlying IP address.

Why the history lesson?

Microsoft never retired the hosts file* and bad guys abuse it.

For example, screwing up the mapping of names to numbers can prevent antivirus software from self-updating. Another tactic is to change the entry for bank websites. A computer with a maliciously modified hosts file can send someone to a duplicate copy of a bank web site, one that looks totally legit, but is designed to steal userids and passwords.

By default, the hosts file is used before DNS, a poor design decision by Microsoft. Protecting the hosts file from modification is thus a standard practice for antispyware software.

What brings this up, is a recent comment by security expert Steve Gibson on his Security Now podcast. Gibson is the rare techie that actually uses the hosts file for its original intent. One day when he couldn't reference some computers by name, he tracked down the problem to Microsoft's new Security Essentials (MSE).

It turned out that when he installed Security Essentials, it replaced his hosts file. MSE gave him a new empty file after making a backup of the original. This may be a good decision, but it wasn't externalized, Gibson had to figure it out on his own.

MSE doesn't always replace the hosts file. On my test Windows XP computer, an unmodified hosts file was left unchanged by the installation of Security Essentials.

This made me wonder what happens if the hosts file gets modified while MSE is running. It's easy to test and so I did, as described below.

The tests were run under Windows XP SP3 on October 23, 2009 using MSE version 1.0.1611.0. The Virus definitions were created Oct 23, 2009 at 2:49AM. Both the virus and spyware definitions were version 1.69.64.0. Real time protection was enabled during all tests.
 
TESTING

The first thing I noticed was that a restricted user can't modify the hosts file. This protection is provided by Windows, not by Security Essentials. Good thing too, as this is just the sort of thing restricted users shouldn't do.
         
An administrator can modify the hosts file and, surprisingly, Security Essentials didn't prevent the modification and didn't object afterward. A website of mine that I haven't updated in years, computergripes.com, has its own IP address. So I modified the hosts file to send another of my websites, javatester.org to the gripes site. Technically, the change was

216.92.35.29   javatester.org  #really computergripes.com

After saving the hosts file, I verified that javatester.org was indeed, re-directed to computergripes.com. Security Essentials was running the whole time and didn't object.

To try and tell MSE that I was doing something bad, I had it scan the WINDOWS\system32\drivers\etc folder. No complaints. Then I scanned just the hosts file and again, Security Essentials assured me that all was well.

Thinking that MSE just doesn't care about my dinky websites, I more directly mimicked what the bad guys do.

An IP address of 127.0.0.1 always refers to your computer. If you want to insure that a computer can't access a website, you assign the website name to 127.0.0.1 in the hosts file. Some people do this to avoid ads on web pages. Bad guys do this so that antivirus and antispyware software can't download new definitions. I added the following lines to my hosts file:

127.0.0.1   microsoft.com
127.0.0.1   symantec.com   
127.0.0.1   antivirus.com   #really trend micro

Symantec was blocked  and so too was Trend Micro. I tested with both Internet Explorer version 7 and Firefox version 3.5.3.

But, then I almost fell off my chair. I was able to get to microsoft.com. How can this be?

It got redirected to a page at www.microsoft.com and the hosts file is very exact. That is, blocking microsoft.com does not block www.microsoft.com. So, I added an entry for www.microsoft.com, saved the hosts file, restarted the browsers, cleared their cache and was still able to get to Microsoft's website.

The actual web page that was being displayed was www.microsoft.com/en/us/default.aspx. Taking no chances on a typo, I copied this into the hosts file from the browser address bar. No matter, I could still get to the page.

Microsoft seems to have put special rules in Windows for its own website.

While testing access to Microsoft's website, I finally got warned by Security Essentials about the modifications I had made to the hosts file. The warning, however, came quite a while after the first change. I had updated the hosts file a few times before MSE picked up on it.

As had been reported elsewhere, Security Essentials did not do a good job of cleaning up the hosts file. Specifically, it removed the entries for microsoft.com and symantec.com but left entries for antivirus.com and avira.com, both of which where were pointed to 127.0.0.1. It also left the entry for javatester.org.

I wondered if bad guys could prevent antivirus software from downloading new definitions by assigning vendor websites to other sites rather than nulling them out (the net effect of assigning them to 127.0.0.1). 

To test this, I added still more antivirus related websites to the hosts file and waited for MSE to object again. It did, and after cleaning the hosts file, left these entries behind
 
216.92.35.29   microsoft.com
216.92.35.29   www.microsoft.com
216.92.35.29   symantec.com
216.92.35.29   www.symantec.com

Sure enough, preventing access to Symantec's site (by assigning it to 127.0.0.1) is considered bad, but assigning it to computergripes.com is just fine.

Then I made additional changes to the hosts file and again let Security Essentials clean it up. As with Symantec, it left behind entries for avg.com and avira.com that pointed to my gripes site.

Until this changes, bad guys can redirect antivirus websites to any other website on the Internet to prevent them from self-updating.

RESTRICTED USER

The cleanups happened while logged on as an administrator. Next, I wondered if Security Essentials could clean up the hosts file while logged on as a restricted user. So, I zapped it again and rebooted.

Since there was quite a delay between the first modification of the hosts file and the first warning, I guessed that Security Essentials only checked it periodically. After logging on as a restricted user, I waited. And waited. And waited. A couple hours went by and no warning about the hosts file modifications.

As before, the heavily modified hosts file got a clean bill of health when scanned directly. Specifically, these are the modifications that were not considered a problem:  
 
216.92.35.29  javatester.org  #really computer gripes
127.0.0.1  www.microsoft.com/en/us/default.aspx
127.0.0.1   antivirus.com
216.92.35.29   avg.com
216.92.35.29   www.avg.com
216.92.35.29   avira.com
216.92.35.29   www.avira.com
127.0.0.1  www.microsoft.com
127.0.0.1  microsoft.com
127.0.0.1  ftp.microsoft.com
127.0.0.1  symantec.com
127.0.0.1  www.symantec.com
127.0.0.1  security.symantec.com  

Again, I deleted the browser cache and verified that www.symantec.com was unreachable. Still, Security Essentials was all green. I waited some more. Still green. I even opened the file in Notepad, even though I couldn't save it. Still no objection from MSE.
 
Could it be that Security Essentials works differently when logged on as an Administrator vs. a restricted user? The same entries that it removed when running as an Administrator, it ignores when running as a restricted user.

To test this, I logged back on as an Administrator and scanned the hosts file again. All green. I used the system for a while doing other work and never got warned about the modified hosts file.

If there's a pattern here, I can't find it.

You may want to opt for a more mature product.

Update: October 24, 2009. To try and get MSE to detect the modified hosts file while logged on as restricted user, I tried a quick scan. It found nothing.