January 2013 IT Business Consulting Newsletter

An Anti-Virus Gotcha - It could happen to You!

By Tom K

My client had excellent Corporate Anti-virus installed and auto-updating on every device in every site. We were centrally monitoring and updating Microsoft Updates across the enterprise. And still, the business network got slammed to its knees with a vicious virus attack.

We were completely protected... but we still got slammed! And it could happen to You!

In this month’s newsletter I discuss how we got blindsided, and what you need to be aware of to prevent this from happening to your Company.

What We Saw

We started seeing hundreds of alerts being sent from the Corporate Anti Virus (AV) software advising that essentially all of the company PCs and servers were trying to launch applications suspected of containing viruses. The AV software was preventing the launches on the PCs and servers, but the attempts were relentless. We forced an immediate full virus scan on all devices in the enterprise, and all reported that virus files were found and deleted. The alerts continued (hundreds/hr), and additional scans found and deleted the same files found and deleted on the previous scans. They were being copied over as quickly as the AV software could delete them. Network bandwidth was being gobbled up by all the file transfer activity and EVERYTHING was slowing down.

Then, user logons started to get blocked. The infection was running a routine to bust into the servers by running password hacks on every user ID. After 20 failed attempts, the user’s account was locked out (per our security rules), so the real users couldn’t sign on to the network.

Result: The network was running like molasses and most users couldn’t even log in.

Fortunately, our AV software had properly identified the virus that was attempting to infect everything on the network. This provided us with the info necessary to mount a defense to regain control of the network, and an offense to find any infected devices and kill the virus.

What We Did

We created some rules to lock down the network and used Group Policy to push the rules down to all the PCs and servers in the enterprise (see my May 2011 newsletter “Use Group Policy to Centrally Tune YOUR Business Computing Environment”). This stopped the password hacking routine from running so we could unlock all the user accounts and get everyone logged back in. It also slowed the attempts to push infected files to the uninfected devices, so the network became reasonably responsive. Everyone could work!

We checked again to insure all our devices were protected by AV with current definitions and current Microsoft security patches, and they all were. While they were seriously affected by the attack onslaught, none of our devices had been infected. But something on the network was definitely infected and was still trying to infect anything it could access.

We then ran a special application that scans every device in the enterprise looking for the specific virus we had identified. It found one infected device, identified by IP address and name. Unfortunately, we had no record of the device, and had no idea what it was or where it was connected, other than it was plugged into the cabling somewhere in the main office.

What We Found

We assumed it was a laptop that was brought in and connected without permission and without our knowledge. The staff was asked and we performed a visual inspection, but no unauthorized laptop was found. We ran some more tools to help us determine the nature of the infected device, and after a few hours we finally figured out what it was and located it!

The phone system vendor had plugged their Voice Mail (VM) server into our Private Business Network without advising or asking. The VM server didn’t need to be on our network to function, they just plugged it in for their convenience so they could get to it via the Internet for remote management. It hadn’t received a Microsoft security update since it was installed! It had no AV software!! And it was physically connected to every device in the client’s enterprise!!!

The Fix

At the time, we didn’t know if there was a functional reason for the VM server to be connected to the Private network. Since voice mail is a pretty important business tool, we couldn’t just pull the box off the network. So, we immediately contacted the phone vendor and had them give us credentials to access their VM server. We installed our AV software, which located and killed the virus. We ran a few other AV tools on it just to be sure, but no further traces were found. The attacks ended. We pushed an AV scan to all our devices and the virus files were deleted and didn’t reappear. Whew!

The phone vendor refused to allow us to set up automatic Microsoft security updates on their VM server, so we had them pull it from our Private network. We suggested that they put a Wi-Fi card in it and access it through the Internet via the Public (and completely isolated) wireless network we provide for guests, and they did so. As the VM server is no longer on our Private Business Network, its security issues are no longer a critical concern for us while the vendor determines how to deal with managing security updates on their device.

The Moral of the Experience

No matter how well you protect your devices, if an unprotected device gets access to your network it can cripple you. Don’t allow ANYONE to connect ANYTHING to your Private Business Network unless you are sure it complies with your security requirements.

Areas of concern include vendor equipment as noted above as well as vendor techs with laptops, and your staff with personal laptops, tablets, and smart phones.

I’d suggest you include a line in your IT Use and Abuse Policy that clearly states it is forbidden to connect any device to the Private Business Network without written approval (see my January 2012 newsletter “Employee IT Use & Abuse Policy”).

For guests, vendors, and staff with personal laptops, tablets, and smart phones, we typically provide unsecured Internet access through a Public Wireless Network (see my July 2009 newsletter “Securely Implement a Public Wireless Hot Spot”). This network is completely isolated from the Private Business Network so the security status of these devices can’t affect our devices on the business network. The Public Wireless Network's availability is provided “as is” with the understanding that it is public, open, and unsecured.

If you do provide staff with access to your business systems through a Private Wireless Network, make sure it is very well secured.

I often see instances where sub-contractors (Sales Agents) have to provide their own PCs, which are allowed to connect to the Private Business Network and access company resources. You need to ensure these PCs comply with your security requirements before they attach to your business network. My clients require that these privately owned PCs use the corporate Anti Virus solution and be managed and monitored with the same security utilities used on the company owned PCs if they are to access the company’s business network.

Endpoint Security Policy

You should consider creating an Endpoint Security Policy. It can start out very simple and be expanded as needs arise. It is a formal, published statement of your company’s security requirements for any endpoint device (PC, Laptop, Tablet) connected to your Private Business Network. This policy can also contain a statement forbidding unauthorized connections to the business network, and the process to get authorization.

Besides defining a critical set of requirements, once documented, it becomes much easier to educate your staff how to comply with the rules, and to enforce those rules... especially when the Endpoint Security Policy is part of a properly developed IT Use and Abuse Policy.

Additional Protections

There are a couple of things that can be set up in the environment to prevent unauthorized access / connections to your Private Business Network, but they are somewhat pricey &/or complex.

* Endpoint Security Enforcement Devices

There are systems available that evaluate a device whenever the device is plugged into the network and turned on. It compares the device to a set of security rules and can take action such as preventing the device from connecting, sending an alert to an administrator, or forcing security updates to the devices if the device doesn’t comply. These systems are quite expensive and require a lot of care and feeding.

* Switch Port Lockdown

This entails having a managed Ethernet switch for each site, and configuring every port to allow only a specific device to connect through that port. This is rather time consuming and must be updated every time a device is added, moved, or replaced.

If you have any concerns with the security of your environment, the effectiveness of your protections, or your security requirements and policies, or you have any questions or comments concerning this article, I’d be happy to discuss this with you at your convenience. Feel free to contact me at TomK@TomKConsulting.com, or via my cell 443.310.5110.

Next month I’ll discuss how to provide secure wireless access to your Private Business Network if staff need to access company resources using wireless devices. See "Provide Wireless Access to Business Systems???"