In 2005, every major anti-virus vendor announced software vulnerabilities in various desktop and gateway products. These vulnerabilities include buffer overflows, which are particularly dangerous because they can allow a remote attacker to execute code on vulnerable machines. Buffer overflows have been at the heart of most major malware exploits.

Security software makes a logical target for several reasons. Attackers routinely hit ubiquitous software, and anti-virus is a must-have on corporate and consumer PCs and mail servers. It's also easy for attackers to reach. "Security software is usually the first inline that looks at incoming data," says Neel Mehta, team lead for ISS's X-Force security team. Finally, security software such as anti-virus runs with local system privileges, so attackers who compromise that software can inherit those privileges.

The security community is raising awareness of the danger. The SANS Top 20, an annual listing of the most critical security vulnerabilities in Windows and Unix, broke new ground in 2005 by adding cross-platform applications, including anti-virus, to the Top 20. The list cites security vulnerabilities in 14 anti-virus products, including those from Symantec, McAfee, and Trend Micro.

Anti-virus software isn't the only potential target. This past November a software vulnerability was discovered in Microsoft's anti-spyware beta that could potentially allow the execution of arbitrary code. Exploitable vulnerabilities were also uncovered in Snort, the popular open-source IDS software.

So what's a security architect to do? To start, quiz your vendors about the steps they take to build secure products out of the box. For instance, do they perform sufficient QA to ensure that problems such as buffer overflows are found and removed before the product ships? Next, ensure that all patches are up to date. Fixes do exist for the anti-virus flaws listed in the SANS Top 20.

Another option is to incorporate an additional layer of security software. Two classes of security products can provide another layer of defense. The first is software that detects and prevents generic buffer overflows. The second is behavioral-based host intrusion prevention software that looks for malicious activity on individual servers and desktops.

Of course, trying to solve software problems with more software can become a recursive nightmare. After all, there's nothing to stop bad guys from exploiting the security software protecting your security software.