Contents

• Overview of Vulnerability Scanners
• Key Questions to ask
• Sources

Overview of Vulnerability Scanners

Vulnerability assessment scanners do pretty much what their name suggests -- they are systems that are automated to scan a network in search of security holes and to report on the holes that are found. Scanners to date are not primarily intelligent systems. They are programmed to search for known vulnerabilities. There is no reason to expect a scanner to do a better job ferreting out security holes than a well-trained vulnerability assessment team. Combine that with the fact that a recent test showed that none of 7 highly regarded scanners was able to detect all of a set of 17 very well known vulnerabilities and it is clear that a scanner will not form the bedrock of your security arsenal. Scanners are best used for periodic checks in the absence of an in-house security check and in between more thorough, human-powered security assessments.

A scanner can be evaluated in terms of the breadth of its scan (how many vulnerability checks it performs), the precision of its scan (how well it does in finding the most important vulnerabilities, and how few false positives it returns), the depth of its scan (does it search for secondary exploits?), the customizability of its scan (can it be updated/amended by hand?), and the results of its scan (does it produce a useful report? Is the report customizable?). Breadth is more likely to figure in vendor literature, but the same test referred to above found that having more checks did not mean that a scanner was more precise -- the scanners with the most checks also produced many false positives, and were not the best at detecting the most important well-known exploits.

The ability to customize is important, given the speed at which news of new vulnerabilities travels. Some scanners include periodical updates, some include automated updates, but for companies that can devote the time and expertise, customized updating is a good idea -- you never know just how uptodate or thorough the vendor updates will be. Open source scanners are obviously the most fully customizable -- so long as the expertise is available -- but some commercial scanners are now somewhat programmable.

Scanners are beginning to develop more sophisticated analyses, which can use information found on one server in an attack on another server, or take an attack deeper after finding a significant vulnerability. These technologies are not yet standard though, and the literature does not have much to say about the degree to which they are effective.

• What platform is the scanner hosted on?
• Does it scan firewalls for security-rule misconfiguration?
• Can old scans be retrieved to perform a trend analysis?
• does the report provide fix information?
• Does it report suspected vulnearabilities that it can't confirm, but which should be investigated by hand?
• Does it consistently recognize platforms and operating systems on targeted boxes, including NetWare, VMS?
• Can the scanning process be monitored as it occurs?
• Can reports be customized?
• How much network traffic does the scan create?
• Can an open licence be purchased?
• Do per-device charges include all machines, or are exceptions made for desktop machines not operating as servers, which will require very few checks?
• How long does a scan take? Are there different levels of scanning that can be chosen?