Management platforms put to test

While there are a lot of benefits to using network-management platforms, no product is perfect.

Just ask Vanderbilt University, which slashed its network downtime by installing Cabletron Spectrum Enterprise Manager. Instead of getting phone calls from angry users half an hour after a problem starts, network managers now get pages from the software within three or four minutes, says John Brassil, network engineer at Vanderbilt in Nashville.

“It’s really been a lifesaver in terms of 24-7 notification of our engineering staff,” he says.

The downside is that Spectrum doesn’t have the richest support for non-Cabletron hardware — and getting the software up and running was no easy task, Brassil says. “Spectrum has a stack of manuals about two feet high. It’s pretty intimidating,” he says.

Such mixed evaluations are typical for management software these days. The 1999 Network World (US) Network Management Survey results show that you’re generally content with your arsenal of network management tools, although no product really stands tall above all others.

While there are no overwhelming victories, survey results still identify leaders and laggards. Cabletron Spectrum Enterprise Manager is tops for enterprise management, while Computer Associates Unicenter TNG leaves the most room for improvement.

On the LAN management front, the Hewlett-Packard OpenView family garnered the highest satisfaction ratings, while Seagate Desktop Management Suite came in at the back of the pack.

Those are just a few highlights from the survey we conducted to see how you grade the leading network management platforms. The ratings focus on four key aspects of a management platform – scalability, integration, management capabilities and overall value. Moreover, we asked respondents for a bottom-line indication of their overall satisfaction with the network management tools they use.

Our rating scale ranges from a low of 60 (not at all satisfied) to a high of 100 (extremely satisfied). A score of 90 or greater counts as an A; 80 to 89 a B; 70 to 79 a C; and 60 to 69 a D. Products earning overall satisfaction scores of 85 or above merit a place in our Honour Roll.

We divided the results into two general types of management tools — enterprise management and LAN management. While the areas occasionally overlap, one common distinction is scale. The bulk of the companies with small networks represented in the survey rely solely on LAN management platforms, while the firms with the most networked clients are more likely to use enterprise management tools or both types of products.

Enterprise Achievers

Cabletron Spectrum Enterprise Manager is clearly at the head of the class of management products. The product has top marks for all key satisfaction categories. Customers are most impressed with Spectrum’s scalability, giving it an A grade of 90.

HP OpenView came in second for overall value, though the product matches Spectrum point for point in integration capabilities.

While BMC Software Patrol and Tivoli platforms tied for overall satisfaction and management capabilities, Patrol bested Tivoli in scalability, integration and overall value. (The Tivoli rating includes Tivoli Management Framework and NetView for OS/390 because too few survey respondents use either product to characterize them separately.)

CA Unicenter TNG lagged behind its competitors in every category except for scalability. The product’s lowest grade was an 81 for overall value.

Indeed, many users are stunned by the cost of the largest enterprise platforms, says Ron Gallagher, president of Oust Consulting, a firm in Paradise Valley, Arizona, that helps companies outsource their IT services. “The price tends to put them off,” he says. Because the products can take a lot of effort to install and use, “even if companies have platforms, they’re not effectively using them,” Gallagher says.

Perhaps that’s one reason why Unicenter and Tivoli platforms scored relatively low for integration with other management tools, which is somewhat surprising given that both vendors market their products as management frameworks.

LAN Leaders

A look at the LAN management ratings shows that other vendors can learn a thing or two about customer satisfaction from HP. HP OpenView ManageX was the only platform in the survey to earn an A for overall satisfaction, and its sibling HP OpenView also received high marks.

Network professionals viewed OpenView ManageX and OpenView’s scalability and integration with other management tools as especially superior, which might be one reason why some of you use the platform to manage your entire enterprise.

On the other hand, the scalability features of Intel LANDesk and Network Associates’ ZAC Suite left the most room for improvement. However, ZAC Suite scored a relatively high 87 for management capabilities.

Microsoft Systems Management Suite (SMS) and Novell ManageWise users rated both products the same in terms of overall satisfaction. SMS had the edge when it came to scalability, while ManageWise boasted higher scores for integration and overall value. In fact, SMS offers among the poorest integration of the products featured in the survey, along with Seagate Desktop Management Suite.

As you might expect, our analysis of the survey results shows that scalability, integration, management capabilities and overall value are closely tied to overall satisfaction.

But if vendors were to concentrate on improving just one aspect of their products, it would be wise for them to focus on overall value. This factor is the single biggest determinant of satisfaction and has a predictive strength of 72 per cent. Put another way, if you know a product’s rating for overall value, 72 per cent of the time you’ll be able to predict the product’s overall satisfaction rating.

Platform scalability and management capabilities are strong predictors of overall satisfaction, too, and have individual predictive strengths of near 60 per cent. Finally, a management tool’s integration capabilities are still important, but this rating has the least impact on overall satisfaction.

Zeroing In on the Details

The core ratings reflect the opinions of all respondents who use a particular product. However, the people who use a tool as their primary management platform can generally provide a more thorough evaluation of its capabilities, so we asked them to judge more than a dozen additional product attributes.

Already the leader among general users, Spectrum also takes the prize for keeping its primary users happy. The product matches or surpasses the competition in nearly all categories.

Spectrum particularly distinguishes itself with a 92 for performance management and also earns an A for scalability, management capabilities and configuration management. Likewise, the platform excels in problem management, network optimization capabilities, systems optimization, automated diagnosis and alert correlation capabilities.

One key ability for network managers is alert correlation. Spectrum earns high marks in part because it can look at a list of network events and errors, and pick out which ones caused all the rest.

“With its fault-isolation engine, Spectrum is able to identify what problems there are, as opposed to just turning all the devices that are affected red” on the screen, says Dan Speers, senior network engineer at investment bank Jefferies & Co. in New York.

Spectrum also can weed out small problems. “Sometimes routers send a ‘link down’ trap, and by the time you get to the router, the link is back up,” Speers says. Spectrum waits a while before paging a network manager to make sure the link is indeed down for the count and requires a manager’s intervention.

Tivoli platforms do well, too, tying with Spectrum for an 89 in overall satisfaction. Tivoli offers seemingly better overall value, asset/inventory management, backup management and database management.

Primary users of CA Unicenter TNG aren’t quite as pleased. In fact, the product ranks below its competitors in every factor evaluated. Users identified its biggest weaknesses as asset/inventory management and directory integration.

BMC Software Patrol isn’t included in the detailed ratings because too few survey respondents use the product. And when it comes to LAN management, almost 75 per cent of you use HP OpenView, Microsoft SMS or Novell ManageWise as your primary LAN management platform. Because less than five per cent of respondents primarily use any other tool, we can only provide in-depth analysis of the leading platforms.

OpenView beats SMS in most categories, and its users are especially pleased with the platform’s integration features, problem management and automated diagnosis. Conversely, OpenView customers would likely welcome improvement in the areas of directory integration and software distribution.

SMS” strengths are asset/inventory management and software distribution, but the product is tarnished with the lowest score of the survey, 78, for automated corrective actions. Moreover, SMS could stand to improve its automated diagnosis capabilities.

ManageWise pulls away from the pack in terms of server management and directory integration. The Novell platform’s most apparent weakness is Web server management.

Repeat Customers

Would you buy your network management platform again if given the chance? Roughly 75 per cent of respondents say they would if they had it to do over. This decision is unrelated to how much the product cost, how long it took to install or when it was purchased. Naturally, the answer is consistent with overall customer satisfaction.

Four areas of satisfaction most drive the inclination to purchase management platforms again: problem management, systems optimization, asset inventory management and configuration management. Survey respondents who say they wouldn’t purchase the same platform again are significantly less satisfied with these functions of their products.

Not surprisingly, Spectrum is the enterprise management platform that most users would buy again. Tivoli is in second place but doesn’t even come close to touching Spectrum’s edge with customers.

The LAN management race is much closer. The largest proportion of HP OpenView users have no regrets about their purchasing decisions, followed by SMS and ManageWise.

Then again, some respondents didn’t have to buy network management tools in the first place. Freeware tools aren’t included in the survey, but one user says his organization depends on two free tools, Multi Router Traffic Grapher (MRTG) and Big Brother. The organization, which has points of presence scattered around the county, found platforms such as NetView and OpenView too cumbersome.

MRTG helps spot bottlenecks and capacity problems, while Big Brother watches processes and systems to make sure they’re up and running, says Terry Moenich, general manager of the Wood County Internet Council in Ohio.

“Those are our two main tools that we live and die by, and the odd thing is that they’re free,” Moenich says.

( Ellerin is president of STAT Resources, a strategic research and consulting firm in Boston. Jeff Caruso, a Network World senior editor, and Lynne Castronuovo, research manager of STAT, also contributed to this story.)