Exploring 10G Ethernet and beyond

While 10G Ethernet deployments are expected to start more widely once the standard is ratified next June, this higher-speed Ethernet may not be practical in some enterprise and carrier networks until supplementary technologies – such as TCP/IP off-load and higher-speed firewalls – are ready.

Those were sentiments expressed by experts at a 10G Ethernet workshop held recently at the San Diego Supercomputer Center, part of the University of California San Diego, for engineers from the vendor and standards communities. Issues on the table included what technologies are needed to support security and traffic shaping on 10Gbps networks, as well as what comes after 10G Ethernet – 40Gbps, 100Gbps or 160Gbps?

With the worldwide installed base of Gigabit Ethernet ports expected to increase more than fivefold by 2004, according to market researcher International Data Corp., experts say demand for even higher-speed aggregation technologies like 10G Ethernet will become inevitable. 10G Ethernet is projected to be mainly a carrier technology for metropolitan-area service providers, but enterprise network applications could include backbone aggregation of Gigabit links in a data centre or switch-to-switch connectivity across a large campus.

When these applications start rolling out next year, says Fred Worley, a Hewlett-Packard Co. engineer, initial deployments will not have everything necessary to make a 10G network run as efficiently as a 1Gbps network.

“It’s not that [10G Ethernet] is too fast,” Worley says. “It’s just that new technologies will need to evolve.”

TCP/IP off-load – implementing part of an IP stack as hardware instead of software in a device for faster performance – and new algorithms for 10Gbps packet inspection are in development. But these technologies won’t be far along enough when 10G products start shipping en masse next year, Worley says. These kinds of technologies will be important for ensuring that high-speed links are secure and that devices attached to these networks can keep up with such a high packet-forwarding rate.

An example Worley gave of how 10G Ethernet is challenging network hardware design was a 10G Ethernet network interface card (NIC) that HP is co-developing with Nortel. (A prototype of the product was demonstrated last month at NetWorld+Interop 2001 in Atlanta.) While the switch vendors may have figured out how to shunt packets along at 10Gbps, Worley says, his research is dealing with such challenges as how NICs on servers with next-generation, high-bandwidth bus technologies (such as Non-Uniform Memory Access and InfiniBand) can take in and process that much data from such a big pipe.

“With the speeds we’re talking about,” he says, “one second of buffering [on an NIC] requires 1.2GB of RAM. How many people want to have a full server’s worth of memory on their NICs?”

What’s After 10G?

Even with the 10G Ethernet standard still not put to bed completely, engineers at the workshop discussed what could be next. While speeds beyond 10Gbps probably won’t find their way into enterprise networks anytime soon, they are headed for carrier networks and could have a big impact on the kinds of services delivered.

“Fifty percent of the 802.3ae [10G Ethernet] task force thinks the speed should be 40 Gigabit,” says Johnathan Thatcher, chair of the IEEE 802.3ae (10G Ethernet) Working Group and CTO of Worldwide Packets. “But we’ve always gone up by factors of 10.”

IEEE engineers don’t like to reinvent the wheel, Thatcher says. Just as the physical layer (PHY) design of Gigabit Ethernet was based largely on Fibre Channel, and some 10G PHYs were tied to OC-192, 40G Ethernet could be a likely next step, as 40Gbps is close to the OC-768 SONET technology that already exists.

Others think differently.

“A while back, some people [in the IEEE] wanted to do a 400Mbps standard,” says Bob Grow, president of the 10 Gigabit Ethernet Alliance and an Intel engineer. “Look what happened to that effort.”

While many IEEE members would like to see 40G, Grow is more inclined to either 100Gbps or 160Gbps Ethernet.

“Forty Gigabits per second does not have the bandwidth increase that holds interest for me,” Grow says.

But for the sake of staying close to SONET technology, Grow says he could live with 160Gbps Ethernet, which would adhere to OC-3072 – neither of which will probably exist for several years.