Coast guard, DFO get their houses in order

The Canadian Coast Guard is used to rescuing ships in distress. What it’s not accustomed to is requiring rescue itself. But that was exactly the situation the coast guard’s Pacific regional office faced as it foundered in a sea of slow data and battled a serpentine mess of old cabling.

Fortunately, with the help of a systems integrator and a newly invented network traffic analysis tool, the coast guard managed to get its network in order. The Pacific branch is just now completing a roll-out of switched fast Ethernet to every desktop and has plans in place that should allow its network to sail along smoothly for years to come.

The Pacific office’s networking dilemma began in 1996 when the Canadian government decided to shift the Canadian Coast Guard from Transport Canada to the Department of Fisheries and Oceans (DFO). As a result, the Pacific region coast guard office in Vancouver was to be moved and merged with the Pacific region fisheries office, also in Vancouver.

At its old home, the coast guard had been running a Windows NT network for 150 staff over a mix of Ethernet and fast Ethernet links, said Henk Razenberg, the coast guard’s regional supervisor of network services.

“It was fairly good,” he said. “Users were pretty happy with it.”

But the network at the coast guard’s new home in the DFO office was a different story. The network, which had grown gradually with the office, was supporting about 300 users on shared Ethernet over a mix of coaxial cable, odd bits of fibre and twisted pair.

“After we had a look in the ceiling and saw the state of the existing network, we decided we wanted every bit of cable ripped out,” Razenberg said.

Another inconvenience was the network was using routers in places where it could have been using switches.

“It was very cumbersome,” Razenberg said. “For a workstation to talk to a server, it would get routed by one or two routers and that was within the office.”

So the coast guard and DFO wrote up a list of requirements for a new network design. What they wanted was to recable the two floors of the fisheries office and install a small amount of new equipment to get a network that could run existing applications at 10Mbps with room to grow to 100Mbps throughout the office.

After responses to the request for tender rolled in, the coast guard and DFO ended up selecting IBM, based on price and network design.

Fred Klassen, a certified IT specialist, networking, with IBM Global Services, who submitted the successful design, didn’t think the recabling and minor implementation project would work as well as the coast guard and DFO were hoping. He convinced the departments to let him do a network analysis.

“I found their current infrastructure to be totally inadequate,” he said. “It had some real problems.”

Klassen used a traffic analysis approach he had recently co-invented to find trouble spots in the fisheries building’s network. Called Apparent Network Speed Analysis (ANSA), the approach tells users what their network can’t do. Most other tools, Klassen noted, only tell network managers what their network is doing at a given moment.

One problem area the tool identified was on a 10Mbps metropolitan-area service the fisheries office leased from a carrier to connect several sites in Vancouver. It turned out it was only running at 1.5Mbps because the carrier had used a T-1 for part of the link. After speaking to the carrier, the department got the speed on the link increased.

Another bottleneck was the fisheries department’s old 100Mbps FDDI backbone which was supporting only 25Mbps because of congested uplinks.

“They said there was nothing wrong,” Klassen remembers. “They said things were slow and everyone was complaining but that the backbone wasn’t the problem because they were only at 25-per-cent utilization on it.”

However, that 25 per cent turned out to be the maximum speed the backbone could support. So what looked like low utilization was in fact maximum utilization.

The coast guard and DFO asked Klassen for network repair recommendations. He considered several repair options, but in the end decided the only way to get the network running properly was a completely new installation.

The coast guard and DFO agreed with Klassen’s recommendation and approved a totally new network. Installation began late in 1996.

A big challenge in installing the new equipment was the network couldn’t be taken down for more than a couple of hours at night. The DFO regional office had WAN links to fisheries field offices throughout British Columbia, as well as a connection to the DFO in Ottawa. The links carried crucial fisheries data and scientific applications for fish management, as well as large financial programs.

Because the network couldn’t be taken off-line, IBM and the two departments decided to handle the installation in four phases. Each phase would cover half a floor of the two-floor fisheries office. As each portion of the new network was installed, it would be connected to the remains of the old network so all employees would stay connected.

A temporary “swing space” was created on a separate floor in the fisheries tower and attached to the network through a fibre optic cable to accommodate users whose workstations were being connected to the new portion of the network. Once their half-floor was finished, users moved out of the swing space and back to their old location. The next half-floor of users then moved into the swing space.

“What was interesting,” Razenberg noted, “was that as each phase was brought on-line, even though the entire system was integrated with the old network, the performance increased immediately.”

Perhaps the biggest hurdle in installing the new network was dealing with the piles of cable used by the original network. Razenberg said the main computer room in the fisheries office had a 10-inch raised computer floor completely filled with coils of cable.

“You couldn’t stick a toothpick in there,” he remembers.

Because the installation was taking part in phases, it was a challenge to figure out which old cable to cut out and remove.

“If anything was a problem for us up until the end, it was to keep the old and new running simultaneously,” Razenberg said. “And they had to be integrated, because they were using the same servers.”

The entire installation lasted a year and a half, winding up in the spring of 1998. The new all-NT network, which supports 400 combined fisheries and coast guard employees, consists of approximately 20 3Com workgroup switches located in four wiring closets running back into a core 3Com switch in the computer room. Also hanging off the core switch are several servers and Cisco routers used for WAN connections.

Initially, Razenberg rolled out 100Mbps Ethernet only for the servers and employees using high-bandwidth CAD applications. Now, approximately half of Razenberg’s users have 100Mbps connections and soon every user should have a fast Ethernet connection.

Razenberg is pleased with the new network’s performance. After the network was finished, IBM did a traffic test and found utilization to be only 0.44 per cent. That compares to high double digits on the old network, according to Razenberg.

The improved performance has been very obvious to Razenberg’s help desk staff. The queue for the old network had been too long to handle; now network problems are virtually non-existent.

“About the only time there’s a problem on the network is if someone’s kicked the jack on the floor and shorted something out,” Razenberg said.

The network also has a lot of room to grow.

“We built a lot of redundancy in,” Razenberg said. “There’s spare capacity left in the ceiling in a junction box, so if fisheries decides to turn a boardroom into six cubicles then we only have to come out of that one zone box to service that area. We don’t have to start recabling or anything like that.”

Overall, Razenberg was happy with the entire implementation. There were very few hiccups and those that did occur were so minor they were hardly noticeable. His only regret is that the project took so long.

“If I were doing this again, I’d want to do the whole thing at once,” he said. “A year and a half for a cabling project is a long time.”