Server downtime is not an option for Calgary-based Fraser Milner Casgrain LLP (FMC).
Four years ago the law firm decided to get more proactive about disaster recovery. Its network team compiled a list of 16 possible disaster scenarios and set out to determine which ones were most likely to occur, said Greg Dawson, who was the network services supervisor at that time.
“(The scenarios) ranged from your 9/11 type of disaster where the office could lose the entire server farm and all the bodies associated with maintaining it, down to a patch release blue-screening a server.”
Of all the possibilities, FMC determined that patch management was the activity most likely to cause a server to go down. On average, FMC’s IT staff installed between four and 11 Microsoft security patches every month. “We know we have to make fundamental changes to our servers with patches, which makes (server downtime) a common risk.”
Patching-related risks escalated when IT staff found they could not remove some patches via the regular Windows uninstall process, Dawson said. Manually removal was possible in some cases, but it took too long to determine exactly what to uninstall.
FMC decided to image its servers on a monthly basis through a ghosting process, whereby the IT team would spend 12 hours on a Thursday evening taking down the servers, imaging them, deploying the patches and testing the systems to make sure everything worked before the next day. But as FMC added servers to its infrastructure, ghosting became too time-consuming; the imaging process took up almost the entire maintenance window, leaving little time for patching.
“Ghosting was great when we had a dozen servers,” Dawson said. “But when you get beyond that, it becomes a nightmare.”
While searching for a product that could image its servers in a shorter time frame, FMC discovered Acronis True Image Server about 18 months ago when the software was still at version 6. According to Dawson, True Image Server gave network staff more flexibility during the imaging process than did ghosting.
“This product allowed us to take a snapshot of the server on a Thursday night and we were able to do that when the server was in production, so people wouldn’t notice that we were doing an image.” Every time IT staff needed to make a critical change to the server, True Image Server let them take the image, make the changes and if they did not like the modifications, they could roll the servers back to their previous status.
Version 8 adds the ability to do incremental or differential backups of only the changes that have occurred in the last 24 hours since the last backup, rather than doing a full image of the server every night. According to Dawson, incremental backups take only five minutes and 50MB of storage space, as compared with one hour and 500MB to 2,000MB of storage for a full server image.
Before choosing True Image Server, FMC tested three other products, including Symantec’s LiveState Recovery Advanced Server (previously V2i Protector), which at the time offered similar features. FMC chose Acronis’ product after the vendor added a central server management console to version 8 of True Image Server — a feature that had also been available in Symantec’s product.
Total testing time lasted about nine months because FMC tried out multiple products in several offices and re-tested products when they came out with new features. “It was a process of deliberately breaking things, doing restores and seeing that everything was seamless,” Dawson said. “Then we were able to move to less mission-critical devices and test some of those.”
Three of FMC’s six Canadian offices have implemented True Image Server. Of the remaining offices, one had planned to implement the product but never picked up a copy; another extended its backup suite; and the last one went with a completely different solution.
At the Calgary office, the True Image Server installation was “wonderful and idiot-proof,” Dawson said. Network staff only had to “slap in the CD, click, sit back and watch it work….The most difficult thing was typing in the license key.”
The implementation at the Calgary office took about two weeks because it was done in stages, starting with some redundant servers during the week and moving on to others afterward. Now FMC does a full system backup every Thursday night and a differential backup on other weeknights. That way, if a server dies three Tuesdays after a patch rollout, FMC can roll it back to a previous Friday’s image, or as far back as the image taken right after the last patch deployment.
Since installing True Image Server, FMC has reduced its maintenance window by at least 25 per cent. “We changed our entire strategy on how we do patching every month based on our ability to recover the server,” Dawson said. Instead of installing one patch, rebooting the server, checking the logs and the applications and repeating the process for subsequent patches, IT staff can now take an image, bundle all of the patches and push them out at the same time.
The firm has also reduced its server recovery time. About five months ago, one of FMC’s servers died during a power shutdown mandated by the City of Calgary. The server in question facilitated the document scanning process at the company. According to Dawson, it took about five minutes to recover the server, whereas before it would have taken at least an hour to install Windows, then add all the applications and configure the system.
Quick Link: 051319