Apple's Leopard has security holes, experts say

The security features introduced in Apple's Leopard operating system need work.

That's according to security experts who have been putting the new version of Mac OS X through its paces, since the upgrade was introduced last Friday.

Leopard introduces a number of important security features to the Mac, but they are often implemented incompletely, leaving users vulnerable to attack, said Thomas Ptacek, a researcher at Matasano Security, who Monday wrote a detailed assessment of Leopard's security.

"They've done a really good job of robbing Microsoft advocates of their talking points," he said. But, "I don't see anything that they've done out of the box, where it's really any more resistant to attack than Tiger was," he added, referring to the previous update to Apple's operating system. According to Ptacek, two of Apple's key security enhancements -- Sandboxing and Library Randomization -- are great ideas that are imperfectly applied within Leopard.

Take Library Randomization. It's a new feature that's supposed to make it hard for some of the most commonly used computer attacks like buffer overflows, where the attacker takes advantage of a software bug to place code somewhere in the computer's memory where he knows it will be run. Microsoft developed a similar technology for Vista, called Address Space Load Randomization. Library Randomization makes it much harder, if not impossible, for the attacker to know where to place this code, reducing the risk of attack.

The problem is that Apple did not randomize all of the parts of the operating system that it should have, according to Ptacek. In particular, Apple's Dynamic Link Library has not been randomized.

Security researcher Dino Dai Zovi said he's used this library in several of the Mac exploits he's written over the past few years. He has taken advantage of the fact that this library is not randomized, he agreed with Ptacek's assessment that this feature, as it's implemented in Leopard, would simply make things a little more difficult for attackers.

Sandboxing is another feature that could ultimately make Mac OS X more secure. Sandboxing restricts software running on Mac OS so that even if it's hacked, it can't do things that it shouldn't, such as add new software to the computer. The problem is that Apple hasn't sandboxed many of the most commonly attacked applications such as the browser, mail client, or instant messaging software, Ptacek said.

And the programs that have been sandboxed have not been walled off as thoroughly as they should be, he added.

For example, the Quick Look file viewer has been sandboxed, but only to restrict network access. The software can still be misused to write malicious files where they will be automatically launched, Dai Zovi said. "Most of the things that were sandboxed were network services," he said. "Increasingly these days IM, e-mail and Web surfing are where most of the attacks are coming from, not directly on your network."

Independent consultant Rich Mogull said that his biggest problem was with the Leopard firewall, which he said suffered from a confusing interface that made it very difficult to control access to individual services on the Mac. "It was very complicated and very hard to get the right settings," he said.

Worse, when he installed Leopard, he found himself suddenly without a firewall. "It turned off my firewall when I upgraded, despite that being a default setting." he said.

Like Ptacek and Dai Zovi, Mogull said he had been expecting more from Apple with the Leopard release, but he agreed that the new security features were a step in the right direction. "I think that Apple has started down the right path but they are not as far as they communicated that they would be," he said. "The firewall is the big negative; they really messed that up."

Apple declined to comment in detail on its new security features. Company spokesman Anuj Nayar said via e-mail that "Apple takes security very seriously and has a great track record of addressing potential vulnerabilities before they can affect users."

Ptacek said that it is great that Apple has begun adding these security features even when the Mac has not been the target of a widespread worm or virus outbreak. "I'm impressed that when they didn't have to do it, they went after low-level features that no one will understand," he said. "I like the direction they're headed. I'm just saying that they've got a long way to go to catch up with Microsoft."