A vulnerability in the Pretty Good Privacy (PGP) encryption protocol could let a snooper trick recipients of encoded messages into deciphering them and sending them back to the would-be spy, researchers warned Monday as they released a paper documenting tests of such an attack.

The loophole, which researchers have known of for several years, is a difficult one to exploit, largely because the pre-encryption compression done by most programs foils the attack. Still, implementations that precisely adhere to the widely used OpenPGP standard would be susceptible to attack, according to the researchers, who recommend changes to the standard.

The attack involves an element of social engineering as well as code-hacking. Known as a “chosen-cipher attack,” the ploy relies on a user having set his e-mail software to automatically decrypt incoming messages. If an attacker can intercept a message en route, he could apply an algorithm to garble the message, then pass it along to the intended recipient with his own e-mail address in the reply line. Receiving the mangled message, the recipient might reply back to ask what was being sent.

Completing the attack requires receiving from the recipient a quoted copy of the garbled message, after it’s been run through the recipient’s decryption. That decrypted text can then be de-scrambled by the attacker.

The attack was tested against two popular PGP implementations, PGP 2.6.2 and GnuPG, by researchers Bruce Schneier, founder and chief technical officer of IT security services firm Counterpane Internet Security Inc. in Cupertino, Calif., and Kahil Jallad and Jonathan Katz, who were with Columbia University when the research was conducted. Both programs compress data by default before encrypting it, complicating or thwarting the attack, but that option can be disabled by users.

PGP and GnuPG users should avoid turning off compression and be wary of including the full text of messages when sending replies, the researchers said.

Further details on the vulnerability and the researchers’ are available online at http://www.counterpane.com/pgp-attack.html, and will be presented at the 2002 Information Security Conference, which begins in late September in S