Tor, TrueCrypt, Tails topped the NSA's 'most wanted' list in 2012

In January 2012, it saw Internet traffic anonymizing tool Tor (The Onion Router), Linux distribution Tails (The Amnesic Incognito Live System) and disk encryption system TrueCrypt as the biggest threats to its ability to intercept Internet traffic and interpret other information it acquires.

Since then, flaws have been found in Tor and the FBI has unmasked Tor users and a vulnerability was found in Tails allowing attackers to determine users' IP addresses.

And while a source-code audit gave TrueCrypt a relatively clean bill of health in April, TrueCrypt's anonymous developers inexplicably abandoned the software a few weeks later, warning it was insecure.

That the NSA considered these tools dangerous is perhaps little surprise: In July it was revealed that the agency's XKeyScore traffic interception tool contains rules for tracking who visited the websites of the Tor and Tails projects.

But now German magazine Der Spiegel has published further documents from the cache leaked by Edward Snowden, including one outlining, on page 25, the tools the NSA most wanted to crack in order to intercept and decrypt its targets' communications.

The tools were ranked by their impact, from trivial to catastrophic, and their use risk, from current highest priority targets down to experimentation by technical thought leaders.

In the slide deck, the NSA explained that, with rare exceptions, it only developed "application-specific solutions" based on those two criteria, impact and use risk. In a resource-constrained environment, it said, the need for responses to current threats would always trump speculative work on threats that might become more widespread. Der Spiegel had something to say about those constraints: Of the NSA's 2013 budget of over US$10 billion, some $34.3 million was allocated to "Cryptanalysis and Exploitation Services."

Top of the NSA's list of major or catastrophic threats, capable of causing a majority or near-total loss or lack of insight into the highest-priority targets' communications or online presence, were Tor, Tails and TrueCrypt.

Of course, it's unlikely that the published attacks on Tor and Tails were developed by the NSA -- but with the Tor unmasking attack costing researchers just $3,000, the NSA could certainly have done something similar with its budget over the last three years. Although some of the wilder conspiracy theories linking TrueCrypt's demise to the NSA have evaporated, there is still no convincing explanation for why the developers abandoned a tool that had just come through a code audit with no major flaws found.

Other tools were also considered major or catastrophic threats, but of lesser priority because they were not yet, or no longer, used by the highest priority targets. Among the tools the NSA feared it might need to crack in future was encrypted telephony tool Redphone, which uses Phil Zimmermann's ZRTP secure key-agreement system for RTP (Real-Time Transport Protocol) voice communications.

Over two decades ago Zimmermann also developed PGP (Pretty Good Privacy), an encryption tool the NSA is still having trouble cracking, as illustrated by this slide published by Der Spiegel.

That PGP was not top of the NSA's most-wanted list could be down to its usability, which is such as to put off all but the more tech-savvy targets.

However, with ZRTP used to encrypt voice communications in off-the-shelf smartphones like the Blackphone, it's a fair bet that Redphone and its ZRTP-using ilk will be moving higher up next year's list.

The slide deck revealing the most-wanted list also held another couple of technical challenges the NSA faces -- ones that might be more familiar to enterprise users.

One slide lamented that "Excel tops out at a million rows," making Microsoft's spreadsheet inadequate for handling more than a couple of weeks' "summarized active user events" from one of the NSA's data capture programs alone. Using four or five pivot tables to visualize the data from each of thirty target sets, two weeks' data would generate 100 to 150 slides, the NSA presentation said.

Like many other organizations, the NSA apparently had a big problem with unstructured data. Slide 37 warns that "TKB/UTT (Target Knowledge Base/Unified Targeting tool) are victims of years of 'fill in the blank' freeform data entry." As of 2012, this was "very slowly being addressed" with a target date for completion of "~2015."

With Snowden's trove of documents all predating May 2013, when he fled from Hawaii to Hong Kong, we'll have to wait for another leaker to come forward before we find out whether the NSA hit that 2015 deadline, and what progress it has made with its other software challenges.

Peter Sayer covers general technology breaking news for IDG News Service, with a special interest in open source software and related European intellectual property legislation. Send comments and news tips to Peter at peter_sayer@idg.com.