How open sourced software helped build Prism

Over the last week there has been no shortage of online articles, blogs and tweets about Prism, the program used by the United States National Security Agency for collection data in real time from companies such as Apple, Facebook, Google and Microsoft.

Public uproar over the until then secret, court approved Internet snooping by U.S. federal intelligence and investigation agencies has been load and swift, but a recent article on the online business site BloombergBusinessweek posits that similar groundswells of online communication used in crowdsourcing techniques and development of open sourced software are partly to blame for Prism and other surveillance technologies.

For instance, just as Google and Facebook have cheaply developed the technologies and techniques to glean from our online interactions how to calculate an individual’s preference for clothes, they have also help bring down the cost of analyzing a person’s chat logs.

It is also often thought that government agencies would likely chose proprietary Windows systems over Linux operating systems because open sourced systems could introduce risks since their underlying structures are open to other researchers. However, open sourced software benefit from the myriad of researchers working on it who can spot more bugs and work together to fix them.

The NSA and other covert organizations still spend millions of dollars in super computers such as those produced by Cray for pattern matching, says Bloomberg writer Ashlee Vance. But for co-relating nuggets of data culled from a myriad of online, mobile and other sources, open sourced software is a spy agency’s special weapon of choice, she reported.

Is it any coincidence then that Facebook, Google and Yahoo, all avid users and propagators of open source programs, also happen to be on the list of companies alleged to have complied with court orders to release information to federal agencies?

Vance also cites that for years, companies such as IBM, Microsoft and Oracle have sold their revolutionary database technologies to so-called “three-letter agencies.” More recently, Hadoop, the open source software framework for big data and MapReduce, Google’s programming model for processing large data sets, have were included in the leaked NSA slide which identified the technologies as vital in the NSA’s data mining operation.

Both technologies were open source applications develop by Google and Yahoo and later modified by thousands of developers. Such collaborative works gave rise to the big data era and its accompanying data analytics tools.

As early as 2009, the NSA had admitted it had taken source codes from companies such as Yahoo, Facebook and Twitter which have open sourced their underlying infrastructure. The NSA said it modified some of these codes for data analytics purposes.

In so doing, NSA was able to drink from the spigot of intelligence of thousands of computer scientists and programmers from all over the world.

Read the whole story here

Would you recommend this article?


Thanks for taking the time to let us know what you think of this article!
We'd love to hear your opinion about this or any other story you read in our publication.

Jim Love, Chief Content Officer, IT World Canada

Featured Download

Nestor E. Arellano
Nestor E. Arellano
Toronto-based journalist specializing in technology and business news. Blogs and tweets on the latest tech trends and gadgets.

Featured Articles

Empowering the hybrid workforce: how technology can build a better employee experience

Across the country, employees from organizations of all sizes expect flexibility...

What’s behind the best customer experience: How to make it real for your business

The best customer experience – the kind that builds businesses and...

Overcoming the obstacles to optimized operations

Network-driven optimization is a top priority for many Canadian business leaders...

Thriving amid Canada’s tech talent shortage

With today’s tight labour market, rising customer demands, fast-evolving cyber threats...

Staying protected and compliant in an evolving IT landscape

Canadian businesses have changed remarkably and quickly over the last few...

Related Tech News

Tech Jobs

Our experienced team of journalists and bloggers bring you engaging in-depth interviews, videos and content targeted to IT professionals and line-of-business executives.

Tech Companies Hiring Right Now