Making Hadoop faster with GPU

By enabling the distributed processing of large data sets across clusters of servers, Hadoop has transformed the way organizations manipulate big data.

The open source framework is capable of rapidly scaling from a single server to thousands of machines and processing information really fast. However, is it possible to make Hadoop faster?

Researchers hypothesize that by offloading calculations from a central processing unit (CPU) to a graphics processing unit (GPU) designed for complex 3D and mathematical tasks, it could be possible to bump up Hadoop’s performance. This is because GPUs can perform calculations 50 to 100 times faster than their CPU counterparts.

RELATED CONTENT

Big Data will mean big year for Hadoop
3 awesome data analytics success
4 ways to tweak business analytics tools
Enterprise software myths dispelled

Using GPU to speed up Hadoop is not a new concept. There have been projects in the past which combined the Hadoop of MapReduce approach with a GPU. For instance, the Mars MapReduce-GPU project managed a 1.5 to 16 times increase in performance in analyzing Web data and processing Web documents.

An article written by the research and development team of big data specialists Altoros Systems Inc., indicates that there is a demand for accelerating parallel computing systems with GPUs . The article also illustrates how organizations can try it in a large scale.

Hardware vendors such as Cray have released machines equipped with GPUs and configured with Hadoop, according to the researchers.

In some scenarios, a GPU can accelerate computations by nearly five to 25 times per node. Some developers claim that if clusters consisting of several nodes are deployed, performance boost can reach 50x to 200x.

To find out more about the tools available to create a Hadoop + GPU system and the issues you will face, click here.

 

 

Would you recommend this article?

Share

Thanks for taking the time to let us know what you think of this article!
We'd love to hear your opinion about this or any other story you read in our publication.


Jim Love, Chief Content Officer, IT World Canada

Featured Download

Featured Articles

Cybersecurity in 2024: Priorities and challenges for Canadian organizations 

By Derek Manky As predictions for 2024 point to the continued expansion...

Survey shows generative AI is a top priority for Canadian corporate leaders.

Leaders are devoting significant budget to generative AI for 2024 Canadian corporate...

Related Tech News

Tech Jobs

Our experienced team of journalists and bloggers bring you engaging in-depth interviews, videos and content targeted to IT professionals and line-of-business executives.

Tech Companies Hiring Right Now