Conference summary offers big data leads

The open source Hadoop framework for storing and processing large datasets is attracting an increasing number of organizations who want to at least try it because their information stores are rapidly increasing.

After a recent big data conference Ben Lorica, chief data scientist at O’Reilly Media pulled together a detailed summary of things he heard which deserve to be followed up.

For example, many companies are struggling with how to process and mine near real time data streams. An Intel official talked about how the chipmaker uses the in-memory cluster computing research from the University of California at Berkeley called Spark and Shark.

SQL-on-Hadoop solutions are now of interest with the release of Cloudera’s Impala query engine that runs on top of Hadoop, and Hadapt’s data-driven schema. Both were discussed at the conference.

And there was also a session about a corner of data science that I haven’t heard of called  adversarial analytics — think of behavioral models that try to detect cyber intrusions and black hat hackers that try to evade them.

Sometimes you don’t have to go to a conference to pick up nuggets that are worth pursing on your own time. That makes this column worth scanning.

Read the full column here

Howard Solomon
Howard Solomon
Currently a freelance writer, I'm the former editor of and Computing Canada. An IT journalist since 1997, I've written for several of ITWC's sister publications including and Computer Dealer News. Before that I was a staff reporter at the Calgary Herald and the Brampton (Ont.) Daily Times. I can be reached at hsolomon [@]

Would you recommend this article?


Thanks for taking the time to let us know what you think of this article!
We'd love to hear your opinion about this or any other story you read in our publication.

Jim Love, Chief Content Officer, IT World Canada

Featured Download

ITW in your inbox

Our experienced team of journalists and bloggers bring you engaging in-depth interviews, videos and content targeted to IT professionals and line-of-business executives.

More Best of The Web