Comments on: Facebook: Hive – A Petabyte Scale Data Warehouse Using Hadoop

By: Anjo

Anjo — Wed, 22 Jul 2009 14:43:11 +0000

There is a new kid on the block, it is called HadoopDB, I think it can combine the strong points of Hadoop and a RDBMS. See http://dbmsmusings.blogspot.com/2009/07/announcing-release-of-hadoopdb-longer.html

Maybe you can use Oracle instead msql or progresql?

By: Chris Adkin

Chris Adkin — Wed, 17 Jun 2009 21:38:40 +0000

I’ve been working with Oracle for over eleven years and I would not want to give the impression that I’m a RDBMS or Oracle zealot. However, as much as I appreciate one size does not always fit all, I would dispute the claim that what the Facebook have got is a data warehouse in the context of something that can be mined or allow value to be extracted from using BI tools. I would assume that what they have both lacks the tooling and constructs (things like mviews, bit mapping indexes etc . . ) to allow this be performed on such a large amount of data. However, if this does what want, is scalable and runs off cheap commodity hardware, you can’t knock it too much. What would be really interesting would be something that allows Petabyte scalability of web content and BI + data mining. It will be interesting to see what comes along with 11gR2. Within the constraints of not breaching any NDAs, people have told me that there is some interesting stuff coming along. Could this be the sort of stuff that will take Oracle more in the direction of what Facebook have done with hadoop . . . I guess we shall find out at Open World 2009.

By: Noons

Noons — Thu, 11 Jun 2009 00:33:38 +0000

Heck, we pump 0.3TB/day into our DW with a commercial db and 1/10000 of the budget of Facebook.
I defy anyone to prove in simple terms, not market-speak, what value can be extracted and in what terms/what purpose from a 15TB/day DW!
But I have no doubts it can be demonstrated: Stonebraker has been at the forefront of impossible projects for so long nothing surprises me anymore on that front…