IssueRaiser

Most people miss this important point: the most important part of MapReduce is the shuffle step, which is a global, partitioned disk-to-d...

Most people miss this important point: the most important part of MapReduce is the shuffle step, which is a global, partitioned disk-to-disk sort. See the FlumeJava for a bit more discussion on why this is relevant. Everything else about MapReduce is just framework to make programmer's lives easier. BTW, not having strong typing in classic MR is a major pain point. Flume goes a long way to addressing this in a practical way. BTW, this point elucidates the importance of this classic Google Interview question: http://www.glassdoor.com/Interview/Sort-a-million-32-bit-int...

Posted by dekhn on Jul 18, 2014, 3:52 PM

Tags

Related issues

Raise real-world problems, find people facing the same issue, and discuss solutions.9,500 issues0 comments

hackernews (Other)

Most people miss this important point: the most important part of MapReduce is the shuffle step, which is a global, partitioned disk-to-d...

dekhnJul 18, 2014, 3:52 PM

Most people miss this important point: the most important part of MapReduce is the shuffle step, which is a global, partitioned disk-to-disk sort. See the FlumeJava for a bit more discussion on why this is relevant. Everything else about MapReduce is just framework to make programmer's lives easier. BTW, not having strong typing in classic MR is a major pain point. Flume goes a long way to addressing this in a practical way. BTW, this point elucidates the importance of this classic Google Interview question: http://www.glassdoor.com/Interview/Sort-a-million-32-bit-int...

Comments (0)

No comments yet. Start the discussion.

Comment on this issue