I just heard through the Berkeley grapevine about the BashReduce effort at Last.fm: MapReduce in 126 lines of bash script! Awesome. I’m sure it doesn’t do X, Y and Z. So ask yourself: do you need X? Y? Z? Maybe instead you want V and W. Maybe you should roll your own tool.
Makes you think.
2 Comments
Or to turn this inside-out yet again, you can write a bash script and run it in parallel via Hadoop using ParBASH. I can’t wait for an easy implementation of Java map and reduce functions running over the BashReduce engine! Somebody please?
See also this implementation of HBase in bash: http://pastebin.com/f64fc84a9.