http://blog.chrisgoffinet.com Random thoughts. Wed, 24 Jun 2009 03:59:52 +0000 http://backend.userland.com/rss092 en So at Digg, we have been working our own Hadoop cluster using Cloudera's distribution. One of the things we have been working through is how can we split our large compressed data and run them in parallel on Hadoop? One of the biggest drawbacks from compression algorithms like Gzip is ... http://blog.chrisgoffinet.com/2009/06/parallel-lzo-splittable-on-hadoop-using-cloudera/