转载:Hadoop 应该用C++实现，而不是Java

http://www.trendcaller.com/2009/05/hadoop-should-target-cllvm-not-java.html
Sunday, May 10, 2009
Hadoop should target C++/LLVM, not Java (because of watts)
< type="text/javascript">
digg_url="http://www.trendcaller.com/2009/05/hadoop-should-target-cllvm-not-java.html";
Over the years, there have been many contentious arguments about the
performance of C++ versus Java. Oddly, every one I found addressed only
one kind of performance (work/time). I can't find any benchmarking of
something at least as important in today's massive-scale-computing
environments, work/watt. A dirty little secret about JIT technologies
like Java, is that they throw a lot more CPU resources at the problem,
trying to get up to par with native C++ code. JITs use more memory, and
periodically run background optimizer tasks. These overheads are
somewhat offset in work/time performance, by extra optimizations which
can be performed with more dynamic information. But it results in a
hungrier appetite for watts. Another dirty little secret about Java vs
C++ benchmarks is that they compare single-workloads. Try running 100
VMs, each with a Java and C++ benchmark in it and Java's hungrier
appetite for resources (MHz, cache, RAM) will show. But of course, Java
folks don't mention that.
But
let's say for the sake of (non-)argument, that Java can achieve a 1:1
work/time performance relative to C++, for a single program. If Java
consumes 15% more power doing it, does it matter on a PC? Most people
don't dare. Does it matter for small-scale server environments? Maybe
not. Does it matter when you deploy Hadoop
on a 10,000 node cluster, and the holistic inefficiency (multiple
things running concurrently) goes to 30%? Ask the people who sign the
checks for the power bill. Unfortunately, inefficiency scales really
well.
Btw, Google's MapReduce framework is C++ based. So isn't Hypertable
, the clone of Google's Bigtable distributed data storage system. The rationale fo