Increasing Performance of a 100-Node Hadoop Cluster: The Story of Joyent

by Alex KhizhniakAugust 30, 2013
Joyent now produces disk I/O response times 3x faster than similar solutions from other IaaS providers.

Hadoop in the cloud

Previously, companies used to run Hadoop on their own premises. However, a possibility to use resources on demand and pay for the capacities that are actually used seemed rather attractive to many customers. So, recently, Joyent—a leading IaaS provider—partnered with Hortonworks and rolled out a cloud-based Hadoop offering.

When preparing to deliver its Hadoop clusters, Joyent turned to Altoros to get independent evaluation of their system. The results showed that virtual Hadoop clusters can provide bare-metal performance with nearly a three times faster disk input/output speed compared to a similarly sized infrastructure from other vendors.

Ben Wen (Joyent) and Renat Khasanshyn (Altoros) at NoSQL Now! 2013

At NoSQL Now! 2013, Altoros and Joyent conducted a workshop based on this real-life case study. The participants learned how our company helped Joyent to provide the highest-performance Hadoop cluster on the market.

See the slides below to learn about the tools used for performance evaluation of different Hadoop clusters (50–250 nodes) and what kind of initiatives were implemented to achieve this significant improvements. Or, read about the technical details here.


Want details? View the slides!


Further reading