Select News

The news in this category has been selected by us because we thought it would be interestingto hard core cluster geeks. Of course, you don't have to be a cluster geek to read the news stories.

In Case You Were Wondering: Apache Spark, Hadoop, and HPC

Moving beyond MapReduce, if that is your cup of tea

Recently Hadoop distribution provider Hortonworks, announced it was Extending Spark on YARN for Enterprise Hadoop. If you are not up to speed in the Hadoop world there are a few points of interest for HPC in this announcement. First, while Hadoop version 1 was synonymous with MapReduce, Hadoop version 2 has "demoted" MapReduce to an application framework that runs under the YARN (Yet Another Resource Scheduler) resource manager. Thus, Hadoop version 2 has opened up a Hadoop Cluster to many more applications other than MapReduce.

Read more: In Case You Were Wondering: Apache Spark, Hadoop, and HPC

Update and Help on "Shellshock" BASH Vulnerability

Keep Calm and Patch On

The BASH vulnerability has taken everyone by surprise -- much like finding the wheels of your 20-year old bike can easily fall off. For Red Had based distributions, you can follow the progress at Bash Code Injection Vulnerability via Specially Crafted Environment Variables (CVE-2014-6271, CVE-2014-7169). There were two updates to BASH for Red Hat Based systems over the last few days. It is safe to say the extent of the vulnerability is still not fully known and exploitation vectors are still being investigated. While web servers can immediately benefit from the work of the community, home and small office routers may also be at risk. Obviously fixing BASH is the best approach to reduce the risk. The current, least vulnerable, version is:

bash.x86_64 0:4.1.2-15.el6_5.2

and should be available at most repositories by now. A good discussion and latest un-official patch of the previous and new issues can be found on Google security researcher Michal "lcamtuf" Zalewski blog. A test to check for the latest vulnerabilities is the following line:

foo='() { echo not patched; }' bash -c foo

If you run this on your systems, and get "echo not patched," then you are at risk. If it shows "command not found", you have the latest patch. Of course other measures such as making sure cgi_module is not loaded by Apache are a good idea in any case (Unless you are using cgi scripts, which is not a good idea!). Other mitigation strategies are offered by Red Hat.

Docker versus KVM: Hard Numbers for HPC

From the numbers are nice department

HPC users have always want to be "close to the metal." A recent report by IBM has tested the popular Docker project against native performance (bare metal) and KVM (Kernel-based Virtual Machine). The tests include Linpack, STREAM memory benchmark, nuttcp for network bandwidth, netperf for network latency, fio and Redis for block I/O speeds, and the SysBench oltp benchmark for testing MySQL. The scripts used for testing are available for download.

The paper entitled An Updated Performance Comparison of Virtual Machines and Linux Containers (PDF) provides some solid numbers that may help in designing future HPC systems. As on might suspect, Docker containers were found to perform similar to native bare metal tests. For instance, Docker matched the bare metal Linpack tests while KVM was slower by more than half. Of course, you can have flexibility and bare metal performance using something like Warewulf.

Invest 27 Mintues, Learn The Truth

From the "It had to be said department"

Do yourself a favor and take a few moments and view the video Computers are Sadness: I am the Cure by James Mickens. It is the best state-of-the-art analysis of MapReduce, the Cloud, and Security. And it is hilarious!

Turn Up the Volume: This Week in HPC

There is a new HPC Podcast worth your time. This Week in HPC is presented by HPC analyst firm www.intersect360.com. The presenters are Addison Snell and Michael Feldman. Some advice; pay attention to these guys, they know what they are talking about. The full press release can be found below.

In addition, if you are interested in a discussions about more technical HPC issues, check out Brock Palen's RCE an HPC Podcast. There are also other videos and podcasts at HPC news site insideHPC. Have at it.

Read more: Turn Up the Volume: This Week in HPC

Search

Login And Newsletter

Create an account to access exclusive content, comment on articles, and receive our newsletters.

Feedburner

Share The Bananas


Creative Commons License
©2005-2012 Copyright Seagrove LLC, Some rights reserved. Except where otherwise noted, this site is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License. The Cluster Monkey Logo and Monkey Character are Trademarks of Seagrove LLC.