What Floats Your HPC Boat?

Published on Monday, 26 June 2006 11:47
Written by Douglas Eadline
Hits: 4924
Microsoft has announced their HPC cluster version. There has been plenty of discussion on the Beowulf List (look for MS HPC). In addition, I highly recommend reading Joe Landman's thoughts at his scalability.org Blog (Tactics versus strategy for the HPC market, How the Microsoft WCC could be good or bad, and A cluster system from Microsoft. Everyone in the Linux HPC community seems to be digesting and analyzing this news. A good thing in my opinion. It is what the community is not doing that troubles me.

Microsoft seems to think they can make clustering easier, well maybe they can make setting up and managing a cluster easier, but as I have said in the past, HPC is still hard and as far as I know, no one can point-and-click their way through the issues. And, setting up Linux Cluster is not that difficult in any case. In a way this is fine, the more people trying to do cluster HPC, the better. I like to think, Microsoft may raise the tide for everyone in the market. This assumption, however, implies that you have a boat without leaks.

So let's take a look at the HPC Linux Cluster boat. When MS puts their marketing machine in gear they will be sending a clear an consistent message to the HPC community about the Microsoft boat and the rising tide of HPC. At this point I have to ask "What is the Linux Cluster message and who is driving the boat?"

Over the last 3-4 years I have been involved in trying to help formulate and broadcast the Linux HPC message for the Linux Cluster community, first with ClusterWorld Magazine, and now ClusterMonkey.net. I also worked quite a bit on putting together what I call The Cluster Agenda In particular, I designed the Cluster Agenda to be a community Wiki that will provide a "road map of best practices and challenges" to the Linux cluster community. My vision was that it would provide a starting point for those interested in HPC cluster computing and provide a consistent message from what seems a fragmented Linux HPC space.

To date, the Cluster Agenda has seen minimal input from the community. (Even after numerous invitations to those on ClusterMonkey.net and the Beowulf Mailing list) In addition, I have seen minimal interest from vendors to help support ClusterMonkey.net or the Cluster Agenda. I find this both puzzling and troubling. As the HPC tide rises, the Linux HPC boat may not even float.

I am not sure how to interpret the inability of Vendors, who make a living in this market, to see the value in supporting the community. It reminds me of the short-sighted and self centered vendor strategy used in the UNIX wars. And, we know whose boat is still floating after the dust settled.

From the community chatter on the Beowulf list, the sense I get is that many people are concerned about the MS foray into this space, but short of stating various doom and gloom scenarios, see very little else that can be done. Perhaps it doesn't matter. I'm interested in what the community thinks. If you have some comments, please add them below (As away to reduce comment spam, we ask that you register first).

As for the ClusterMonkey.net and the The Cluster Agenda, some of the ClusterMonkey volunteers and myself will continue to help support the community with these efforts (you can help as well), but our boat still has a few leaks and the water is rising.

Unfortunately you have Javascript disabled, please enable Javascript in order to experience the comments correctly