Where is the Cluster? Microsoft HPC at SC05

Published on Monday, 21 November 2005 19:00
Written by Douglas Eadline
Hits: 8078

Some thoughts, some opinions, and some things to ponder

I'm sitting in the overflow room for the Bill Gates Keynote at Supercomputing '05 (SC05). Odd, his talk is being given in a medium sized room, I am told they have bigger rooms in the convention center. Ah, but why risk empty seats. First, let me say how bazaar this is. Bill Gates at SC, like the CEO of Pepsi addressing the Coke shareholders meeting, no it is worse, it is like Terrel Owens explaining the word humble to the Dali Lama. The term non sequitur comes to mind.

At first, I thought, why go. What could Bill Gates possibly tell the people who have pioneered high performance computing. But then, my curiosity got the better or me. You know, mixing unknowns. Alchemy is interesting. New science if you will.

Some history may be helpful. High Performance Computing (HPC) has always been a UNIX market. Why? A real well-engineered Operating System was needed. HPC pushes the limits of everything. More interesting is how Linux, with its "Plug and Play" proposition for UNIX users, has literally taken over the HPC market. There are other reasons some rather subtle, which I wrote about in Why Linux On Clusters? Now it seems Microsoft wants in. I say welcome, but be aware. This market is one of collaboration, community, and results. High-tech hard ball, I like to say.

Before I offer some other thoughts, let's see what Bill has to say. He began of with some Altair/Basic Interpreter comments to which many people could relate. He then launched into a "This is HPC" kind of presentation. I found it interesting that the audience, who developed most of what he was talking about, never did get acknowledged. He spoke as if some how Microsoft discovered or invented this new market. He mentioned that "Technical Computing" was an area MS was going to enter. The challenges of transitioning from 32-64 bits, parallel software, digitization of everything, etc. Yawn, we know that already. So what is new?

The concept of personal supercomputer and a great marketing phase "time to insight" was mentioned. A standard kind of Grid diagram showing data generation, data mining, sharing, and finally insight. As I understood it will be managed by a Data Workbench and use Visual Programming. That is good news, we need tools, so how will it work together?

It seems XML is the glue that will be used to connect the flow of data from cluster to computer to user and back. A great and obvious idea. XML is a an open standard. If used in an Open way XML is good tool. Oh yes, there is that Open word splashed all over everything Microsoft at the show. I am not sure what they were trying to say. Did it mean their booth is Open for visitors?, Were they Open to new ideas?", or were they trying to convince people they use things like Open Source, but just could not get the last word out? I don't know. In any case, Bill stated that XML is a revolution at Microsoft and will be used for everything.

{mosgoogle right} Next, there were some examples. By way of the ten, yes ten, worldwide Microsoft HPC institutes, a local example of research on under ocean lava vents was shown. Interesting application, but one has to wonder, was this not what Grid computing people have been saying for years. Remote sensors, collecting data, processed some place else, available to many etc.

The next example was a live demonstration. Kyril Faenov, Director of HPC at Microsoft showed a Reduced Time to Insight for Cancer detection using a Genetic Algorithm (GA). The application and science seemed sound enough, GAs are powerful tools and ideal for clusters. The demonstration was a little funny. Back stage they used a small Linux Cluster and off site they used a larger Windows Cluster. Both systems were working together to produce the needed results -- using XML I bet. The idea was that you could easily move from a small cluster to a larger one with very little changes. Again, I keep getting this Grid computing deja-vu, but I am sure it is bug in the matrix.

So we end up with Bill telling us the next ten years are going to be important. And, now the questions. This part is what we came to see. Those hard ball questions about playing fair and not using standards as way to eliminate competition. The first question was about interoperability with current standards and working with the GGF (Global Grid Forum). Bill's response went something like, "protocols and lots of standards need to be independent, we started with XML, and are moving to WSDL (Web Services).." and "project Indigo." That sounds interesting and inviting, though I doubt anyone believes it. Another question was about programming. Bill quite rightly, in my opinion, suggested that declarative languages are needed to program parallel computers. Now there is a place to plant a flag.

In general, the talk was rather bland. It did not inspire me or strike the fear of God in me because Microsoft was entering HPC. The talk and the Microsoft exhibit did generate some questions, however. First and foremost. Where is the "Microsoft Cluster?" I fully expected to see at least two racks of Dell servers running the Compute Cluster Server 2003 (beta) software in the Microsoft booth area. Understand that the exhibit hall was two big rooms connected by a large corridor with Microsoft on either side of the corrodor. You could not miss them. They had a bunch of ISVs, a place for presentations, but no big cluster. I assume they had been to Supercomputing in the past. You know, the place where everybody is running a cluster. Why no cluster on the main drag? Why no big cluster that says "Hey we can play to, look at us, we have blinking lights". For the life of me I do not know why they had no cluster. I'm not even going to speculate. [It was pointed out to me that there were clusters running MS Cluster Server, which I assumed, but to my suprise there was the lack of a "big cluster" that we are all accustomed to seeing at SC] The other omission, at least from my perspective, was no "dot net" talk. I had assumed this was going to be one of the foundations for MS clusters. Not that it would work that well for all of HPC, but still the only "dot net" I heard was when I was telling everyone to go to "ClusterMonkey.net"

In terms of Microsoft entering the market. I welcome them. I think it will raise the tide for the whole market. The more people are thinking about clusters and cluster programming the better. I think Microsoft can contribute as well. There is always room in our community for those that help us move forward. As we like to say, give a little, get a lot.

Now about those fears that Microsoft will come in an dominate the market by any means possible. I doubt it can happen. Microsoft is able to use this tactic when they own a market. They do not own HPC, Linux and open source are the "Lego" we use to build solutions. You need to make sure your bricks fit.

Finally, it is said Microsoft spent one million dollars on the event -- complete with the Sheryl Crow concert. (I did get to enjoy the last three songs, by the way). I would suggest the best way to embrace this community is to spend some of that "kind of money" on Open projects that benefit everyone. And, next year "free as in beer" will work just as well on this audience.

Douglas Eadline is the swinging Head Monkey at ClusterMonkey.net.

Unfortunately you have Javascript disabled, please enable Javascript in order to experience the comments correctly