4. Communication speed (throughput) between nodes is important
The joke about a station wagon full of tapes being the fastest way to send large amounts of data is quite true. Intuitively, you know something is missing because we use wires and fiber instead of station wagons to send data. It is called latency.Doug's Latency Experiment: Next time you are in the shower, which for some of you fellow geeks may be next week, adjust the water to where it is comfortable. Now turn the hot water off and start counting. Note the time between when you turn the faucet until you scream. That time is the latency of your shower water. There is also the flush the toilet version of this experiment, but that takes two people.
So ask yourself, "How many times an hour can I make myself scream?" For those following along from home, this can be considered a thought experiment. If there was less of a delay between screams (low latency), you could increase the number of messages (screams) you can send via the shower.
So that is latency. If you have an application that needs to send a lot of messages in a short time (like a football team using cell phones) then latency is important. On the other hand if you only need to send a few messages, like our marathon runners, then latency is not that important.
Of course how much data we can send in a certain amount of time is also important. This is called throughput. Throughput is a rate, (bits/second) think of that station wagon full of tapes, OK a Hummer for you youngsters, moving along the highway. Latency is how long it takes to load and unload the tapes.
5. Communication speed and latency are important
Now that you think you are so smart. Throughput and latency are not the whole story. How much work a processor must do to move data is also important. If your crew that fills the Hummer with tapes is also responsible for creating the tapes, then work must be shared and the overall performance may suffer.
6. Communication speed, latency, and processor overhead are important
Now you think you are really smart. Not so fast. Communication speed, latency, processor overhead are all important, right? It depends.You now are a cluster expert. The answer to 90% of all cluster questions can be answered with the following:
It all depends on the application
The right hardware and software design can be very dependent on the application. Some applications (marathon types) run well on almost any cluster while other (football types) need very high performance parts.
"Why not just use faster parts and then every application works fine?" Cost is the issue here Skippy. When designing a cluster there are certain constraints. Cost is usually one such constraint. The old bang for the buck idea. Or, as I recall someone mentioning at some point price-to-performance.
7. The Top 500 list is the ultimate measure of computer performance
Behold the Top 500 supercomputers. Got to get me one of those thingies. The Top 500 list is a measure of how fast a specific program runs on a computer or cluster. It is a single data point. So take out a sheet of paper and put a point in the middle. That point is the performance of the worlds fastest computer running something called the "linpack benchmark". Now take out another piece of paper and put a similar point in the middle. That point is the performance of the last computer on the Top 500 list running a program called BLAST. Hold the two pieces of paper next to each other. Now which one is faster. Get the idea. In programming terminology, the scope of a clusters rank in the Top 500 is limited to the Top 500. When running other programs Your Mileage May Vary YMMV.Here is the last thing to remember.
HPC/Beowulf clusters are about building machines around problems
If you need a fast interconnect, then buy it. If you don't, then don't buy it and instead buy more processors. Maximize your performance and buy what you need to solve your problem. This process does require more engineering on the end users part, however. Of course, Cluster Monkey is there to help.
In conclusion, you should now be able to totally discount all those cluster urban legends you see all the time. And, if you have already forgotten the things you were supposed to remember, here is one last sentence to sum it all up:
there is no free lunch with clusters -- just a reasonable priced buffet on which to feed your computing needs
Douglas Eadline is the swinging Head Monkey at ClusterMonkey.net.
