Power Usage and GigE Switches - Let's Talk

Article Index

Good GigE Switches?

On March 5, 2004, Russell Nordquist posted to the Beowulf list looking for recommendations for good 24-port GigE switches. He asked for input but wanted to stay away from "low-end" vendors (pick your definition or "low-end" here). There were a number of recommendations, but the thread ended with a very nice tool for testing switches. However, let's start with the recommendations. Please remember that these are personal recommendations from individuals on the Beowulf mailing list. Their experiences don't reflect any endorsements from this website or me. As always, YMMV (Your Mileage May Vary), test your codes.

Update: There was a recent and interesting discussion in March of 2006. Look for Gigabit switch recommendations by Joshua Baker-LePain. The thread continues into April where Joshua put an SMC SMC8748L2 through a combined netpipe and netperf testing regime and found it held up quite well.

The very experienced Mark Hahn posted that he liked the SMC 8624T switches. He mentioned that the number 140 cluster in the Top500 used them (this was before the June 2004 Top500 list). Lars Henriksen posted that they used HP 2724 switches. He said that they were stable under heavy load and were about $2,000 for 24-ports. However, Joel Jaeggli pointed out the the HP switches can't do jumbo frames which could be an issue for improved performance for some applications. Gerry Creager posted that he just bought some Foundry EdgeIron 24-port switches for about $3,000 a switch. He also mentioned a few personal opinions and facts: he doesn't like 3Com switches for clusters; HP has been using Foundry components (he couldn't confirm that); and he would stay away from Asante. Trent Piepho asked what people thought of the Dell unmanaged 2624, 24-port GigE switch that was selling for around $330. There was no response.

The final post of this thread is from the very knowledgeable Bill Broadley, who posted a link to a tool he's written to test the performance of switches. It uses MPI_Send to send various size packets between sets of nodes (something like NetPIPE) through the switch. The tool can measure some latency and bandwidth estimates through the switch and also watch for network saturation. [Note: Also check out Microway's MPI Link-Checker (tm)]

In a similar thread, on May 21, 2004, Konstantin Kudin asked what people recommended for good Gigabit switches as he had heard stories of inexpensive switches that choked under heavy load.

The first response was from William Harman who recommended two 1U HP Procurve switches, the Procurve 2824 (list price of $2,499 for 24 GigE ports), and the Procurve 2848 (list price of $4,899 for 48 ports). The 2824 has a backplane bandwidth of 48 Gbps (Gigabits/second) and the 2848 has a backplane bandwidth of 96 Gbps. Michael Hanulec recommended the Nortel Baystack 5510 series of switches. Michael said it had a backplane bandwidth of 160 Gbps in a single chassis, and 1280 Gbps when eight switches are stacked together. Mark Hahn also posted that he thought the cheaper/smaller switches from smaller, less well known companies are not necessarily worse because they probably use the same chips as the larger manufacturers.

{mosgoogle right}

Sidebar One: Links Mentioned in Column

kill-a-watt

Bonnie++

pthread.c

nrelay.c

This article was originally published in ClusterWorld Magazine. It has been updated and formatted for the web. If you want to read more about HPC clusters and Linux you may wish to visit Linux Magazine.

Jeff Layton has been a cluster enthusiast since 1997 and spends far too much time reading mailing lists. He can found hanging around the Monkey Tree at ClusterMonkey.net (don't stick your arms through the bars though).

    Search

    Feedburner

    Login Form

    Share The Bananas


    Creative Commons License
    ©2005-2012 Copyright Seagrove LLC, Some rights reserved. Except where otherwise noted, this site is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License. The Cluster Monkey Logo and Monkey Character are Trademarks of Seagrove LLC.