Interconnects
Getting the data where it needs to go is only half the story. getting it there quickly and with minimal latency is the issue with clusters. Whether it is one byte or a gigabyte, interconnects are the get the work done.
Experiments With Switchless 10GigE: The Bonded Loop
- Published on Thursday, 11 April 2013 12:27
- Written by Douglas Eadline
- Hits: 499
When the best solution just won't fit the box and the budget
Modern Ethernet technology is based on network adapters and switches. Using Ethernet without a switch only happens in rare situations where a small number (e.g. two) systems need to be directly connected together. Such a connection is often called a "cross over" connection because a special cable may be needed.
The cost of adapters and switches follow a very predictable commodity pricing trend. At first the cost of systems is quite high and decreases as the sales volumes increase. Currently Gigabit Ethernet (GigE) enjoys low cost and wide availability from multiple vendors. Ten Gigabit Ethernet (10GigE) is now experiencing greater acceptance and thus decreased costs. Although volumes are growing, 10GigE still commands a high per port price (Adapter/Switch) and thus can be an expensive option for many small projects.
Read more: Experiments With Switchless 10GigE: The Bonded Loop
Benchmarking OpenFOAM on GigE, 10GigE, and InfiniBand
- Published on Friday, 18 March 2011 11:11
- Written by Douglas Eadline
- Hits: 2724
Nothing like real data from real machines running real problems, really
The best way to evaluate any technology is through benchmarks. Recently, I read this short white paper entitled CORE-Direct: The Most Advanced Technology for MPI/SHMEM Collectives Offloads. A long title, but it talks about a good idea Mellanox has added to it's end-to-end InfiniBand (IB) hardware stack. (By the way, Mellanox also has end-to-end 10 GigE solutions). After reading this paper, I wondered if there were any benchmarks for the CORE-Direct feature. After a little searching, I found some pdf presentation slides that are worth reviewing.
Read more: Benchmarking OpenFOAM on GigE, 10GigE, and InfiniBand
Probing Gigabit Ethernet
- Published on Wednesday, 24 August 2005 20:00
- Written by Douglas Eadline
- Hits: 19095
How to Benchmark TCP/IP Ethernet Performance
There are two ways to look at designing HPC clusters. On the positive side, there is a plethora of hardware and software options. On the negative side, there is a plethora of hardware and software options! The cluster designer has a special burden because unlike putting together one or two servers for the office, a cluster multiplies your decisions by N, where N is the number of nodes. A wrong decision can have two negative consequences. First, a fix for the problem will probably require more money and time. And, second, before the problem is fixed you may only be getting a fraction of the performance possible from your cluster. Clusters have a way of amplifying bad decisions.
