File Systems O'Plenty Part Two: NAS, AoE, iSCSI, and more!

Published on Wednesday, 01 October 2008 21:00

Written by Jeff Layton

Hits: 17647

Ever hear of an exabyte?

In our second part of File Systems O'Plenty we take a look at NAS, Distributed File Systems, AoE, iSCSI, and Parallel File Systems. In case you missed part one you can find it here. In this part, we will also point out why IO is important in HPC clustering. Many a CPU cycle is wasted waiting for that data block. Read on how to feed you data appetite.

Read more: File Systems O'Plenty Part Two: NAS, AoE, iSCSI, and more!

0 Comments

File Systems O'Plenty Part One: The Basics, Taxonomy and NFS

Published on Sunday, 27 July 2008 20:00

Written by Jeff Layton

Hits: 14691

Storage: its where we put things

Clusters have become the dominant type of HPC systems but that doesn't mean they aren't perfect (sounds like a Dr. Phil show doesn't it?). While you get a huge bang for the buck from them, somehow you have to get the data to and from the processors. Moreover, some applications have fairly benign IO requirements and others need really large amounts of IO. Regardless of your IO requirements you will need some type of file system for your cluster.

I wrote a file system/storage survey article for clusters in the past, but as always things change rather rapidly in the HPC arena. Originally, I had wanted to update the original article, however, the updates became so large that it's really an entirely new article. So this article, I hope, is a bit more in depth and a bit more helpful than the past file system article.

Read more: File Systems O'Plenty Part One: The Basics, Taxonomy and NFS

0 Comments

Using Strace to Examine Your IO

Published on Monday, 14 January 2008 19:00

Written by Jeff Layton

Hits: 9862

A special article for IO Knuckleheads

In my experience there are quite a few people who don't really know the IO performance of their application(s) nor the IO pattern. So rather than complain about this situation, as I'm apt to do, I decided to write a column that talks about how to capture the IO behavior of your code(s). These steps can be done for codes where you only have the binary, not necessarily the source. It's fairly easy to capture the information. Processing it is another story, however.

This article will give you an introduction in how to use strace, a commonly used debugging tool for system administrators, as a tool to gather information on the IO pattern of your code(s). I'll show you how to use and gather the information, including how you use it for MPI codes. Read on if you want to learn more about how to use this simple tool to make yourself a better cluster, well..., monkey.

Read more: Using Strace to Examine Your IO

0 Comments

So Why Use Disks on Clusters?

Published on Wednesday, 24 May 2006 21:00

Written by Jeff Layton

Hits: 9860

Spinning platters is what matters

A convergence of several key technologies has begun to change some of the old assumptions about how to build clusters. These technologies are high-speed affordable interconnects, high-speed file systems and storage network protocols, MPI implementations with MPI-IO, and high performance processors. This nexus of technologies is allowing the next possible steps in cluster evolution to become mainstream - diskless nodes.

Read more: So Why Use Disks on Clusters?

0 Comments