Blue Collar Computing: HPC for the Rest of Us

Article Index

Blue Collar™ High Performance Computing

Economic forces will continue to shape High Performance Computing (HPC). It is clear that the US has reached a critical juncture with regard to HPC, and the central challenge will be to sustain sources of funding so that our leadership position in HPC is not diminished. However, another view of this challenge might be more illuminating: can HPC be realistically viewed as one of the critical economic drivers for our future? Is this view of HPC realistic or even realizable? And if HPC is one of a relatively small number of critical economic differentiators, what level of national investment in HPC is justified by its economic potential?

Blue Collar Computing is high performance computing for industries that do not currently have the expertise or the time to be an HPC incubator or research new HPC applications. High performance programming languages, training, and collaborations are required to open up greater capability and competitiveness to business, science, and engineering users. Blue Collar Computing scales up the number of processors beyond the one or two CPU boxes that companies usually run. The focus should be on high-productivity languages, industry and supercomputer lab collaborations, and the training needed to provide the expertise needed to allow the greater capability and efficiency of HPC to be utilized by business users.

National labs that do large-scale "heroic computing" -- the Departments of Energy and Defense (DOE and DoD) and the National Science Foundation (NSF) -- will continue to concentrate on monumental, "grand challenge" problems. However, if new resources are focused on industry HPC requirements, everyone would benefit in the long run. Blue Collar Computing is the computing needed in order to sustain the nation’s aspirations, and the computing needed to ensure that the US will remain an economic and scientific leader.

Looking at Figure One, we notionally describe the current HPC market as follows. We currently have relatively few users that utilize the nation’s most powerful HPC installations. And on the lower end of the spectrum, use of more than one- to two- processors is relatively rare. Programmer productivity is maximized when only a few processors need to be harnessed, but a large number of applications are available for those few processors. The need for more processors has not been demonstrated, since the vast majority of average users are generally happy with using only one processor, since it offers immediate ROI at a minimal cost.

The current HPC market composed of easy pickings and heroic computing
Figure One: The current HPC market composed of easy pickings and heroic computing

Looking at Figure Two, Blue Collar computing demands from industry may allow us to push the mid-range of the market to a higher level--especially in the auto, petroleum, financial, and pharmaceutical industries. Collectively, there could be as much -- or more -- need for HPC in industry as now found in the DoD, NSF, and DOE. We have the heroes -- DoD, NSF, DOE -- still using HPC to take research and supercomputing to new heights. Yet industry could partake in the benefits of increased HPC investment, processing power, and compatible software.

The ideal HPC market
Figure Two: The ideal HPC market

The future of industrial competitiveness must move beyond the one- or two-processor model if we are to visualize what is shown in Figure Three. Industry will be working on bigger and better solutions using computing languages and employees educated to fit in this new economy. In an ideal market for industrial HPC, training and education is a major component -- even chemists, engineers, and other scientists who use research labs and HPC centers need training in computer and computational science. This environment can be a true economic generator, moving away from traditional manufacturing to real "knowledge economy" applications.

Blue Collar Computing
Figure Three: Blue Collar Computing

Ultimately, the entire HPC market will grow by enabling industry to solve problems and develop better products more quickly. In the next decade, this is where we will see dramatic gains: increased productivity gains in industry and engineering and increased gains in scientific discovery from those hero HPC applications that will always push the envelope.

Blue Collar HPC Applications

If we can expand the HPC market – change the shape of the market to reflect a greater use of Blue Collar Computing – we can guarantee that parallel processing, clusters, and all of the associated software is made available to industry. However, the computing market today is like the automobile market of the early 1900s. We produce a huge number of very reliable, very useful Model Ts and relatively few high-end Duesdnbergs. But – if we can follow the lead of the automotive market and focused on increasing the functionality of HPC by providing standardized interfaces and highly reliable software, we may be able to transform the HPC market so that it feeds a constantly improving cycle of innovation. Once the needed tools are developed and software interfaces are made more accessible to today’s domain scientists and engineers – the move to significant industrial use of HPC may take hold.

Sidebar Two: HPC Software: A Shared Approach

The science and engineering research community has been able to take advantage of HPC not only because of the hardware that is available, but also because there is a pool of software libraries and tools that can be applied to scientific calculations. Available HPC software includes partial differential equation solvers, grid decomposition utilities, fast approximate string search libraries, and much more. The success of HPC developers hinges on the ability to leverage existing code in creating new applications to solve their particular problems.

The contributions made by desktop and server software are also extremely important. These include the Linux kernel, the GNU C library and compilers, all the Unix utilities provided by the Free Software Foundation and many others. Without this code base it would be much more difficult to do the necessary daily tasks we all now take for granted to administer HPC machines.

As the open source model of software development has proved itself to be a cornerstone of high-performance scientific computing, we are eager to see how a dedicated effort to bring open source software development to industrial HPC computing might revitalize and transform that segment as it has done for science and engineering.

How do we create the tools? First of all, we need a public/private partnership to work on sustaining interfaces and software tools. Second, individual corporate entities may have the capability to solve problems on one or two processors, but they may not be able to spend the funds required to look at more advanced applications that will utilize many more CPUs. However, companies might be convinced to invest funds in long-term HPC research if there is a concomitant matching investment by government.

In order to use these applications, we must remove the industrial barriers we have discussed above. The result will be an ability to create better products, help our innovators to "think faster," and actuate the next long-lived productivity expansion. The heroes and the rest of the HPC community will also reap the benefits.

SUMMARY

We have argued that a fundamental shift in the HPC market -- a shift to Blue Collar computing—needs to take place in order to revitalize U.S innovation. We are at the beginning of a new phase in the evolution of computing, and we’ve only begun the journey. We have proposed some general solutions that involve high productivity languages and education; however, a focused attempt to solve specific industrial HPC problems and barriers is vitally needed.

We propose that the HPC community start working on a public/private partnership to develop the elements of Blue Collar Computing that are most pressing. Next steps involve focusing on possible implementation plans for high productivity languages and radically changing computer science education. There is a need for both undergraduate and graduate personnel with computational science expertise, which takes extraordinary level of institutional cooperation. One possibility could be to allow companies to provide curricular material to stimulate teaching in the direction of solving current and future industrial "grand challenges."

Ultimately, the first question that needs to be asked is if are we ready for a shift to Blue Collar Computing. Can a commitment be made to embrace it? A transition to Blue Collar Computing will naturally break down many of the current barriers to entry, and provide the nation with a vitally important economic edge. This shift will allow the United States to maintain its economic leadership in the global marketplace. But, it requires a paradigm change in our way of thinking, our way of teaching, and our way of approaching HPC.

{mosgoogle right}

Sidebar Three: Resources

Ohio Supercomputer Center Blue Collar Computing

Council on Competitiveness HPC Users Advisory Group

International Data Corporation

This article was originally published in ClusterWorld Magazine. It has been updated and formatted for the web. If you want to read more about HPC clusters and Linux, you may wish to visit Linux Magazine.

Stanley C. Ahalt is Executive Director of the Ohio Supercomputer Center and Kathryn L. Kelley is Director of Government and Community Relations please contact them through The Ohio Supercomputer Center.

    Search

    Feedburner

    Login Form

    Share The Bananas


    Creative Commons License
    ©2005-2012 Copyright Seagrove LLC, Some rights reserved. Except where otherwise noted, this site is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License. The Cluster Monkey Logo and Monkey Character are Trademarks of Seagrove LLC.