Using Bright Cluster Manager to leverage Cloud Resources


By Bill Harries | May 07, 2020 |



I just finished reading an article on The Next Platform – “The Outlook for Infrastructure is Cloudy – in a Good Way”, and feel compelled to comment, especially given my previous post on High Performance Clusters in the Cloud?

In that post, I made the case that although there are all kinds of reasons for not using the cloud for HPC, it is, however, relatively easy to do.  Only very extraordinary circumstances would warrant a deep investment in cloud resources to address your current HPC requirements.  Well, it seems we are in the midst of the most extraordinary of circumstances in what they refer to as “the Great Infection”.

The article makes the following key points (I’m paraphrasing here, but hopefully I've nailed the spirit of their points):

  • Supply chain issues for hardware will be impacted through at least Q2, but IT budgets will be tightened throughout the year.
  • Hyperscalers and public clouds have not taken over the world, and when all this is over, HPC users are unlikely to continue using cloud resources because of the price premium for cloud resources.
  • Paradoxically though, cloud IT infrastructure spending will increase over the next 4 years, whereas non-Cloud IT infrastructure will decline.

What to make of all this?  I suspect HPC administrators are going to be pressed to do more with less… more than ever before.  CapEx budgets will be squeezed, while workload demands on their HPC systems will steadily increase while periodically fluctuating. 

Conveniently, public cloud infrastructure is ideally suited for addressing these extraordinary times.  Additional compute resources are available without the hassle of getting capital expenditures allocated and approved – cloud charges are typically operating expenses.  You can get access to additional HPC resources instantly, and you only pay for those resources as you use them.  Now, admittedly, you pay a premium for that kind of “convenience,” but in these times, it may be your only choice.

With all that said, I refer you back to my previous post where I explain how Bright clusters make it easy to expand and contract cluster resources in the cloud, and how all of these on-premise and cloud resources are managed as a single cluster – eliminating the pain of administering such diverse resources across diverse workloads.  In the ideal world, you would simply expand your on-premise footprint…but, in as understated a way as I can possibly say it, these are not ideal times.