▷ the Bright blog

Auto-scaling HPC and Kubernetes

Bright CEO Bill Wagner’s recent blog post “The Convergence of HPC and A.I.” pointed out that creating HPC and Kubernetes silos within a shared HPC infrastructure is more like “coexistence than convergence”. The solution I will be highlighting  is Bright Auto-scaler, which automatically resizes HPC and Kubernetes clusters (workload engines) according to workload demand and configured policies. This post describes how to configure Bright Auto-scaler to achieve true convergence of HPC and A.I. through a scenario.

The convergence of HPC & A.I. is a hot topic right now, and for good reasons.  In practice, there is a symbiotic relationship between (for example) HPC simulations and machine learning.  HPC simulations generate tons of data, and as luck would have it, machine learning requires tons of data to train models.  If we turn the value proposition around, the quality and effectiveness of HPC simulations can be improved if we use machine learning to identify the parameters in simulation data that have a significant effect on the outcome and focus subsequent iterations of simulation on those parameters. 

Bright has been working collaboratively with one of our valued US Technology Consulting partners based out of the northeast, for a number of years, so it was a real privilege to be invited to give a presentation at their annual Sales Kick-Off, last week. Together, Bright and our consulting partner offer advanced infrastructure management solutions to organizations across North America who are looking to harness the power of their HPC, deep learning, AI, and OpenStack.

With SC 18 in Dallas just a few days away, I thought I’d write a brief blog post to give our community an update on how Dell EMC and Bright are collaborating at the supercomputing industry’s largest event of the year.

I’d like to draw your attention to a webinar taking place tomorrow, Wednesday October 31^st, at 11am CDT. This is a must-attend webinar for anyone who is exploring how to deploy and manage an Artificial Intelligence (AI) environment. 

This blog post was originally published on the HPE community blog - Servers: The Right Compute.