THE BRIGHT BLOG

Auto-scaling HPC and Kubernetes

Bright CEO Bill Wagner’s recent blog post “The Convergence of HPC and A.I.” pointed out that creating HPC and Kubernetes silos within a shared HPC infrastructure is more like “coexistence than convergence”. The solution I will be highlighting  is Bright Auto-scaler, which automatically resizes HPC and Kubernetes clusters (workload engines) according to workload demand and configured policies. This post describes how to configure Bright Auto-scaler to achieve true convergence of HPC and A.I. through a scenario.

The convergence of HPC & A.I. is a hot topic right now, and for good reasons.  In practice, there is a symbiotic relationship between (for example) HPC simulations and machine learning.  HPC simulations generate tons of data, and as luck would have it, machine learning requires tons of data to train models.  If we turn the value proposition around, the quality and effectiveness of HPC simulations can be improved if we use machine learning to identify the parameters in simulation data that have a significant effect on the outcome and focus subsequent iterations of simulation on those parameters. 

Bright has been working collaboratively with one of our valued US Technology Consulting partners based out of the northeast, for a number of years, so it was a real privilege to be invited to give a presentation at their annual Sales Kick-Off, last week. Together, Bright and our consulting partner offer advanced infrastructure management solutions to organizations across North America who are looking to harness the power of their HPC, deep learning, AI, and OpenStack.

With SC 18 in Dallas just a few days away, I thought I’d write a brief blog post to give our community an update on how Dell EMC and Bright are collaborating at the supercomputing industry’s largest event of the year.

I’d like to draw your attention to a webinar taking place tomorrow, Wednesday October 31^st, at 11am CDT. This is a must-attend webinar for anyone who is exploring how to deploy and manage an Artificial Intelligence (AI) environment. 

Big Picture Efficiency

Efficiency as a business concept approaches an almost revered status for IT admins and bean counters alike. In an HPC (High Performance Computing) environment, cluster performance for discrete tasks represents the majority of actual work done efficiency, but most major HPC cluster providers remain fairly price competitive in the pay for processing market. 

This will be my first trip to ISC, and as an American that resided in Germany for two decades, one that I am very much looking forward to.

Hot on the heels of the Dell EMC HPC Community Meeting which took place earlier this week, we’re gearing up for NVIDIA GTC, March 26-29 in San Jose, and I’m pleased to say that Bright will be present on the Dell EMC booth at the event.

This blog post has been written by Govnet Events, the organisers of UK HPC & Big Data conference.

Alison Kennedy is the Director of the STFC Hartree Centre, based at the UK’s Science and Facilities Council’s National Laboratory. The STFC’s Hartree Centre is a high performance computing (HPC) and data analytics research facility which was founded by the government in collaboration with IBM, leaders of information technology.

This blog post has been written by Govnet Events, the organisers of UK HPC & Big Data conference.

###

Delivered by the UK's leading speakers within the technology sector, the HPC & Big Data conference presents a tightly packed agenda exploring the key areas of computational research throughout the public and private sector.