We’re always busy at Bright HQ in Amsterdam, making sure our products are the best that they can be. A portion of my time at Bright is spent challenging the resilience, scope and power of Bright OpenStack, to discover and address any limitations; continually innovating towards a better product.
As part of this exercise, we recently spun up a virtualized 500-node OpenStack/Ceph private cloud inside our existing 13-node physical OpenStack private cloud. I’m pleased to say we did this in a single afternoon. In this post I explore some of the techniques and features of Bright OpenStack which helped to achieve this.
Let’s start with some context. At Bright, we have our own internal OpenStack private cloud, dubbed “Krusty the Cloud”. Each developer at Bright has their own isolated environment in Krusty, with full control over their virtual clusters… and these virtual clusters are directly accessible from our office network. It’s a great DevOps environment, you can read more about it here.
We recently upgraded Krusty from OpenStack Icehouse to Juno, adding 10GigE Ceph network fabric for Ceph data replication/recovery. We installed some Intel DC series SSDs for Ceph Journals, and updated to Bright OpenStack 7.1.
As part of the validation, verification and testing process, we challenged our OpenStack environment by spinning up a 500-VM strong, virtualized OpenStack cloud.
First, we created a virtualized Bright OpenStack deployment, comprising 500 Bright OpenStack nodes (mostly nova-compute), virtualized inside of a physical Bright OpenStack deployment composed of a modest number of only 13 physical hypervisor nodes. Next, we filled that virtual OpenStack deployment with large amounts OpenStack objects (thousands of users, tenants, networks, ports), to see how everything performed at scale inside of relatively slow, aggressively overcommitted, virtual environment.
We weren’t disappointed. We carried out the entire process in a single afternoon, from scratch.
Here’s a screenshot of the Admin management interface of our physical OpenStack cloud:
And now the Admin management interface for our virtualized OpenStack cloud:
I have a lot of data on the process and detailed results, for those who are interested.
I have to say, that we are happy with the results. We put Bright OpenStack under heavy duress, caused by all the Ceph/OpenStack services inside of the virtualized OpenStack deployment, and it performed well. We continue to innovate.