About the APPUiO Cloud API

In a previous blog post we talked about how we handle billing for APPUiO Cloud. This time we’re going to talk about how we have extended the Kubernetes API to provide a custom API for our product.

Just like in the aforementioned blog post, the links in this text point to pages in the APPUiO Cloud for System Engineers documentation, with more technical details.

As a reminder, APPUiO Cloud consists of several independent OpenShift 4 clusters called zones. They can be individually operated upon through the Kubernetes API and Console.

Being a shared platform, a precondition for users to use APPUiO Cloud is to belong to an organization, that is, the entity we can send invoices to. This organization must be the same for all zones; which means that users get a single invoice, no matter which zone their applications are running in.

Managing organizations individually on each zone would be tricky, and would quickly lead to issues of bi-directional synchronization; this is commonly known as a hard problem in IT, and of course we would rather like to avoid it. And this problem would get worse in the future, every time we open a new APPUiO Cloud zone; so, thanks but no thanks.

We needed a tool to manage organizations, independent from (yet intimately related to) any and each APPUiO Cloud zones. And since we are strong believers of both „API First“ and „Kubernetes First“, we decided to extend Kubernetes‘ own API to do this.

We created Custom Resource Definitions (CRDs) defining organizations and teams, placing them at the core of our APPUiO Cloud API. This gave us access control features for free, and as a huge bonus, we got a free client to talk to said API: kubectl itself (as well as its OpenShift counterpart, oc!)

This single design decision made the APPUiO Cloud API compatible with the larger Kubernetes ecosystem. We used this API to build a whole web application on top of it, and this turned out to be quite easy, largely thanks to said ecosystem.

For example, based on Kubernetes‘ own objects, rules, and groups, we can make the user interface of our web application reactive to the permissions of the current user; this is entirely handled by the API thanks to RBAC. Which means, no need to touch the UI code when permissions change!

The important thing to keep in mind is that the CRDs above are only used to represent data stored in a separate system; we developed adapters (in the shape of native Kubernetes controllers and operators) in charge of synchronizing the organizations and teams data between the API and the systems where those objects are stored.

Specifically, organizations and teams are groups and subgroups stored in a Keycloak instance; but this could be replaced by any other identity provider (IdP) in the future. This means that the identity data model of APPUiO Cloud is completely independent of the underlying store used at any time.

As it is generaly known from Kubernetes, the spec fields in those Custom Resources represent the desired state to be synced to the connected system; status fields represent the actual state on the connected system.

Thanks to the pre-existence of CRDs, the development of the UI and the adapters thereof could progress independently from one another, working as a common de facto contract between different parts of APPUiO Cloud. Even better, during development the behavior of the adapters could be stubbed, by directly manipulating the custom resources.

And to thank you for reading until the end, here’s a bonus detail: we’re using vcluster in our quality management process. What is vcluster? It’s a way to create virtual Kubernetes clusters in another cluster. It allows us to run multiple instance of the API on one Kubernetes or OpenShift cluster, without risking to run into CRD version conflicts or other issues. We use this approach to quickly spin integration and testing environments through our test automation procedures (driven by GitHub Actions), including feature branch testing. This single tool has boosted our productivity tremendously.