Introduction to Kubernetes Operators, Operator Framework, and Operators SDK
This blog post is part of the series How to leverage Kubernetes operators using the Operator SDK framework.
Section 1 – Kubernetes Operators, Operator Framework, and Operators SDK:
- Here we discuss in a general setting about Operators, Operator Framework, and Operators SDK.
- Then we will discuss about the Operators SDK emerging popularity in GitHub, and in general about the “Operator SDK workflow” adopted for generating and handling operators.
Section 1 – Kubernetes Operators, Operator Framework, and Operators SDK
a) Operators are Kubernetes applications
A Kubernetes application is an application that is both deployed on Kubernetes and managed using the Kubernetes APIs and kubectl tooling. To be able to make the most of Kubernetes, you need a set of cohesive APIs to extend in order to service and manage your applications that run on Kubernetes. You can think of Operators as the “runtime that manages this type of application on Kubernetes“.
Thus, an Operator is a method of packaging, deploying and managing a Kubernetes application. Conceptually, an Operator takes human operational knowledge and encodes it into software that is more easily packaged and shared with consumers. We can think of an Operator as an extension of the software vendor’s engineering team that watches over your Kubernetes environment and uses its current state to make decisions in milliseconds. Operators follow a [maturity model] from basic to having specific logic for an application.
We’ve seen in the last years that Operators’ capabilities differ in sophistication depending on how much intelligence has been added into the implementation logic of the Operator itself. We’ve also learned that the creation of an Operator typically starts by automating an application’s installation and self-service provisioning capabilities, and then evolves to take on more complex automation, this depending on the specific use case. As result, advanced operators are, nowadays, designed to handle upgrades seamlessly, react to failures automatically, and not take shortcuts, like skipping a software backup process to save time.
b) Operator Framework
Operators are Kubernetes native applications that facilitate the management of complex stateful applications on top of Kubernetes, however writing such operators can be very difficult because of challenges such as using (i) low level APIs and (ii) a lack of modularity which leads to duplication, inconsistencies, and unexpected behaviors.
To address such issue, several tools are now being launched (e.g., the Operator Framework, Kooper, Metacontroller, etc.) as results of years of work and experience of the Red Hat, Kubernetes, and CoreOS open source communities in building Operators. Specifically, Red Hat and the Kubernetes open source community shared the Operator Framework — an open source toolkit designed to manage operators in a more effective, automated, and scalable way.
The Operator Framework is an open source toolkit composed by several low-level APIs. We believe that the new Operator Framework represents the next big step for Kubernetes by using a baseline of leading practices to help lower the application development barrier on Kubernetes. The project delivers a software development kit (SDK) and the ability to manage app installs and updates by using the lifecycle management mechanism, while enabling administrators to exercise operator capabilities on any Kubernetes cluster.
The Operator Framework includes:
- Operator SDK: Enables developers to build Operators based on their expertise without requiring knowledge of Kubernetes API complexities.
- Operator Lifecycle Management: Oversees installation, updates, and management of the lifecycle of all of the Operators (and their associated services) running across a Kubernetes cluster. Once built, Operators need to be deployed on a Kubernetes cluster. The Operator Lifecycle Manager is the backplane that facilitates management of operators on a Kubernetes cluster. With it, administrators can control what Operators are available in what namespaces and who can interact with running Operators. They can also manage the overall lifecycle of Operators and their resources, such as triggering updates to both an Operator and its resources.
- Operator Metering (joining in the coming months): Enables usage reporting for Operators that provide specialized services. In a future version, the Operator Framework will also include the ability to meter application usage – a Kubernetes first, which provides extensions for central IT teams to budget and for software vendors providing commercial software. Operator Metering is designed to tie into the cluster’s CPU and memory reporting, as well as calculate IaaS cost and customized metrics like licensing.
Simple, stateless applications can leverage the Lifecycle Management features of the Operator Framework without writing any code by using a generic Operator (for example, the Helm Operator). However, complex and stateful applications are where an Operator can shine. The cloud-like capabilities that are encoded into the Operator code can provide an advanced user experience, automating such features as updates, backups and scaling.
In the next subsection we discuss about the Operators SDK emerging popularity in GitHub, and in general about the “Operator SDK workflow” adopted for generating and handling operators.
c) Operators SDK popularity
The Operator-SDK is a toolkit, recently built on top of the Operator Framework, that provides the tools to build, test and package Operators. Initially, the SDK facilitated the marriage of an application’s business logic (for example, how to scale, upgrade, or backup) with the Kubernetes API to execute those operations. However, over time, the SDK is evolving to allow engineers to make applications smarter and have the user experience of cloud services. As consequence, leading practices and code patterns that are shared across Operators are included in the SDK to help prevent reinventing the wheel.
From a developer perspective, the entry point is the Operator SDK, originating from CoreOS, which is offered as part of the Operator Framework that is, according to its self-description, “an open source toolkit to manage Kubernetes native applications, called Operators, in an effective, automated, and scalable way”. The SDK specifically targets Go developers and applications, and even if support for other programming languages (e.g., Java, C, etc.) is currently lacking, future plans for their integration are already in place.
In GitHub, the Operator SDK is becoming a very active project, which already gained a high visibility/popularity with:
- over 200 developers that forked its contributions on GitHub, most of them playing/extending some simple examples of operators;
- around 1000 commits done since the first release (5 Apr 2018).
- over 10 releases produced in less than a year.
- an increasing number of GitHub issues opened by other contributors in recent months
- [some of them were opened also by us, see https://github.com/operator-framework/operator-sdk/issues/651 and https://github.com/operator-framework/operator-sdk/issues/927]
- over 1200 stars it is one of the most popular/trending Go projects on Github https://github.com/trending/go?since=monthly
However, even if the project is becoming with the time more popular, its project Status is still “pre-alpha“, which means that “are expected breaking changes to the API in the upcoming releases“.
Thus, the Operator SDK toolkit requires still a bit more of maturity to be used in wider practical working scenarios. As researchers, we believe that this software development kit (SDK) will be widely adopted in future, as it will support the developers during the management of app installs and updates by using the lifecycle management mechanism, while enabling administrators to exercise operator capabilities on any Kubernetes cluster (see the following Figure, it highlights the overall view of envisioned Operator SDK support).
As follow we talk about the Operators SDK General Workflow.
d) Operators SDK General Workflow
The Operator-SDK is a toolkit that provides the tools to build, test and packageOperators, as shown in the following Figure.
Specifically, the following specific workflow is provided by the toolkit for supporting the writing, building, testing and packaging of a new Go operator:
- Create a new operator project using the SDK Command Line Interface (CLI)
- Define new resource APIs by adding Custom Resource Definitions (CRD)
- Define Controllers to watch and reconcile resources
- Write the reconciling logic for your Controller using the SDK and controller-runtime APIs
- Use the SDK CLI to build and generate the operator deployment manifests
In this context, the Operator SDK uses for its workflow the controller-runtime library, which makes the writing of operators easier by providing:
- High level APIs and abstractions to write the operational logic more intuitively.
- Tools for scaffolding and code generation to bootstrap a new project fast.
- Extensions to cover common operator use cases.
A simple example to create and deploy a simple operator with the SDK toolkit is provided in the official operator SDK GitHub repository:
The resulting automatically generated GO operator will present the following reference Structure:
Contains the directory tree that defines the APIs of the Custom Resource Definitions (CRD).
This pkg contains the controller implementations.
Contains the Dockerfile and build scripts used to build the operator.
Contains various YAML manifests for registering CRDs, setting up RBAC, and deploying the operator as a Deployment.
The Go Dep manifests that describe the external dependencies of this operator.
In the next blog post we will talk about the Operators SDK current status, e.g., available versions and workflows.