(This post was originally published on the Crossplane blog.)

Crossplane has recently celebrated its fifth birthday, but at VSHN, we’ve been using it in production for almost three years now. In particular, it has become a crucial component of one of our most popular products. We’ve invested a lot of time and effort on Crossplane, to the point that we’ve developed (and open-sourced) our own custom modules for various technologies and cloud providers, such as Exoscale, cloudscale.ch, or MinIO.

In this blog post, we will provide an introduction to a relatively new feature of Crossplane called Composition Functions, and show how the VSHN team uses it in a very specific product: the VSHN Application Catalog, also known as VSHN AppCat.

Crossplane Compositions

To understand Composition Functions, we need to understand what standard Crossplane Compositions are in the first place. Compositions, available in Crossplane since version 0.10.0, can be understood as templates that can be applied to Kubernetes clusters to modify their configuration. What sets them apart from other template technologies (such as Kustomize, OpenShift Template objects, or Helm charts) is their capacity to perform complex transformations, patch fields on Kubernetes manifests, following more advanced rules and with better reusability and maintainability. Crossplane compositions are usually referred to as „Patch and Transform“ compositions, or „PnT“ for short.

As powerful as standard Crossplane Compositions are, they have some limitations, which can be summarized in a very geeky yet technically appropriate phrase: they are not „Turing-complete“.

• Compositions don’t support conditions, meaning that the transformations they provide are applied on an „all or nothing“ basis.
• They also don’t support loops, which means that you cannot apply transformations iteratively.
• Finally, advanced operations are not supported either, like checking for statuses in other systems, or performing dynamic data lookups at runtime.

To address these limitations, Crossplane 1.11 introduced a new Alpha feature called „Composition Functions“. Note that as of writing, Composition Functions are in Beta in 1.14.

Composition Functions

Composition functions complement and in some cases replace Crossplane „PnT“ Compositions entirely. Most importantly, DevOps engineers can create Composition Functions using any programming language; this is because they run as standard OCI containers, following a specific set of interface requirements. The result of applying a Composition Function is a new composite resource applied to a Kubernetes cluster.

Let’s look at an elementary „Hello World“ example of a Composition Function.

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: example-bucket-function
spec:
  compositeTypeRef:
    apiVersion: example.crossplane.io/v1
    kind: XBucket
  mode: Pipeline
  pipeline:
  - step: handle-xbucket-xr
    functionRef:
      name: function-xr-xbucket

The example above shows a standard Crossplane composition with a new field: „pipeline“ specifying an array of functions, referred to via their name.

As stated previously, the function itself can be written in any programming language, like Go.

func (f *Function) RunFunction(_ context.Context, req *fnv1beta1.RunFunctionRequest) (*fnv1beta1.RunFunctionResponse, error) {
    rsp := response.To(req, response.DefaultTTL)
    response.Normal(rsp, "Hello world!")
    return rsp, nil
}

The example above, borrowed from the official documentation, does just one thing: it reads a request object, modifies a value, and returns it to the caller. Needless to say, this example is for illustration purposes only, lacking error checking, logging, security, and more, and should not be used in production. Developers use the Crossplane CLI to create, test, build, and push functions.

Here are a few things to keep in mind when working with Composition Functions:

• They run in order, as specified in the „pipeline“ array of the Composition object, from top to bottom.
• The output of the previous Composition Function is used as input for the following one.
• They can be combined with standard „PnT“ compositions by using the function-patch-and-transform function, allowing you to reuse your previous investment in standard Crossplane compositions.
• In the Alpha release, if you combined „PnT“ compositions with Composition Functions, „PnT“ compositions ran first, and the output of the last one is fed to the first function; since the latest release, this is no longer the case, and „PnT“ compositions can now run at any step of the pipeline.
• Composition Functions must be called using RunFunctionRequest objects, and return RunFunctionResponse objects.
• In the Alpha release, these two objects were represented by a now deprecated „FunctionIO“ structure in YAML format.
• RunFunctionRequest and RunFunctionResponse objects contain a full and coherent „desired state“ for your resources. This means that if an object is not explicitly specified in a request payload, it will be deleted. Developers must pass the full desired state of their resources at every invocation.

Practical Example: VSHN AppCat

Let’s look at a real-world use case for Crossplane Composition Functions: the VSHN Application Catalog, also known as AppCat. The AppCat is an application marketplace allowing DevOps engineers to self-provision different kinds of middleware products, such as databases, message queues, or object storage buckets, in various cloud providers. These products are managed by VSHN, which frees application developers from a non-negligible burden of maintenance and oversight.

Standard Crossplane „PnT“ Compositions proved limited very early in the development of VSHN AppCat, so we started using Composition Functions as soon as they became available. They have allowed us to do the following:

• Composition Functions enable complex tasks, involving the verification of current deployment values and taking decisions before deploying services.
• They can drive the deployment of services involving Helm charts, modifying values on-the-go as required by our customers, their selected cloud provider, and other parameters.
• Conditionals allow us to script complex scenarios, involving various environmental decisions, and to reuse that knowledge.
• Thanks to Composition Functions, the VSHN team can generalize many activities, like backup handling, automated maintenance, etc.

All things considered, it is difficult to overstate the many benefits that Composition Functions have brought to our workflow and to our VSHN AppCat product.

Learnings of the Alpha Version

We’ve learned a lot while experimenting with the Alpha version of Composition Functions, and we’ve documented our findings for everyone to learn from our mistakes.

• Running Composition Functions in Red Hat OpenShift used to be impossible in Alpha because OpenShift uses crun, but this issue has now been solved in the Beta release.
• In particular, when using the Alpha version of Composition Functions, we experienced slow execution speeds with crun but this is no longer the case.
• We learned the hard way that missing resources on function requests were actually deleted!

Our experience with Composition Functions led us to build our own function runner. This feature uses another capability of Crossplane, which allows functions to specify their runner in the Composition definition:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
[...]
  functions:
  - name: my-function
    type: Container
    container:
      image: my-function
      runner:
        endpoint: grpc-server:9547

Functions run directly on the gRPC server, which, for security reasons, must run as a sidecar to the Crossplane pod. Just like everything we do at VSHN, the Composition Function gRPC Server runner (as well as its associated webhook and all of its code) is open-source, and you can find it on our GitHub. As of the composition function beta, we replaced the custom GRPC logic with the go-sdk. To improve the developer experience, we have created a proxy and enabled the gRPC server to run locally. The proxy runs in kind and redirects to the local gRPC server. This enables us to debug the code and to test changes more efficiently.

Moving to Beta

We recently finished migrating our infrastructure to the most recent Beta version of Composition Functions, released in Crossplane 1.14, and we have been able to do that without incidents. This release included various bits and pieces such as Function Pipelines, an ad-hoc gRPC server to execute functions in memory, and a Function CRD to deploy them directly to clusters.

We are also migrating all of our standard „PnT“ Crossplane Compositions to pure Composition Functions as we speak, thanks to the functions-go-sdk project, which has proven very helpful, even if we are missing typed objects. Managing the same objects with the „PnT“ and Composition Functions increases complexity dramatically. As it can be difficult to determine where an actual change happens.

Conclusion

In this blog post, we have seen how Crossplane Composition Functions compare to standard „PnT“ Crossplane compositions. We have provided a short example, highlighting their major characteristics and caveats, and we have outlined a real-world use case for them, specifically VSHN’s Application Catalog (or AppCat) product.

Crossplane Composition Functions provide an unprecedented level of flexibility and power to DevOps engineers. They enable the creation of complex transformations, with all the advantages of an Infrastructure as Code approach, and the flexibility of using the preferred programming language of each team.

Check out my talk at Control Plane Day with Crossplane, where I walk you through Composition Functions in Production in 15 minutes.

On the evening of August 25th, 1991, Linus Torvalds wrote a short message on the comp.os.minix Usenet newsgroup:

I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) for 386(486) AT clones.

Fast-forward 30 years, and to make a long story short, well: Linux is now both big and professional. It’s nearly everywhere, hidden behind the shiny graphical user interface of your smartphone or tablet (well, except for iOS users), in your gaming box, in your car, and even on Mars.

I discovered Linux as a young boy still at school, in 1998. It was in a book store where I was attracted by a shiny package (I had no idea what Linux was) called „SuSE Linux 6.0“ (nowadays available for download from archive.org), and since then I couldn’t stop working with it.

Over time, I got more and more into Linux, and it became an important hobby, with all my PCs running Linux, and also on my home server under my desk at home. Many years later, in 2011, I could finally start working full-time as a Linux administrator. My desktop computer has been only powered by Linux since around 2004, and I’ve been a big fan of KDE ever since.

Today, the Linux Kernel is more present, yet less visible than ever. We interact with it on containers and in VMs on the cloud, and it gets more and more abstracted away, deep down in serverless architectures, making the Kernel even more invisible than ever before. Albeit out of sight of most users, Linux has become much more solid, mature, and pervasive, and does its great job behind the scenes without interruption.

Linux empowers VSHN at all levels; not only it is the dominating technology we use every day, empowering Kubernetes, containers, and cloud VMs, but it is also the operating system that the majority of VSHNeers (around 66%, according to an internal poll) use for their daily job, the remaining third using macOS or Windows.

Some numbers: of those two thirds of VSHNeers that use Linux every day in their laptops, 61% chose Ubuntu (or one of its various derivatives); 17% use Arch, 11% Fedora, and others use Debian, Mint, and other distributions. Some even contemplate switching to Qubes OS soon! As for desktop environments, around 35% use GNOME, 25% use KDE, 20% use i3, and 6% use Cinnamon.

Each one of us VSHNeers has a unique feeling about Linux; here are some thoughts about what Linux means to us:

Before using Linux, I was primarily focused on how to use and work with computer systems. With the switch to Linux I started to understand how they actually work.

What I really appreciate about Linux is that it’s (still relatively) lightweight, powerful, transparent and adaptable. I do heavyweight gaming and video livestreaming on the same OS that runs my file servers and backup systems (not all on the same machine, don’t worry). Even my car and my television run Linux! This universality combined with the permissive licenses means that whenever one industry improves Linux (the kernel), every other industry profits.

I originally just wanted to play Minecraft with my friends. Suddenly I had to learn how to host this on a Linux server, which then turned into a fascination on how things work on the backstage. It’s the urge to see how our modern society works!

Linux is the operating system of our software-driven world.

On to the next 30 years of Linux!