Upgrade to OpenShift version 4.16

As we start to prepare the upgrade to OpenShift v4.16 for all our customers clusters, it is a good opportunity to look again at what’s new in the Red Hat OpenShift 4.16 release. The release is based on Kubernetes 1.29 and CRI-O 1.29 and brings a handful of exciting new features which will make VSHN Managed OpenShift even more robust. Additionally, the new release also deprecates some legacy features which may require changes in your applications.

The Red Hat infographic highlights some of the key changes:

Changes which may require user action across all VSHN Managed OpenShift, including APPUiO

For VSHN Managed OpenShift, we’re highlighting the following changes which may require user action in our Release notes summary

Clusters which use OpenShift SDN as the network plugin can’t be upgraded to OpenShift 4.17+

This doesn’t affect most of the VSHN Managed OpenShift clusters since we’ve switched to Cilium as the default network (CNI) plugin a while ago and most of our older managed clusters have been migrated from OpenShift SDN to Cilium over the last couple of months.

The proxy service for the cluster monitoring stack components is changed from OpenShift OAuth to kube-rbac-proxy

Users who use custom integrations with the monitoring stack (such as a Grafana instance which is connected to the OpenShift monitoring stack) may need to update the RBAC configuration for the integration. If necessary, we’ll reach out to individual VSHN Managed OpenShift customers once we know more.

The ingress controller HAProxy is updated to 2.8

HAProxy 2.8 provides multiple options to disallow insecure cryptography. OpenShift 4.16 enables the option which disallows SHA-1 certificates for the ingress controller HAProxy. If you’re using Let’s Encrypt certificates for your applications no action is needed. If you’re using manually managed certificates for your Routes or Ingresses, you’ll need to ensure that you’re not using SHA-1 certificates.

Legacy service account API token secrets are no longer generated

In previous OpenShift releases, a legacy API token secret was created for each service account to enable access to the integrated OpenShift image registry. Starting with this release, these legacy API token secrets aren’t generated anymore. Instead, each service account’s image pull secret for the integrated image registry uses a bound service account token which is automatically refreshed before it expires.

If you’re using a service account token to access the OpenShift image registry from outside the cluster, you should create a long-lived token for the service account. See the Kubernetes documentation for details.

Linux control groups version 1 (cgroupv1) deprecated

The default cgroup version has been v2 for the last couple OpenShift releases. Starting from OpenShift 4.16, cgroup v1 is deprecated and it will be removed in a future release. The underlying reason for the pending removal is that Red Hat Enterprise Linux (RHEL) 10 and therefore also Red Hat CoreOS (RHCOS) 10 won’t support booting into cgroup v1 anymore.

If you’re running Java applications, we recommend that you make sure that you’re using a Java Runtime version which supports cgroup v2.

Warning for iptables usage

OpenShift 4.16 will generate warning event messages for pods which use the legacy IPTables kernel API, since the IPTables API will be removed in RHEL 10 and RHCOS 10.

If your software still uses IPTables, please make sure to update your software to use nftables or eBPF. If you are seeing these events for third-party software that isn’t managed by VSHN, please check with your vendor to ensure they will have an nftables or eBPF version available soon.

Other changes

Additionally, we’re highlighting the following changes:

RWOP with SELinux context mount is generally available

OpenShift 4.16 makes the ReadWriteOncePod access mode for PVs and PVCs generally available. In contrast to RWO where a PVC can be used by many pods on a single node, RWOP PVCs can only be used by a single pod on a single node. For CSI drivers which support RWOP, the SELinux context mount from the pod or container is used to mount the volume directly with the correct SELinux labels. This eliminates the need to recursively relabel the volume and can make pod startup significantly faster.

However, please note that VSHN Managed OpenShift doesn’t yet support the ReadWriteOncePod access mode on all supported infrastructure providers. Please reach out to us if you’re interested in this feature.

Monitoring stack replaces prometheus-adapter with metrics-server

OpenShift 4.16 removes prometheus-adapter and introduces metrics-server to provide the metrics.k8s.io API. This should reduce load on the cluster monitoring Prometheus stack.

Exciting upcoming features

We’re also excited about multiple upcoming features which aren’t yet generally available in OpenShift 4.16:

Node disruption policies

We’re looking forward to the “Node disruption policy” feature which will allow us to deploy some node-level configuration changes without node reboots. This should reduce the need for scheduling node-level changes to be rolled out during maintenance, and will enable us to say confidently whether a node-level change requires a reboot or not.

Route with externally managed certificates

OpenShift 4.16 introduces support for routes with externally managed certificates as a tech preview feature. We’re planning to evaluate this feature and make it available in VSHN Managed OpenShift once it reaches general availability.

This feature will allow users to request certificates with cert-manager (for example from Let’s Encrypt) and reference the cert-manager managed secret which contains the certificate directly in the Route instead of having to create an Ingress resource (that’s then translated to an OpenShift Route) which references the cert-manager certificate.

Changes not relevant to VSHN customers

There are a number of network related changes in this release, but these are not relevant for VSHN managed clusters as these are mostly running Cilium. In particular, OVNKubernetes gains support for AdminNetworkPolicy resources, which provide a mechanism to deploy cluster-wide network policies. Please note that similar results should be achievable with Cilium’s CiliumClusterWideNetworkPolicy resources, and Cilium is actively working on implementing support for AdminNetworkPolicy.

Summary

OpenShift 4.16 brings deprecates some features which may require changes to your applications in order to make future upgrades as smooth as possible. Additionally, OpenShift 4.16 is the last release that supports OpenShift SDN as the network plugin and disables support for SHA-1 certificates in the ingress controller. For those interested in the nitty gritty details of the OpenShift 4.16 release, we refer you to the detailed Red Hat release notes, which go through everything in detail.

VSHN customers will be notified about the upgrades to their specific clusters in the near future.

Interested in VSHN Managed OpenShift?

Head over to our product page VSHN Managed OpenShift to learn more about how VSHN can help you operate your own OpenShift cluster including setup, 24/7 operation, monitoring, backup and maintenance. Hosted in a public cloud of your choice or on-premises in your own data center. 

As we start to prepare to rollout upgrades to OpenShift v4.15 across all our customers clusters it is a good opportunity to look again at what was in the Red Hat OpenShift 4.15 release. It brought Kubernetes 1.28 and CRI-O 1.28 and it was largely focused on small improvements in the core platform and enhancements to how OpenShift runs on underlying infrastructure including bare-metal and public cloud providers.

The Red Hat infographic highlights some of the key changes:

For our VSHN Managed OpenShift and APPUiO customers, we want to highlight the key changes in the release that are relevant for them.

Across all VSHN Managed OpenShift clusters – including APPUiO

Our summary highlights that apply are the following:

• OpenShift 4.15 is based on Kubernetes 1.28 and CRI-O 1.28
• Update to CoreDNS 1.11.1
• There are some node enhancements (such faster builds for unprivileged pods, and compatibility of multiple image repository mirroring objects)
• The release also brings updated versions for the monitoring stack (Alertmanager to 0.26.0, kube-state-metrics to 2.10.1, node-exporter to 1.7.0, Prometheus to 2.48.0, Prometheus Adapter to 0.11.2, Prometheus Operator to 0.70.0, Thanos Querier to 0.32.5)
• It also includes some additional improvements and fixes to the monitoring stack
• There are some changes to the Bare-Metal Operator so that it now automatically powers off any host that is removed from the cluster
• There are some platform fixes including some security related ones like securing the cluster metrics port using TLS
• OLM (Operator Lifecycle Management is being introduced as v1 and this brings three new life cycle classifications for cluster operators that are being introduced: Platform Aligned, for operators whose maintenance streams align with the OpenShift version; Platform Agnostic, for operators who make use of maintenance streams, but they don’t need to align with the OpenShift version; and Rolling Stream, for operators which use a single stream of rolling updates.

On VSHN Managed OpenShift clusters with optional features enabled

The changes that might relate to some VSHN Managed OpenShift customers who have optional features enabled would include:

• OpenShift Service Mesh 2.5 based on Istio 1.18 and Kiali 1.73
• Enhancements to RHOS Pipelines
• Machine API – Defining a VMware vSphere failure domain for a control plane machine set (Technology Preview)
• Updates to hosted control planes within OSCP
• Bare-Metal hardware provisioning fixes

Changes not relevant to VSHN customers

There are a number of network related changes in this release, but these are not relevant for VSHN managed clusters as these are mostly running Cilium. It is also interesting to note the deprecation of the OpenShift SDN network plugin, which means no new clusters can leverage that setup. Additionally, there are new features related to specific cloud providers (like Oracle Cloud Infrastructure) or specific hardware stacks (like IBM Z or IBM Power).

The changes to handling storage and in particular storage appliances is also not relevant for VSHN customers as none of the storage features affect how we handle our storage on cloud providers or on-prem.

Features in OpenShift open to customer PoCs before we enable for all VSHN customers

We do have an interesting customer PoC with Red Hat OpenShift Virtualization which is an interesting feature that continues to mature in OpenShift 4.15. We are excited to see the outcome of this PoC and to potentially making that available to all our customers looking to leverage VMs inside OpenShift. We know due to the pricing changes from Broadcom that this is an area many companies and organizations are looking at. Moving from OpenShift running on vSphere to running on bare metal and having VMs inside OpenShift is an exciting transformation, and we hope to be able to bring an update on this in an upcoming separate blog post.

Likewise, we are open to customers who would like to explore leveraging OpenShift Serverless (now based on Knative 1.11 in Openshift 4.15) or perhaps with the new OpenShift Distributed Tracing Platform that is now at version 3.2.1 in the OpenShift 4.15 release (this version includes both the new platform based on Tempo and the now deprecated version based on Jaeger). This can also be used together with the Red Hat Open Telemetry Collector in OpenShift 4.15. There are also new versions of OpenShift Developer Hub (based on Backspace), OpenShift Dev Spaces and OpenShift Local. These are all interesting tools, part of the Red Hat OpenShift Container Platform.

If any of the various platform features are interesting for any existing or new VSHN customers, we would encourage you to reach out so we can discuss potentially doing a PoC together.

Summary

Overall, OpenShift 4.15 brings lots of small improvements but no major groundbreaking features from the perspective of the clusters run by VSHN customers. For those interested in the nitty gritty details of the OpenShift 4.15 release, we refer you to the detailed Red Hat release notes, which go through everything in detail.

VSHN customers will soon be notified about the upgrades to their specific clusters.

Interested in VSHN Managed OpenShift?

Head over to our product page VSHN Managed OpenShift to learn more about how VSHN can help you operate your own OpenShift cluster including setup, 24/7 operation, monitoring, backup and maintenance. Hosted in a public cloud of your choice or on-premises in your own data center.