22.08.2023 - Mikel Jason Münnekhoff - 21 min read Using Renovate to scale up automatic pull request creation on GitHub

Running infrastructure and applications is hard, but running those on large scale is even harder. Having multiple environments in different versions and configurations makes installing and operating even a single project a non-trivial task. CI/CD pipelines and the GitOps approach do help out here, but how can we run them at scale? Follow along and find out, how we run our infrastructure and support applications in several different instances, using Renovate to automate pull requests on GitHub (Enterprise Server).

The Scenario

Allow me to explain our setup briefly. We are a team of about 20 individuals developing, supporting and operating our client’s cloud framework. This framework is used for multiple applications (we don’t know in detail), for which multiple environments exist. In this article, we will focus on the Kubernetes part only, as this is the easier one. Regarding our cluster contents, we as the platform team manage some basic applications like storage and network drivers, observability and security. All our charts are stored in a monorepo called charts. Everything related to the specific purpose application, like databases, is stored in their specific monorepos, called charts-A for application A. Every application has multiple stages or environments. As we follow the GitOps approach, we have one repository for each application, e.g. k8s-A-dev contains all references to the deployed charts - generic and specific - for application A in its development cluster. In particular, we use ArgoCD for this.

With a growing number of applications onboarded to the cloud framework and discussions of using this not only on the German market but rolling it out for other countries, too, we need the update management to scale as well. So far, this had been done manually, resulting in the repetitive work of creating hundreds of pull requests.

Dependabot vs Renovate

You might already know dependabot, the magic tool that creates pull requests (PRs) on public GitHub when you don’t care about your last experimental project in a major programming language with package management. Sooner or later, you find an email for a PR that fixes a known security flaw by patching one of your dependencies. Luckily, you not only get the information that there is an update available, but you get a full PR including a fixing commit and some explanations in the PR body. Although you might find these to be annoying for a project you cared about one weekend three years ago, we wanted to have this for our repositories.

dependabot is a GitHub product, so we thought it might be the best tool for us to use. It has some nice integrations with the GitHub interface that no other tool can offer, so why not? Because it neither supports what we need nor allows enough customization to make it work on our side. You can find the full list of supported formats in the GitHub docs (this one is for GitHub Enterprise Server 3.8). You cannot find anything regarding ArgoCD, helm or Kubernetes on the list.

The next big thing we researched was Renovate. Renovate describes itself as a “Universal dependency update tool that fits into your workflows”. Indeed, it supports not only a lot of package managers but also multiple platforms like GitHub, GitLab, BitBucket as well as AWS CodeCommit or Azure DevOps. Besides its managed service, Renovate can be used privately, so there is no big issue of connecting something from the outside to our GitHub Enterprise Server (GHES), looking good so far. So we started configuring it to see what we can achieve.

Let’s Go Deep

We started setting up Renovate for our Kubernetes environment. Each EKS cluster has a corresponding application repository. These contain only namespaces and ArgoCD application definitions. These look like this:

 1apiVersion: argoproj.io/v1alpha1
 2kind: Application
 3metadata:
 4  name: coredns
 5  namespace: argo-A-dev
 6spec:
 7  source:
 8    project: target
 9    repoURL: https://github.company.com/org/charts.git
10    targetRevision: coredns/v1.2.3
11    path: charts/coredns/chart
12  destination:
13    server: https://[...].eks.amazonaws.com
14    namespace: kube-system
15  helm:
16    values: |
17      [...]      
18  syncPolicy:
19    automated:
20      prune: true
21      selfHeal: true

Here, we define that we deploy an instance of coredns. The helm chart is located in our generic charts repository. These are monorepos with a couple of helm charts in subdirectories. Here, we see path: charts/coredns/chart as the relative path in the charts repository. The targetRevision describes the version of the chart as a git revision. coredns/v1.2.3 is a tag in the repository, which lives alongside other version tags of the same application, e.g. coredns/v0.1.2, and tags of other applications in the same repository, e.g. prometheus/v1.1.1.

The ultimate goal for now is: Whenever there is a new internal version of coredns on our charts repository, Renovate opens a pull request to all application repositories to change the version in the ArgoCD application. Once this is merged, ArgoCD takes it away and updates the instances automatically.

Basic Setup as GitHub Action

Since Renovate offers an official GitHub Action, we started with this right away. Usually, one would configure Renovate in the repository containing the dependency declarations. For us, this would mean a few dozen configurations, one for each application repository. We chose the approach to not touch the application repositories at all, but to centralize the config to where the action is executed: our charts repository. The initial setup needed the following steps (assuming you have your action runners configured properly):

  1. Create a feature branch on the charts repository. For the developing/testing process, we did not need to merge to master.
  2. Add a GitHub workflow that triggers the Renovate action
    • For accessing the GHES, we configured a GitHub app and added the ID and private key to the repository’s secrets. Then, we use this GitHub action to get an access token that can then be used by Renovate.
    • Provide the access configuration as an environment variable to the action. Unfortunately, we were unsuccessful in integrating single environment variables (just the access token) in the configuration file. Luckily, Renovate offers several ways to provide configurations that are all merged into one during the execution. This allows to setup of the GHES access in the action definition.
    • Set the trigger of the workflow to every push on the feature branch. This is useful for testing and Renovate is designed to run multiple times on the same scenario, so we don’t expect any downsides here.
    • Set the log level to debug via the environment variable.
  3. Add the file .github/Renovate.json5 to the repository. This is one of the default files the script is looking for. We chose json5 over json to be able to add comments.

The workflow definition should look like this:

 1name: Renovate
 2
 3on:
 4  workflow_dispatch:
 5  push:
 6    branches:
 7
 8jobs:
 9  Renovate:
10    runs-on: self-hosted
11    steps:
12      - name: Checkout
13        uses: actions/checkout@v3
14
15      - name: Generate Token
16        id: generate_token
17        uses: tibdex/github-app-token@v1
18        with:
19          app_id: ${{ secrets.APP_BOT_ID }}
20          private_key: ${{ secrets.APP_BOT_PRIVATE_KEY }}
21
22      - name: Self-hosted Renovate
23        uses: renovatebot/github-action@v34.82.0
24        env:
25          LOG_LEVEL: 'debug'
26          RENOVATE_HOST_RULES: '[{"matchHost": "https://github.company.com/", "token": "${{ steps.generate_token.outputs.token }}"}]'
27        with:
28          configurationFile: "${{ github.workspace }}/.github/renovate.json5"
29          token: ${{ steps.generate_token.outputs.token }}

Let’s jump to the Renovate configuration stored in .github/renovate.json5. You can find all available parameters and their respective descriptions in the official Renovate docs. We will go through the minimal setup parameters one by one:

  • dryRun: The very first setup is to deactivate creating PRs to not annoy other team members. The debug log is sufficient to see available updates, so let’s not create PRs that send emails to users watching the app repositories or being auto-assigned (by other tools).
  • repositories: Renovate needs to know which repositories to check and update. There is an auto-discovery feature, but we want to have fine-grained control over its scope, because we have a centralized configuration.
  • requireConfig: We tell Renovate to not expect a configuration file in the application repositories.
  • username & gitAuthor: Commits authored by the action will be filed with this user and email address. This is like running git config --global user.name or user.email on your local machine.
  • onboarding: There’s an onboarding feature for repositories that creates pull requests to setup a proper configuration, but we don’t want this.
  • platform, gitUrl endpoint: Enable Renovate to talk to our GHES instance.
  • baseBranches: For development purposes, don’t watch the default branches that ArgoCD watches as well, but a new one. This allows to play around with the versions without any effect on the clusters.

Using all of these options, our first configuration looked like this:

 1{
 2  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
 3  "username": "bot",
 4  "gitAuthor": "bot <bot@users.noreply.github.com>",
 5  "onboarding": false,
 6  "requireConfig": "ignored", // no need for config in the app repos, we have everything here
 7  "platform": "github",
 8  "repositories": [
 9    "org/k8s-A-dev"
10  ],
11  "gitUrl": "endpoint",
12  "endpoint": "https://github.company.com/api/v3/",
13  "dryRun": "full",
14  "baseBranches": ["feat/renovate"],
15}

Learning from the first attempt

With all of these parameters in place, the action was ready to run by pushing the feature branch to the GHES. The results of the run can be viewed in the action’s log. First, we see the action picking up all configurations and merging them into one. So far so good, nothing unexpected. Scrolling further down, we get to the first - in our case - unwanted behavior: As per default, Renovate runs with most so-called managers. Managers correspond to package managers and here means which language or dependency type the action is looking for.

DEBUG: Using file match: (^|/)tasks/[^/]+\.ya?ml$ for manager ansible (repository=org/k8s-A-dev)
DEBUG: Using file match: (^|/)requirements\.ya?ml$ for manager ansible-galaxy (repository=org/k8s-A-dev)
DEBUG: Using file match: (^|/)galaxy\.ya?ml$ for manager ansible-galaxy (repository=org/k8s-A-dev)
DEBUG: Using file match: (^|/)\.tool-versions$ for manager asdf (repository=org/k8s-A-dev)
DEBUG: Using file match: azure.*pipelines?.*\.ya?ml$ for manager azure-pipelines (repository=org/k8s-A-dev)
DEBUG: Using file match: (^|/)batect(-bundle)?\.yml$ for manager batect (repository=org/k8s-A-dev)
[...]
DEBUG: Matched 10 file(s) for manager github-actions: [...] (repository=org/k8s-A-dev)
DEBUG: Matched 1 file(s) for manager helm-values: apps/values.yaml (repository=org/k8s-A-dev)
DEBUG: Matched 1 file(s) for manager helmv3: apps/Chart.yaml (repository=org/k8s-A-dev)

Although we seem to have nothing but our ArgoCD applications in the app repositories (so they find nothing and stay silent), some managers kick in for some supporting content: GitHub Actions and helm metadata. We don’t want PRs for outdated actions or metadata. Unfortunately, our ArgoCD applications are not picked up. Thus, we need to tune the config. Fortunately, we can simply enable the argocd manager. To do so, we need to set the managers and configure the ArgoCD one to match files by path:

1{
2  [...]
3  "enabledManagers": [
4    "argocd"
5  ],
6  "argocd": {
7    "fileMatch": ["apps/.*\\.ya?ml$"]
8  },
9}

Running this again leads to our applications being picked up. By path, also Chart.yaml and values.yaml are matched, but the manager correctly detects that these are no ArgoCD files and disregards them. We can see this when the actual package files are the matched files reduced by 2.

DEBUG: Using file match: apps/.*\.ya?ml$ for manager argocd (repository=org/k8s-A-dev)
DEBUG: Matched 42 file(s) for manager argocd: apps/Chart.yaml, apps/templates/apps/aws-alb-controller.yaml, [...] (repository=org/k8s-A-dev)
DEBUG: Found argocd package files (repository=org/k8s-A-dev)
DEBUG: Found 40 package file(s) (repository=org/k8s-A-dev)

Further investigation of the action log leads to the second issue with the minimalistic setup, which is more severe. The manager argocd had been active, so we expected or, more realistically, hoped to see available updates or at least that everything’s up to date. Indeed, Renovate did find our ArgoCD application files, but did not manage to compare versions properly.

DEBUG: packageFiles with updates (repository=org/k8s-A-dev, baseBranch=feat/renovate)
        "config": {
          "argocd": [
            [...]
            {
              "packageFile": "apps/templates/apps/coredns.yml",
              "deps": [
                {
                  "depName": "https://github.company.com/org/charts",
                  "currentValue": "coredns/v0.1.1",
                  "datasource": "git-tags",
                  "depIndex": 0,
                  "updates": [],
                  "warnings": [],
                  "versioning": "semver",
                  "skipReason": "invalid-value"
                }
              ]
            },

With "skipReason": "invalid-value", the log gave us the hint that they cannot parse the versions. By default, Renovate expects semantic versioning, to which we don’t strictly comply with our monorepo and scoped tags. For this, Renovate allows us to adjust the versioning structure. The Renovate docs provide some presets of versioning, but for individual cases like ours, we can use the regex variant. With this, we are required to configure a regular expression that matches some named capture groups. We are interested in major, minor and patch for the version, build and prerelease are irrelevant in our use case. In addition, there is the named group compatibility. According to the docs, this can be used to define build compatibility that will never change, e.g. v1.2.3-linux should never be updated to something with -windows as the suffix. We used this option and let the application scope of the version tag match the compatibility group. Our used versioning regex is

1{
2  [...]
3  "versioning" : "regex:^(?<compatibility>.*)/v?(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)?$"
4}

This finally read and analyzed our applications correctly:

DEBUG: packageFiles with updates (repository=org/k8s-A-dev, baseBranch=feat/renovate)
        "config": {
          "argocd": [
            [...]
            {
              "packageFile": "apps/templates/apps/aws-alb-controller.yaml",
              "deps": [
                {
                  "depName": "https://github.company.com/org/charts",
                  "currentValue": "aws-lb-controller/v0.0.2",
                  "datasource": "git-tags",
                  "depIndex": 0,
                  "updates": [],
                  "warnings": [],
                  "versioning": "regex:^(?<compatibility>.*)/v?(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)?$",
                  "sourceUrl": "https://github.company.com/org/charts",
                  "currentVersion": "aws-lb-controller/v0.0.2",
                  "fixedVersion": "aws-lb-controller/v0.0.2"
                }
              ]
            },
            {
              "packageFile": "apps/templates/apps/coredns.yml",
              "deps": [
                {
                  "depName": "https://github.company.com/org/charts",
                  "currentValue": "coredns/v0.1.1",
                  "datasource": "git-tags",
                  "depIndex": 0,
                  "updates": [
                    {
                      "bucket": "non-major",
                      "newVersion": "coredns/v0.1.2",
                      "newValue": "coredns/v0.1.2",
                      "newDigest": "22c39efb005a9c740d4eca008b527def2df264f3",
                      "newMajor": 0,
                      "newMinor": 1,
                      "updateType": "patch",
                      "branchName": "renovate/https-github.company.com-charts-0.x"
                    }
                  ],
                  "warnings": [],
                  "versioning": "regex:^(?<compatibility>.*)/v?(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)?$",
                  "sourceUrl": "https://github.company.com/org/charts",
                  "currentVersion": "coredns/v0.1.1",
                  "isSingleVersion": true,
                  "fixedVersion": "coredns/v0.1.1"
                }
              ]
            },

The third major problem with this setup is not easy to see when you’re thrilled that it finally works after dozens of attempts. To be honest, we did not learn this with dryRun activated, but with real PRs. Now that we know, we see this quite obviously in the log. Look up again and read the section branchName for the coreDNS update very carefully. We notice that PRs are grouped by the detected dependency. In our simple case, everything comes from the charts repository, and this means all dependency updates are handled in one PR. From our real-run experience, we can tell this also means only one commit. In our opinion, this does not make sense for a couple of reasons:

  • We don’t want to mix up multiple applications in one because
    • We use and enforce conventional commits with the application as scope - we cannot comply with more than one application in one commit (example: feat(coredns): update node selector).
    • If two individuals are working on different applications that should be merged, the responsibility for doing so is on two different individuals. Therefore, we want two PRs with separate approval processes.
    • We don’t want to have all environments up to date immediately. Maybe, we need to hold back one version in a productive environment. If so, the whole PR automation would fail since it would pick up the unwanted update all the time without the option to selectively ignore one.
  • We don’t want to mix up multiple versions. We have other PR automation tools in place, including one that automatically closes PRs on certain conditions. While it might be fairly safe to merge patches automatically (in some environments), we definitely don’t want to do this for major versions.

To summarize, we are pretty restricted with the argocd manager. Take a short look in the source code here to see what it actually matches: depName and currentValue (the version) from the files, plus it sets the datasource to git tags as fixed value (you likely don’t know these parameters yet, but they will make perfectly sense very shortly). We simply don’t have enough flexibility for our setup, so let’s try another strategy.

Use regex manager

For any dependency for which no other manager is suitable, the regex manager comes to the rescue. Here we do everything on our own. As we did it for the versioning schema, we needed to write some regular expressions that match some pre-defined named capture groups. Please find the documentation of the regex manager and all available named capture groups here. For our scenario, we do care about the following groups:

  • depName: This is the real name of the dependency, e.g. coredns or prometheus.
  • packageName: This will hold the charts repository (or any other repo that contains the dependency) URL. Renovate needs to know where to fetch content from. If this is empty, this will be filled with depName which is the reason for the URL being stored in depName when using the argo manager.
  • currentValue: This is the full version, e.g. coredns/v1.2.3
  • datasourceTemplate: We want to set this to GitHub tags as a fixed setting because GitHub is the only source we have (no OCI repository or else).

We could set the fixed value directly. For actual matching, we needed to provide an array of strings as matchStrings parameter. Matching all mentioned values in one regex might be too complex, if not impossible. Therefore, we must set matchStringsStrategy to combination. This enforces that target files match all provided expressions in our array, not only one as per default (this would be strategy any). This also allowed us to restrict the dependency matching further. We just added another regex to enforce that we only match files with kind: Application and apiVersion: argoproj.io/.... Remember: The argocd manager filtered two YAML files, but we cannot expect that from simple regex and ensured no non-argocd files on our own. Plus without a tailing $, we would match kind: ApplicationSet here, too.

Without further ado, bring on the regex manager config!

 1{
 2  [...]
 3  "regexManagers": [
 4    {
 5      "fileMatch": [
 6        "apps/.*\\.ya?ml$"
 7      ],
 8      "datasourceTemplate": "git-tags",
 9      "matchStringsStrategy": "combination",
10      "matchStrings": [
11        "apiVersion: [\"]?argoproj.io[\"]?",
12        "kind: [\"]?Application[\"]?",
13        "repoURL: [\"]?(?<packageName>[^\"\\s]+)[\"]?\\n",
14        "targetRevision: [\"]?(?<currentValue>[^\"\\s]+)[\"]?\\n",
15        "metadata:\\s*\\n\\s{2}name:\\s+[\"]?(?<depName>[^\"\\s]+)[\"]?\\n"
16      ],
17    }
18  ],
19}

Commit the changes and push those to fire up a new Renovate run. Taking a look at the new log, we managed to mitigate our issue of combining all changes into one branch and commit. Now, all update branches are named after the application it takes care of:

DEBUG: packageFiles with updates (repository=org/k8s-A-dev, baseBranch=feat/renovate)
        "config": {
          "argocd": [
            [...]
            {
              "packageFile": "apps/templates/apps/coredns.yml",
              "deps": [
                {
                  "depName": "coredns",
                  "packageName": "https://github.company.com/org/charts",
                  "currentValue": "coredns/v0.1.1",
                  "datasource": "git-tags",
                  "replaceString": "targetRevision: coredns/v0.1.1\n",
                  "depIndex": 0,
                  "updates": [
                    {
                      "bucket": "non-major",
                      "newVersion": "coredns/v0.1.2",
                      "newValue": "coredns/v0.1.2",
                      "newDigest": "22c39efb005a9c740d4eca008b527def2df264f3",
                      "newMajor": 0,
                      "newMinor": 1,
                      "updateType": "patch",
                      "branchName": "renovate/coredns-0.x"
                    }
                  ],
                  "warnings": [],
                  "versioning": "regex:^(?<compatibility>.*)/v?(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)?$",
                  "sourceUrl": "https://github.company.com/org/charts",
                  "currentVersion": "coredns/v0.1.1",
                  "isSingleVersion": true,
                  "fixedVersion": "coredns/v0.1.1"
                }
              ],
              "matchStrings": [
                "apiVersion: [\"]?argoproj.io[\"]?",
                "kind: [\"]?Application[\"]?",
                "repoURL: [\"]?(?<packageName>[^\"\\s]+)[\"]?\\n",
                "targetRevision: [\"]?(?<currentValue>[^\"\\s]+)[\"]?\\n",
                "metadata:\\s*\\n\\s{2}name:\\s+[\"]?(?<depName>[^\"\\s]+)[\"]?\\n"
              ],
              "matchStringsStrategy": "combination",
              "datasourceTemplate": "git-tags"
            },

Setup commit message and PR content

Now that the detection and grouping were fine, we tackled the finishing touches: Commit messages and PR contents. Renovate offers enough flexibility to set the commit message as we like. It even offers support for semantic commits. We simply used this to set the commit messages as follows:

1{
2  [...]
3  "semanticCommits": "enabled",
4  "semanticCommitType": "feat",
5  "semanticCommitScope": "{{{depName}}}",
6  "commitMessageAction": "update",
7  "commitMessageTopic": "to",
8  "commitMessageExtra": "{{{newVersion}}}",
9}

With the usage of the {{{variables}}} Renovate offers, this produces commit messages like feat(coredns): update to coredns/v1.2.3.

Next up: PR content. As we don’t want to mix up Renovate branches with our development branches, we adjust the naming of the branches Renovate will create. For this, we use the option branchPrefix to group all branches, and branchTopic to more verbose naming per branch.

1{
2  [...]
3  "branchPrefix": "dependency/",
4  "branchTopic": "{{{depNameSanitized}}}", // remove version number from branch name
5}

As per default, Renovate populates a PR body with some content like an overview table or changelog/release notes. Since Renovate does not support monorepo here, the automatic changelog includes changes in other applications as well. Here, this is irritating, so we get rid of it. Renovate offers the prBodyTemplate parameter to override the body template. We copied the default from the docs and removed unwanted contents: changelogs (we provide our own), controls (not needed) and footer (unwanted Renovate ad). Instead, we added some custom content in prBodyNotes. We wanted to set a short reminder that this is a plain automation and does not check possible required configuration changes. Also, we wanted to add a link to the GitHub release of the new version, as this contains our release notes. As we have the repository URL as the packageName and the tag as newValue, we can create the link easily.

1{
2  [...]
3  "prBodyTemplate": "{{{header}}}{{{table}}}{{{warnings}}}{{{notes}}}", // remove default changelog (not monorepo-aware) and footer (Renovate advertisement)
4  "prBodyNotes": [
5    "**Release notes**: {{{packageName}}}/releases/tag/{{{encodeURIComponent newVersion}}}", // produces link to release in correct repo!
6    "**Remember**: This PR is a simple update of the version and does not cover any, **probably required**, configuration changes! You can use this PR to add those changes if needed."
7  ],
8}

Lastly, we wanted to set some PR labels. This is important to us for integration with our other PR integration tools. We could easily set the target labels via the labels parameter:

1{
2  [...]
3  "addLabels": [
4    "dependencies",
5    "needs triage"
6  ],
7}

Et voilà, a fully automated pull request to update a single dependency in an environment:

Behind the (writing) scenes

Of course, this blog post is condensed to what works and what is generic enough to be valuable for any developer or engineer out there. But let us quickly discuss two obstacles we encountered, that might be helpful to know about, too.

First of all, let’s talk about variables in dependencies. It might be nice to use helm chart variables in ArgoCD applications. We did so, too. In our case, we had the base URL of our GHES as a variable in Kubernetes. While this absolutely works for the functional part, Renovate does not template helm charts, so using variables is a blocker for Renovate and we needed to remove these first.

The second obstacle was one we honestly don’t understand the reason for. Renovate offers the usage of presets to not start from scratch but have some well-tested configurations. We started using it out of the thought that we use it as a first start and then customize it further and further. However, there’s one caveat you need to be aware of: Preset values take precedence over your custom configuration. We noticed this when we were trying to set the semantic commit type to feat. We were unable to do so, because we were using the config:base preset which sets this to chore. After reading this important detail in the docs by chance, we reverse-engineered all settings our configured presets contain, and dropped them with minimal changes on our side. Luckily, the docs list what the prefixes contain, e.g. config:base. It was pretty annoying to check everything (spoiler: they are built recursively), but we managed to get down to only one preset, :disableDependencyDashboard.

1{
2  [...]
3  "extends": [
4    ":disableDependencyDashboard", // do not create dashboard issue
5  ],
6}

Rebasing & Cronjobs

There’s one thing left to discuss: Rebasing. For all our repositories, we enforce linear git histories. To merge PRs, we need to rebase to the latest main branch. One, this is annoying manual work, but two, due to our automation tools, whoever rebases the branch (e.g. in GitHub Web UI, it’s just one click) cannot approve the PR anymore because they become involved as committer. Renovate has the option to automatically rebase open branches. This can be configured by adding :rebaseStalePrs as a second preset. To regularly rebase all branches, we then need to run our GitHub action as a cronjob, too.

1name: Renovate
2on:
3  workflow_dispatch:
4    schedule:
5      - cron: '0 0 * * *'
6  push:
7    tags:
8      - '*/v[0-9]+\.[0-9]+\.[0-9]+'

We activated rebasing, but it has a side effect. We have another action in place that marks PRs as stale and closes them after certain limits of no activity. Now, Renovate rebases these PR branches, so effectively, these PRs are not going to be marked as stale although there is no human activity. We have not decided on how to deal with this yet, time will tell. For now, we keep the rebasing.

All in all, this setup is ready to go out there and reduce our manual, repetitive work, at least to some extent. We still need to approve manually because we don’t have an overarching versioning strategy on the project yet. And because other tools are used to perform checks and auto-merge, we did not touch the auto-merging capabilities of Renovate, but they definitely exist!

Putting it all together, the configuration now looks like below and we are ready to roll this out and gather some lessons learned in the field. For this, we only need to change the trigger of the action to new tags instead of pushes to the feature branch, drop the custom base branch and the dryRun in the Renovate config and merge it to master.

 1{
 2  "$schema": "https://docs.renovatebot.com/Renovate-schema.json",
 3  "extends": [
 4    ":disableDependencyDashboard", // do not create dashboard issue as no deployment can react to anything there
 5    ":rebaseStalePrs" // auto-rebase branches
 6  ],
 7  "enabledManagers": [ // remove all other managers, esp. gh actions
 8    "regex"
 9  ],
10  "regexManagers": [ // use regex instead of arcocd manager because it is not capable of our monorepo approach
11    {
12      "fileMatch": [
13        "apps/.*\\.ya?ml$"
14      ],
15      "datasourceTemplate": "git-tags",
16      "matchStringsStrategy": "combination",
17      "matchStrings": [
18        "apiVersion: [\"]?argoproj.io[\"]?", // must match only argo app files
19        "kind: [\"]?Application[\"]?",
20        "repoURL: [\"]?(?<packageName>[^\"\\s]+)[\"]?\\n", // repo url as 'packageName' -> cannot use {{ .Values.xxx }} here!
21        "targetRevision: [\"]?(?<currentValue>[^\"\\s]+)[\"]?\\n", // full target rev as 'currentValue' which is then handled by 'versioningTemplate'
22        "metadata:\\s*\\n\\s{2}name:\\s+[\"]?(?<depName>[^\"\\s]+)[\"]?\\n" // read dependency name from metadata.name to distinguish xxx-crd which uses xxx/v0.0.0 tag
23      ],
24    }
25  ],
26  "versioning": "regex:^(?<compatibility>.*)/v?(?<major>\\d+)\\.(?<minor>\\d+)\\.(?<patch>\\d+)?$",
27  "semanticCommits": "enabled",
28  "semanticCommitType": "feat",
29  "semanticCommitScope": "{{{depName}}}",
30  "commitMessageAction": "update",
31  "commitMessageTopic": "to",
32  "commitMessageExtra": "{{{newVersion}}}",
33  "branchPrefix": "dependency/",
34  "branchTopic": "{{{depNameSanitized}}}", // remove version number from branch name
35  "addLabels": [
36    "dependencies",
37    "needs triage"
38  ],
39  "username": "bot",
40  "gitAuthor": "bot <bot@users.noreply.github.com>",
41  "onboarding": false,
42  "requireConfig": "ignored", // no need for config in the app repos, we have everything here
43  "platform": "github",
44  "repositories": [
45    "org/k8s-A-dev"
46  ],
47  "prBodyTemplate": "{{{header}}}{{{table}}}{{{warnings}}}{{{notes}}}", // remove default changelog (not monorepo-aware) and footer (Renovate advertisement)
48  "prBodyNotes": [
49    "**Release notes**: {{{packageName}}}/releases/tag/{{{encodeURIComponent newVersion}}}", // produces link to release in correct repo!
50    "**Remember**: This PR is a simple update of the version and does not cover any, **probably required**, configuration changes! You can use this PR to add those changes if needed."
51  ],
52  "gitUrl": "endpoint",
53  "endpoint": "https://github.company.com/api/v3/",
54  // "dryRun": "full",
55  // "baseBranches": ["feat/renovate"],
56}

Credits

Title image by ESB Professional on Shutterstock

Mikel Jason Münnekhoff

Senior Technical Consultant