Preface

This blog post has been written before version 1.0 of kpt has been released. As there is currently heave development ongoing with kpt, some of the statements in this blog post might be obsolete in the future. Some of the kpt commands used in the blog post might even fail with a kpt version > 0.39.

About kpt

Kpt is a toolkit from google for managing Kubernetes configuration files. It has been internally used at google since 2014 already and has been open-sourced back in March 2020. Kpt features include packaging, customizing, validating, and applying your Kubernetes resources.

This blog post will focus on kpt functions, one of the approaches of kpt for customizing your YAML files. Kpt setters unfortunately offer only fundamental options for YAML file customization. So if you decide to go with kpt for managing your Kubernetes resources, you will be sooner or later in the situation that you have to deal with kpt functions for more complex YAML file transformations.

Kpt functions are a powerful tool for customizing your YAML files as you can modify your YAML file however you want, and you are not limited to specific templating functions like in helm or customize. The bad thing is: google only covers in the docs how to write such functions with Typescript or Golang. The good news is: kpt functions are just containers (there are some other options, too, but this is the common one) that read Kubernetes resources as YAML from stdin, applying the desired modifications on the YAML, and writing them back to stdout. This can, of course, get done with any programming language of your choice. Google also recently published the KRM Functions Specification, which documents in detail the standard for kpt functions.

Writing your kpt function with bash

So let´s get started on how to create your own kpt function with a bash example:

As already mentioned, bash functions are just containers that read from stdin and write back to stdout, so we first need a Dockerfile for our container image, which contains all the tools we need for our function.

FROM alpine:3.12.3

RUN apk add --no-cache jq
RUN apk add --no-cache bash
RUN wget -O /usr/local/bin/yq "https://github.com/mikefarah/yq/releases/download/3.4.1/yq_linux_amd64"
RUN chmod +x /usr/local/bin/yq

COPY executor.sh /usr/local/bin/executor.sh
ENTRYPOINT ["/usr/local/bin/executor.sh"]

In my case, I am installing bash, yq for reading the YAML file and converting it to JSON and at the end back to YAML and jq for doing all the modifications in between. The executor.sh from the entrypoint is our actual implementation of the function.

When executing the kpt function, we receive all the Kubernetes resources defined in the kpt package as a resource list as specified in the KRM Functions specification. We then do our modifications and have to write the resource list back to stdout. If we want to throw errors, we can easily do this by echoing an unstructured text message to stderr and exiting the function with a return code greater than 0.

For an illustration of how this could look like, I created search-replace function similar to the existing one in Google´s functions catalog and implemented it with bash instead of Golang.

#!/bin/sh

IFS=$'\n'

for line in `cat`; do
    echo $line >> /tmp/input.yaml
done

INPUT_JSON=`yq r -d "*" /tmp/input.yaml  -j`
BY_VALUE=`echo $INPUT_JSON | jq '.functionConfig.data."by-value"' --raw-output`
BY_PATH=`echo $INPUT_JSON | jq '.functionConfig.data."by-path"' --raw-output`
PUT_VALUE=`echo $INPUT_JSON | jq '.functionConfig.data."put-value"' --raw-output`

if [ "$PUT_VALUE" == "null" ]; then
    echo "ERROR: you have to specify the function config parameter put-value" 1>&2
    exit 1
fi

if [ "$BY_VALUE" != "null" ]; then
    OUTPUT_JSON=`echo $INPUT_JSON | jq '.' | sed "s/: \"$BY_VALUE\"/: \"$PUT_VALUE\"/g"`
fi

if [ "$BY_PATH" != "null" ]; then
    PATH_ARRAY="["
    for line in `echo $BY_PATH | tr -s "." "\n"`; do
        if [ "$PATH_ARRAY" != "[" ]; then PATH_ARRAY="$PATH_ARRAY,"; fi
        PATH_ARRAY=$PATH_ARRAY'"'$line'"'
    done
    PATH_ARRAY="$PATH_ARRAY]"
    OUTPUT_JSON=`echo $INPUT_JSON | jq --argjson path $PATH_ARRAY --arg putvalue "$PUT_VALUE" '.items[] |= if getpath($path) then getpath($path)=$putvalue else . end'`
fi

echo $OUTPUT_JSON > /tmp/output.json
yq r /tmp/output.json -j  | yq r -P -

Like the original function this one is for searching and replacing fields across all resources. As you can see, this kpt function takes 3 function parameters: `by-value`, `by-path` and `put-value`. With the search matchers `by-value` and `by-path` you define which fields should get replaced (if you provide both matchers they are combined with an OR) and with the `put-value` parameter you set the value for the update for the matching fields.

Matchers

by-value
Match by value of a field. Also matches regular expressions.

by-path
Match by path expression of a field. Jq path expressions are used to deeply navigate and match particular yaml nodes. Please note that the path expressions are not regular expressions and do not accept wildcards.

Testing the function

The docker image with the above-explained kpt function has been pushed to public.ecr.aws/viesure/kptfunctions/search-replace and can easily get tested with imperative executions. Place the following 2 YAML files in a temporary directory:

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config-map
  namespace: ns1
data:
  color: red

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config-map-2
  namespace: ns2
data:
  color: green
  fruit: banana

Test the function with the following kpt commands:

kpt fn source . | kpt fn run --image public.ecr.aws/viesure/kptfunctions/search-replace:unstable -- 'by-path=metadata.namespace' 'put-value=dev'
kpt fn source . | kpt fn run --image public.ecr.aws/viesure/kptfunctions/search-replace:unstable -- 'by-value=green' 'put-value=red'

Summary

Thanks to the open KRM functions specifications, it is quite easy to use your choice´s programming language / toolset to create your kpt functions. It is as easy as reading YAML from stdin, doing your modifications and writing the YAML back to stdout according to the schema specified in the google KRM functions specifications. This also enables potential kpt users unfamiliar with the official recommended programming languages Golang and Typescript to write their own kpt functions and get out most of it when using kpt.