Skip to content

3. Create cluster

The following is a guide for how to create an environment that contains the elements described in Compare and contrast.

To create a cluster using the declarative approach:

  1. Create a declaration and edit it to suit your needs
  2. Apply the declaration


To scaffold a cluster declaration, run the following:

# Usage
okctl scaffold cluster > PATH

# Example
okctl scaffold cluster > cluster.yaml

To create a cluster based on the declaration, run the following:

# Usage
okctl apply cluster -f PATH

# Example
okctl apply cluster -f cluster.yaml


The declarative approach does exactly the same as okctl create cluster used to do, except that it gets all its user input up front. That way you can easily take down and recreate clusters as you experiment. You create a declaration once and apply it as many times as you need.


When running

okctl scaffold cluster > cluster.yaml

you'll end up with a file looking like this:

kind: Cluster

## For help finding values, see
  ## Account ID is your AWS account ID
  accountID: '123456789012'

  ## Name can be anything, but should define the scope of the cluster. Meaning if the cluster is scoped to one product,
  ## you might want to name it the name of the product. If the cluster contains all services and products owned by a
  ## team, the team name might be more fitting.
  name: someclustername

  ## Region defines the AWS region to prefer when creating resources
  # region: eu-west-1

## The cluster root domain defines the domain of which to create services beneath. For example; okctl will setup ArgoCD
## which has a frontend. The frontend will be available at https://argocd.<clusterRootDomain>. For Cognito it will be
## https://auth.<clusterRootDomain>

## For okctl to be able to setup ArgoCD correctly for you, it needs to know what repository on Github that will contain
## your infrastructure.
  ## The name of the repository
  repository: my_repo

  ## The organization that owns the repository
  # organisation: oslokommune

  ## The folder to place infrastructure declarations
  # outputPath: my_infrastructure

## Defines what users can access everything connected to Cognito. Applications connected to Cognito include: ArgoCD UI,
## Grafana. 
#- email:

  ## ArgoCD is a service that watches a repository for Kubernetes charts and ensures the defined resources are running
  ## as declared in the cluster
  argoCD: true

  ## Autoscaler automatically adjusts the size of pods and nodes in your cluster depending on load
  autoscaler: true

  ## AWS Load Balancer Controller handles routing from the internet to your application running inside your okctl
  ## Kubernetes cluster. If you want your applications and services accessible from the internet, this needs to be
  ## enabled.
  awsLoadBalancerController: true

  ## Block storage provides persistent storage for your cluster (Persistent Volumes)
  blockstorage: true

  ## Cognito is an authentication provider that okctl uses to control access to different resources, like ArgoCD and
  ## Grafana
  cognito: true

  ## External DNS handles defining the necessary DNS records required to route traffic to your defined service or
  ## application
  externalDNS: true

  ## External Secrets fetches secrets from external sources and exposes them as native Kubernetes secrets inside the
  ## cluster
  externalSecrets: true

  ## KubePromStack enables Prometheus and Grafana for metrics
  kubePromStack: true

  ## Loki collects logs and exposes them as a data source in Grafana
  loki: true

  ## Promtail scrapes logs from pods and feeds them to Loki
  promtail: true

  ## Tempo collects traces and exposes them as a data source in Grafana. Supports formats like jaeger, zipkin, open
  ## telemetry
  tempo: true

## Defines databases to provision
#  postgres:
## Name defines the name of the database to provision
#  - name: dbname
## Namespace defines what namespace to place the database information (secret for user, pass and configmap hostname, port
#    namespace: relevantnamespace
## User defines what user to provision for operations
#    user: postgres

## okctl creates a Virtual Private Cloud for you which it organizes all the intended resources that require networking.
## A VPC is mandatory, but can be configured by the following attributes.
  ## CIDR defines the VPC IP range. Leave this be if you don't know what it is/does
  # cidr:

  ## HighAvailability means we create redundancy in the network setup. If set to true we will create a NAT gateway per
  ## public subnet, instead of routing all traffic through one.
  # highAvailability: true

Modify the declaration to suit your situation and needs, then use

okctl apply cluster -f cluster.yaml

to have okctl generate a cluster based on the declaration.

That's it. Sit back and enjoy or go do something else while okctl does its thing (can take up to an hour).

NB: You need to have an ssh agent running with ssh-key associated with your github account added. The example below uses default location for ssh key. Replace path if your key is located elsewhere.

eval `ssh-agent`
ssh-add ~/.ssh/id_rsa


Device authentication flow

For okctl to be able to interact with GitHub on your behalf, okctl needs do something called the Device Authentication flow.

In the beginning of the cluster creation process, okctl will ask you to enter a code in a browser window. The code will be presented. Copy it and press Y and enter. okctl will open a window in your browser where you can enter the code.

Copy the code highlighted in red and press Y and then enter device auth

Paste it into the newly opened tab in your browser enter code

Press the green "Authorize oslokommune" button authorize

That's it! You are all set. Switch back to the console and enjoy okctl creating your cluster for you.


In some cases it can be useful to use a service user for authentication.

By passing the flag --aws-credentials-type access-key to okctl apply cluster, okctl will look for the following environment variables to be used to authenticate:

# Example
export AWS_ACCESS_KEY_ID=myid

okctl apply cluster --aws-credentials-type access-key -f cluster.yaml


In some cases it can be useful to use a service user for authentication.

By passing the flag --github-credentials-type token to okctl apply cluster, okctl will look for the following environment variable to be used to authenticate:

# Example
export GITHUB_TOKEN=mytoken

okctl apply cluster --github-credentials-type token -f cluster.yaml

Where do I find this declaration attribute?


Go to Log in as you usually do - find account id as shown here: okctl


The name of the environment depends on what the cluster will be used for. Examples being:

  • production
  • development
  • staging

Basically anything you want.


Go to your infrastructure as code (IAC) repository, and find the name of the repository in the top left corner. It should look something like this: oslokommune/<repository name>


It will be the part after the /.


  • Go to:
  • Search for your team name, for example search for "kjøremiljø"
  • Copy the text after @oslokommune/. This is your team name (in this case kjoremiljo)


Any other attribute

If you don't know what it does, default is probably fine.

Last update: 2021-10-12