Ops-Talks

1.1 K8s Vs ECS

Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services. It is first and foremost a REST API

• Features: (non-exhaustive)
  • 🔹 Service discovery and load balancing
  • 🔹 Storage orchestration
  • 🔹 Automated rollouts and rollbacks
  • 🔹 Self-healing
  • 🔹 Secret and configuration management
  • 🔹 kubectl as a way to interact with the cluster

ECS is a fully managed container orchestration service

• Features:
  • 🔸 Integrated with AWS services
  • 🔸 Easy to pick up / Entry level to container world
  • 🔸 Ease of use from GUI + Serveless experience with Fargate
  • 🔸 ecs-cli and aws-cli as a way to interact with the cluster

1.2 Main concepts - Terminology

• Terminology used in ECS and K8s world

2.1 Basic Navigation

• 🚀 Setup for Kubernetes

2.1.1 Authentication

• Grab EKS configuration

1

aws eks update-kubeconfig --name <CLUSTER_NAME_FROM_EKS> --alias <FRIENDLY_ALIAS>

• Switch to verity-prod cluster

1
2
3
4
5
6

#--- Using kubectx
kubectx verity-prod

#--- Native way
kubectl config set-context verity-prod \
  && kubectl config use-context verity-prod

• Switch to the namespace default

1
2
3
4
5
6

#--- Using kubens
kubens default

#--- Native way
kubectl config set-context verity-prod --namespace=default \
  && kubectl config use-context verity-prod

2.1.2 Navigate Clusters / Deployments / Pods / Specs

• These are a really good resource, no need to bake one more 😃
  • Denny Zhang - Kubernetes Cheatsheet (PDF version)
  • Codefresh - Cheatsheet
  • Official Kubernetes Cheatsheet Documentation

3.1 Deep Dive - Helm

Helm is a package manager for kubernetes – Helm is your new ecs-cli…

• A chart is a collection of files that describe a related set of Kubernetes resources.

• A chart is made of Go templates

• A single chart might be used to deploy something simple, like a memcached pod, or something complex, like a full web app stack with HTTP servers, databases, caches, and so on.

1
2

$ helm create ops-talks
  Creating ops-talks

• Layout of a chart:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# Tree of the created chart
$ tree ops-talks

ops-talks
├── Chart.yaml
├── charts
├── templates
│   ├── NOTES.txt
│   ├── _helpers.tpl
│   ├── deployment.yaml
│   ├── hpa.yaml
│   ├── ingress.yaml
│   ├── service.yaml
│   ├── serviceaccount.yaml
│   └── tests
│       └── test-connection.yaml
└── values.yaml

• Helm charts can be fetched from open source git repositories, here are a few examples:

  • helm install stable/«chart»
  • helm install cp-helm/«chart»
  • helm install jenkinsci/«chart»
  • helm install lowess/«chart»

• Helm 🚰 hydrates a Chart using values contained in values.yaml. Overrides can be defined allowing you to change the default behavior of the chart.

3.2 Deep Dive - Helmfile

Helmfile is a wrapper on top of helm – Helmfile is what Terragrunt is to Terraform…

• Helm is a great tool for templating and sharing K8s manifests… However it can become quite cumbersome to install larger multi-tier applications or groups of applications across multiple Kubernetes clusters.

• Give each Helm chart its own helmfile.yaml and include them recursively in a centralized helmfile.yaml.

• Separate out environment specific values from general values. Often you’ll find while a Helm chart can take 50 different values, only a few actually differ between your environments.

• As well as providing a set of values, either Environment specific or otherwise, you can also read Environment Variables, Execute scripts and read their output (Fetch a secret from AWS SSM)

• Store remote state in git/s3/fileshare/etc in much the same way as Terraform does.

• helmfile project layout:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

helmfile.yaml
environments.yaml
releases
├── kafka-lag-reporting
│   ├── README.md
│   ├── helmfile.yaml
│   └── values
│       ├── ai-kafka-lag-reporting--production.yaml
│       ├── ai-kafka-lag-reporting--staging.yaml
│       └── kafka-lag-reporting--production.yaml
├── kafka-manager
│   ├── helmfile.yaml
│   └── values
│       └── kafka-manager--production.yaml
├── prometheus
│   ├── helmfile.yaml
│   └── values
│       ├── prometheus-mle--production.yaml
│       ├── prometheus-nlp--production.yaml
│       └── prometheus-verity--production.yaml
└── zoonavigator
    ├── helmfile.yaml
    └── values
        └── verity-zoonavigator--production.yaml

• Sample content of an helmfile.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

bases:
  - ../../environments.yaml

---

repositories:
  # Use Lowess (Florian Dambrine) OSS helm chart repo
  - name: lowess-helm
    url: https://lowess.github.io/helm-charts

templates:
  default: &default
    chart: "lowess-helm/karrot"
    missingFileHandler: Error
    namespace: "monitoring"
    labels: {}
    version: "0.1.3"
    wait: true
    installed: {{ and (env "KAFKA_LAG_REPORTING_INSTALLED" | default "true") }}

releases:
  - name: "ai-kafka-lag-reporting--{{ .Environment.Name }}"
    <<: *default
    values:
      - "./values/{{`{{ .Release.Name }}`}}.yaml"

3.3 GitOps

GitOps is a way to do Kubernetes cluster management and application delivery. It works by using Git as a single source of truth for declarative infrastructure and applications.

With GitOps, the use of software agents can alert on any divergence between Git with what’s running in a cluster, and if there’s a difference, Kubernetes reconcilers automatically update or rollback the cluster depending on the case. With Git at the center of your delivery pipelines, developers use familiar tools to make pull requests to accelerate and simplify both application deployments and operations tasks to Kubernetes.

Read more about GitOps on Weaveworks blogpost

#1 THE ENTIRE SYSTEM DESCRIBED DECLARATIVELY

#2 THE CANONICAL DESIRED SYSTEM STATE VERSIONED IN GIT

#3 APPROVED CHANGES THAT CAN BE AUTOMATICALLY APPLIED TO THE SYSTEM

#4 SOFTWARE AGENTS TO ENSURE CORRECTNESS AND ALERT ON DIVERGENCE.

:the_horns: Tips and Tricks

• kubectl shortcuts ❤ (this is gold)