Blog

Along the way, we will learn about typical k8s objects including Deployment, Service, PersistentVolume, Secret, but also related stuff like port-forwarding, authorization, building container images, and many others. Just like the first part, this post is also going to be elaborate containing many details because k8s is a behemoth, and I think it is a good idea to look a little bit under the hood to get a better understanding of how all the pieces work together. What we will do is to simply break it down into smaller parts, and you will see that using a handful of yaml files were enough to easily deploy applications on k8s.

Circling back to the applications, we are going to deploy three items:

1. A frontend written in Nuxt 3 that you will see and interact with

2. A database using Postgres that will store our data

3. A backend api server written in Go

The frontend will be a simple interface where it will list the latest ten UUID items which are stored from a database. We will add an ‘add’ button where a new random UUID is generated and stored into the database. Once that is done, it will do another api call to retrieve the latest UUIDs.

Kubernetes… everyone’s favourite container orchestration. Love or hate it, it is fast becoming the de-facto solution for managing containers at scale. This blog post will go through step by step on how we can create a kubernetes (k8s) cluster on a local machine from the first command to all nodes being ready in about 12 minutes. This post assumes the readers to have basic knowledge of k8s. If you haven’t heard of k8s, you should read this.

Before we jump into the guide, I will outline the tools and the kind of k8s cluster we are going to create. We know that creating a k8s cluster is complex. kubeadm has made the process easier but there are still steps that feel like a maze. Do you not need docker with k8s anymore? What are the various container runtimes? Why are there so many choices for pod networking? What to do with ‘cgroup driver’? This blog post intends to go through each step and be a complete guide into creating a highly available kubernetes cluster.

The plan is to create a series of blog posts to create a production-grade k8s cluster. The first step is to create an HA cluster. This is important because we do not want to have any single point of failure in our architecture. At the end of this post, we will deploy a vanilla nginx web server to demonstrate that we can successfully access its pod with the correct port number.

Second part of the series is going to be deploying more complex applications. A popular architecture for a web application is having a single-page application (SPA) frontend talking to an api backend on a specific port. We will see how we can configure both k8s and our applications to achieve this. We will throw in observability which includes prometheus, jaeger, and loki as well as grafana for visualisation.

The final part is going to be about ingress controller, so we can access our application in local machine from the internet. We will need a domain name for this, and we will configure the DNS to point to our k8s cluster and see how our virtual IP (floating IP) from keepalived plays into this. To make our cluster more production-grade, we also look into backup and restore etcd database.

This guide on the first part does not come out of vacuum. I heavily referred to this video https://youtu.be/SueeqeioyKY and highly available (HA) section of k8s at https://github.com/kubernetes/kubeadm/blob/main/docs/ha-considerations.md#options-for-software-load-balancing. Full list of reference is at https://github.com/gmhafiz/k8s-ha#reference.

Why is k8s short for kubernetes? There are 8 letters in between of ‘k’ and ’s’   😎

If you do not want to read the rest, tldr:

git clone https://github.com/gmhafiz/k8s-ha
cd k8s-ha
make install 
make up
make cluster

http GET http://localhost:3080/api/sqlx/user?favourite_colour=blue;select%20*%20FROM%20users;,--;

This malicious URL is not the only way of trying to inject SQL queries. Others include using AND 1=1 which means the query is always true, and the dreaded ';DROP DATABASE users-- which mean adding a second SQL query to drop users table. You can try out with different urls by running the examples in the provided example/rest.http file.

The post is incomplete without the addition of this comic:

(from https://xkcd.com/327/)

There are two things we want to see how these libraries and ORMs behave. The first being if malicious sql gets injected, and second how do they behave and the output that gets returned.

Typically, to make a transaction, we first start by calling BeginTx() method in database/sql package which returns a Tx transaction struct.

Following the pattern from the Go blog at https://go.dev/doc/database/execute-transactions, we defer the rollback operation immediately after starting a transaction and ends with a commit. This ensures that a rollback is always called even if any error or panic happens in the function.

If you carefully analyse the pattern above, you will notice that a rollback will happen after a commit. Fortunately, if that transaction is already done, Go will ignore the rollback and does not make another round trip to the database - Rollback() will be ignored if the transaction has been committed as seen from the source code,

// rollback aborts the transaction and optionally forces the pool to discard
// the connection.
func (tx *Tx) rollback(discardConn bool) error {
	if !atomic.CompareAndSwapInt32(&tx.done, 0, 1) {
		return ErrTxDone
	}
	...

While knowing the pattern above is important, we want to know a couple of things when it comes to creating a transaction across multiple SQL queries.

1. Reuse existing methods, collect, and run them in a transaction.
2. Adjust transaction isolation levels.

It would be nice if we can just reuse existing methods because why should we repeat ourselves if the operations are the same, but we needed a transaction over them.

Understanding isolation level is essential when deciding which level we want our transaction to run on. Isolation is the ‘I’ in ACID. There are four isolation levels in SQL. In reality, there are only three in postgres as Read Uncommitted behaves like Read Committed. You can read more about isolation level in further details. The default in postgres is read committed. This means that a SELECT query will only see data committed before the query began. In other words, it will not see any in-progress uncommitted changes.

In this post, we will see how these libraries and ORMs fare with creating and using a transaction as well as changing an isolation level.

For pagination, we can either define offset and limit or use the convenience page key: GET /api/users?page=2. Please note that there is a huge penalty LIMIT and OFFSET when you are retrieving records where the offset is in the millions. See Why You Shouldn’t Use OFFSET and LIMIT For Your Pagination. ORDER BY also must be added in all LIMIT and OFFSET.

In our pagination examples, we simply order by ID. This is important to constraint the result rows into a unique order https://www.postgresql.org/docs/15/queries-limit.html.

In short, we will look at these three common list operations

1. Filter by field
2. Sort by field and direction
3. Pagination