Tutorial: Deploying pgEdge Distributed Postgres with kind and a Helm Chart

You can deploy a fully distributed, multi-master pgEdge Distributed Postgres cluster easier than ever with kind (opens in a new tab) and a Helm Chart from pgEdge. The pgedge-helm repository contains Helm Charts, makefiles, and scripts that deploy Postgres in a sample cluster as a StatefulSet. A StatefulSet is used to describe an installation that requires properties such as:

• persistent storage
• stable network identities
• ordered deployment
• scaling

In this tutorial, we'll walk you through using the information in the pgedge-helm repo to deploy a simple but flexible pre-defined cluster topology with Helm. The topology we create is useful for evaluations and to manage simple workflows, and can act as a starting point for creating a more flexible topology.

Prerequisites:

Before starting, you should install:

• Docker Desktop
• kind
• kubectl
• make

If you decide to deploy a multi-zone cluster with the repo content, you'll also need:

• Cilium CLI
• subctl

After installing the prerequisites, clone the pgEdge/pgedge-helm repository (opens in a new tab). The command is:

git clone https://github.com/pgEdge/pgedge-helm.git

Then, navigate into the pgedge-helm/examples repository:

cd pgedge-helm/examples

Then, use make commands to deploy the defined cluster with kind. To create a local Kubernetes cluster with two pods, use the commands:

make single-up make single-install

You can monitor the deployment progress with the command:

kubectl get pods

Once deployed, the kubectl get pods command returns a description of your cluster along with pod names (in the NAME column) and the pod STATUS:

NAME       READY   STATUS    RESTARTS   AGE
pgedge-0   1/1     Running   0          6m19s
pgedge-1   1/1     Running   0          6m19s```

When the deployment is up and running, you will have two pods; you can connect to each pod with psql with the following commands. To connect to pgedge-0 (node 1):

kubectl --context kind-single exec -it pgedge-0 -- psql -U app defaultdb

psql (16.10)
Type "help" for help.
 
defaultdb=>

To connect to pgedge-1 (node 2) with psql use the command:

kubectl --context kind-single exec -it pgedge-1 -- psql -U app defaultdb

To simplify experimentation, connect to each pod in a different terminal window. Then, in pgedge-0, create a table:

CREATE TABLE public.users (id uuid default gen_random_uuid(), name text, primary key(id));

defaultdb=> CREATE TABLE public.users (id uuid default gen_random_uuid(), name text, primary key(id));
INFO:  DDL statement replicated.
CREATE TABLE

Verify that the public.users table was created on both pods with the psql meta command:

\d

defaultdb=> \d
                 List of relations
 Schema |          Name           | Type  | Owner  
--------+-------------------------+-------+--------
 public | pg_stat_statements      | view  | pgedge
 public | pg_stat_statements_info | view  | pgedge
 public | users                   | table | app
(3 rows)

You can test replication by adding rows to pgedge-0; this command adds 3 rows:

INSERT INTO users (name) VALUES ('Alice'),('Bob'),('Carol');

Confirm that the data was inserted correctly in pgedge-1, then do the same in pgedge-0:

SELECT * FROM users;

defaultdb=> SELECT * FROM users;
                  id                  |   name    
--------------------------------------+-----------
 073f865b-6acb-4c0a-be06-fe1eec0686be | Alice
 cfdb60e1-f50b-4ebb-b4d3-0a1e9ac10a83 | Bob
 438132cf-b3de-4f39-8089-51857d1e5b9e | Carol
(3 rows)

To exit psql, use the command:

\q

Clean Up

To remove the cluster, use the following command:

kind delete cluster --name $(kubectl config current-context | sed 's/^kind-//')

Deleting cluster "single" ...
Deleted nodes: ["single-control-plane" "single-worker" "single-worker2"]

Adding the Active Consistency Engine to your Pods

A powerful feature supported by pgEdge Distributed Postgres is the Active Consistency Engine (ACE). ACE is designed to detect and repair data drift across replicated nodes. It quickly and efficiently detects and repairs any data divergence that results from exceptions or infrastructure failures.

The following steps add ACE to your cluster. To install the ACE pod on the Kubernetes cluster, first spin up a cluster, and then add a pod runnning ACE:

kubectl apply -f ace/ace.yaml

You can check the cluster status with the command:

kubectl get pods

pod/ace configured
NAME       READY   STATUS    RESTARTS   AGE
ace        1/1     Running   0          118s
pgedge-0   1/1     Running   0          22m
pgedge-1   1/1     Running   0          22m

Next, we're going to start a bash shell inside of the ACE pod:

kubectl exec -it ace -- /bin/bash

Defaulted container "ace" out of: ace, merge-files (init)
bash-5.1$ 

Then, use ACE to find differences in tables:

./ace table-diff defaultdb public.users

✔ Cluster defaultdb exists
✔ Connections successful to nodes in cluster
✔ Table public.users is comparable across nodes
Estimated row count: 3
Getting primary key offsets for table...
Starting jobs to compare tables...
 
 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3/3  [ 0:00:00 < 0:00:00 , ? it/s ]
✔ TABLES MATCH OK
 
TOTAL ROWS CHECKED = 6
RUN TIME = 0.20 seconds

ACE confirms that our replication is working and our tables match.

ACE can examine and repair tables with millions of rows efficiently and flexibly, and can even be run automatically to monitor and repair conflicts. For more information about ACE, review the online docs.

Customizing a Container Deployment

You can use your choice of editor to review and modify the values.yaml file, specifying values that describe your cluster properties in key/value pairs. Please note that the values provided in the sample file are designed to work with the Makefile and scripts shared in the pgedge-helm repository; modifying some properties may require changes elsewhere within scripts.

Updating the values.yaml File

The first section of the values.yaml file describes properties that are applied globally to each pod:

The pgEdge section of the values.yaml file contains properties that define your Postgres installation and cluster deployment:

The service section of the values.yaml file describes the properties that describe your preferences for the service deployment: