Sync Waves and Hooks

Sync waves are used in Argo CD to order how manifests are applied to the cluster.

Resource hooks break up the delivery of these manifests into different phases.

Using a combination of sync waves and resource hooks, you can control how your application rolls out.

This example will take you through the following steps:

Using sync waves to order deployment
Exploring resource hooks
Using sync waves and hooks together

The sample application that we will deploy is a TODO application with a database and, apart from deployment files, sync waves and resource hooks are used:

Using Sync Waves

A sync wave is a way to order how Argo CD applies the manifests that are stored in git. All manifests have a wave of zero by default, but you can set these by using the argocd.argoproj.io/sync-wave annotation.

Example:

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "2"

The wave can also be negative as well.

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-5"

When Argo CD starts a sync action, the manifests get placed in the following order:

The Phase that they’re in (we’ll cover phases in the next section)
The wave the resource is annotated in (starting from the lowest value to the highest)
By kind (Namespaces first, then services, then deployments, etc …)
By name (ascending order)

Read more about sync waves on the official documentation site.

Exploring Sync Wave Manifests

The sample application that you will deploy has several waves.

First, PostgreSQL with sync wave 0. It has a Deployment:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgresql
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  selector:
    matchLabels:
      app: postgresql
  template:
    metadata:
      labels:
        app: postgresql
    spec:
      containers:
        - name: postgresql
          image: quay.io/redhatdemo/openshift-pgsql12-primary:centos7
          imagePullPolicy: Always
          ports:
            - name: tcp
              containerPort: 5432
          env:
            - name: PG_USER_PASSWORD
              value: admin
            - name: PG_USER_NAME
              value: admin
            - name: PG_DATABASE
              value: todo
            - name: PG_NETWORK_MASK
              value: all

The PostgreSQL Service with sync wave 0:

---
apiVersion: v1
kind: Service
metadata:
  name: postgresql
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  selector:
    app: postgresql
  ports:
    - name: pgsql
      port: 5432
      targetPort: 5432

Second, Database table creation with sync wave 1:

apiVersion: batch/v1
kind: Job
metadata:
  name: todo-table
  annotations:
    argocd.argoproj.io/sync-wave: "1"
spec:
  template:
    spec:
      containers:
        - name: postgresql-client
          image: postgres:12
          imagePullPolicy: Always
          env:
            - name: PGPASSWORD
              value: admin
          command: ["psql"]
          args:
            [
              "--host=postgresql",
              "--username=admin",
              "--no-password",
              "--dbname=todo",
              "--command=create table Todo (id bigint not null,completed boolean not null,ordering integer,title varchar(255),url varchar(255),primary key (id));create sequence hibernate_sequence start with 1 increment by 1;",
            ]
      restartPolicy: Never
  backoffLimit: 1

The TODO application deployment with sync wave 2:

---
apiVersion: "v1"
kind: "ServiceAccount"
metadata:
  labels:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"
  name: "todo-gitops"
  annotations:
    argocd.argoproj.io/sync-wave: "2"
---
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
  labels:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"
  name: "todo-gitops"
  annotations:
    argocd.argoproj.io/sync-wave: "2"
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: "todo-gitops"
      app.kubernetes.io/version: "1.0.0"
  template:
    metadata:
      labels:
        app.kubernetes.io/name: "todo-gitops"
        app.kubernetes.io/version: "1.0.0"
    spec:
      containers:
      - env:
        - name: "KUBERNETES_NAMESPACE"
          valueFrom:
            fieldRef:
              fieldPath: "metadata.namespace"
        image: "quay.io/rhdevelopers/todo-gitops:1.0.0"
        imagePullPolicy: "Always"
        name: "todo-gitops"
        ports:
        - containerPort: 8080
          name: "http"
          protocol: "TCP"
      serviceAccount: "todo-gitops"

The TODO Service with sync wave 2:

---
apiVersion: "v1"
kind: "Service"
metadata:
  labels:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"
  name: "todo-gitops"
  annotations:
    argocd.argoproj.io/sync-wave: "2"
spec:
  ports:
  - name: "http"
    port: 8080
    targetPort: 8080
  selector:
    app.kubernetes.io/name: "todo-gitops"
    app.kubernetes.io/version: "1.0.0"

The TODO Route with sync wave 3:

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  labels:
    app: todo
  name: todo
  annotations:
    argocd.argoproj.io/sync-wave: "3"
spec:
  port:
    targetPort: 8080
  to:
    kind: Service
    name: todo-gitops
    weight: 100

First, the PostgreSQL Deployment will be applied. After that reports healthy, Argo CD will continue with the rest of resources.

Argo CD won’t apply the next manifest in a wave until the previous reports "healthy".

Using Resource Hooks

Now that you’re familiar with sync waves, we can begin exploring applying manifests in phases using resource hooks.

Controlling your sync operation can be further refined by using hooks. These hooks can run before, during, and after a sync operation. These hooks are:

PreSync - Runs before the sync operation. This can be something like a database backup before a schema change
Sync - Runs after PreSync has successfully ran. This will run alongside your normal manifests.
PostSync - Runs after Sync has ran successfully. This can be something like a Slack message or an email notification.
SyncFail - Runs if the Sync operation has failed. This is also used to send notifications or do other evasive actions.

To enable a sync, annotate the specific object manifest with argocd.argoproj.io/hook with the type of sync you want to use for that resource. For example, if you wanted to use the PreSync hook:

metadata:
  annotations:
    argocd.argoproj.io/hook: PreSync

You can also have the hooks be deleted after a successful/unsuccessful run.

HookSucceeded - The resource will be deleted after it has succeeded.
HookFailed - The resource will be deleted if it has failed.
BeforeHookCreation - The resource will be deleted before a new one is created (when a new sync is triggered).

You can apply these with the argocd.argoproj.io/hook-delete-policy annotation. For example:

metadata:
  annotations:
    argocd.argoproj.io/hook: PostSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded

Since a sync can fail in any phase, you can come to a situation where the application never reports healthy!

Although hooks can be any resource, they are usually Pods and/or Jobs.

To read more about resource hooks, consult the official documentation

Exploring Resource Hook Manifests

Take a look at this PostSync manifest which sends an HTTP request to insert a new TODO item:

apiVersion: batch/v1
kind: Job
metadata:
  name: todo-insert
  annotations:
    argocd.argoproj.io/hook: PostSync (1)
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
  template:
    spec:
      containers:
        - name: httpie
          image: alpine/httpie:2.4.0
          imagePullPolicy: Always
          command: ["http"]
          args:
            [
              "POST",
              "todo-gitops:8080/api",
              "title=Finish ArgoCD tutorial",
              "--ignore-stdin"
            ]
      restartPolicy: Never
  backoffLimit: 1

1	This means that this Job will run in the `PostSync` phase, after the application of the manifests in the `Sync` phase.

Since there is no deletion policy, this job will "stick around" after completion.

The execution order can be seen in the following diagram:

Deploying The Application

Taking a look at this manifest file: todo-application.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: todo-app
spec:
  destination:
    namespace: user$USERNUM-todo
    server: https://kubernetes.default.svc
  project: default
  source:
    path: documentation/modules/ROOT/examples/todo
    repoURL: https://github.com/openshiftdemos/openshift-gitops-workshop
    targetRevision: master
  syncPolicy:
    automated:
      prune: true
      selfHeal: false

Create this application:

sed 's/$USERNUM/%USERNUM%/' ~/openshift-gitops-workshop/documentation/modules/ROOT/examples/todo-application.yaml | oc apply -n user%USERNUM%-argocd -f -

application.argoproj.io/todo-app created

On the Argo CD WebUI, you should see another application appear.

Clicking on this "card" should take you over to the tree view.

Observe the sync process. You will see the order that the resource has been applied, first the namespace creation and last the creation of Route to access the application.

Once the application is fully synced. Take a look at the pods and jobs in the namespace:

oc get pods -n user%USERNUM%-todo

You should see that the Job is finished, but still there.

NAME                           READY   STATUS      RESTARTS   AGE
postgresql-599467fd86-cgj9v    1/1     Running     0          32s
todo-gitops-679d88f6f4-v4djp   1/1     Running     0          19s
todo-table-xhddk               0/1     Completed   0          27s

You can get the Route for your application from the topology view, or you can use the following CLI snippet to get the exact URL you need:

oc get route -n user%USERNUM%-todo todo -o jsonpath='{"http://"}{.spec.host}{"/todo.html\n"}'

You need to use /todo.html in the URL to access the application. It does not automatically redirect.

Your application should look like this:

The todo-insert Job is not shown as it was configured to be deleted if succeeded:

argocd.argoproj.io/hook-delete-policy: HookSucceeded