Argo CD Self-Healing Capabilities

⏱️ Estimated Time: 5 Minutes

👩‍💻 Role: Cluster Administrator and/or Developers

Argo CD is capable healing resources when it detects configuration drift. For example, when a resource that should be present is missing it will be recreated by Argo CD. Another example is when a field such as spec.replicas on a Deployment has a value mismatch between what’s stored in the Git repository and the actual value set on the resource in the cluster.

If your Argo CD Application has the selfHeal property set to true then it will automatically detect and correct configuration drift for the resources it is managing. That means if someone accidentally runs a kubectl command against the wrong resource Argo CD will have your back and will restore the resource to the correct configuration!

Observing Argo CD Self-Healing

You currently have a Console Notification being managed by Argo CD. The selfHeal: true configuration is defined on the Application CR responsible for managing it. Go ahead an see what happens when you edit/delete the Console Notification resource using the oc or kubectl CLI:

  1. Login to the OpenShift Web Console as the opentlc-mgr user.

  2. Open the OpenShift Web Terminal using the >_ icon.

  3. Run the following command to list existing Console Notification resources:

    oc get consolenotification

  4. Run the following command to delete the welcome-banner Console Notification:

    oc delete consolenotification welcome-banner

  5. Depending on your timing the Console Notification will briefly disappear, and then immediately reappear.

    The reason that the Console Notification reappears is because the console-customisations Application in Argo CD has the selfHeal property set to true.

  6. List existing Console Notification resources again, and pay attention to the AGE column:

    oc get consolenotification

Did you notice that the age of the Console Notification has changed?

ex5.banner ages

When you deleted the Console Notification it was automatically recreated by Argo CD. This is reflected through the new value displayed in the AGE column.

Toggling the Self-Healing Behaviour

It’s possible to toggle the self-healing behaviour at any time, so long as you have the necessary permissions and access to the Argo CD dashboard/resources:

  1. Open the Argo CD dashboard and log in as the admin user. If you’ve lost the password, you can return to the Accessing the Cluster GitOps Dashboard section to obtain it.

  2. Select the console-customisations Application.

  3. Click the App Details button in the header. An overlay will appear.

  4. Scroll down and click the Disable button next to Self Heal. Click the OK button when asked to confirm the change.

    ex5 argocd disable healing

  5. Click the X icon in the top-right to close the modal window.

  6. Return to the OpenShift Web Console, and open the OpenShift Web Terminal.

  7. Return to the Web Terminal and run the following command to delete the ConsoleNotification:

    oc delete consolenotification welcome-banner

The ConsoleNotification will disappear, and won’t reappear this time. The Argo CD dashboard should be reporting that the console-customisations Application is "OutOfSync".

Use the Sync button to manually synchronise the Application and restore the banner. Consider re-enabling the self-healing configuration for peace of mind!