GitOps with Kubernetes: An ArgoCD Primer

The Problem with kubectl apply

For years, our deployment process looked something like this:

$ ssh into-bastion
$ kubectl apply -f deployment.yaml
$ kubectl rollout status deployment/app
$ # hope for the best

It worked, until it didn’t. The problems were predictable:

• No audit trail — who deployed what, when?
• Configuration drift — production slowly diverged from what was in git
• Manual rollbacks — reverting meant remembering what the previous state was

Enter GitOps

GitOps flips the model: instead of pushing changes to the cluster, you declare the desired state in git, and a controller pulls it into the cluster.

The core principles:

1. Git as the single source of truth for declarative infrastructure and applications
2. All changes via git — no more kubectl apply from laptops
3. Automated reconciliation — the controller continuously ensures the cluster matches git
4. Self-healing — manual changes get automatically reverted

Our ArgoCD Setup

We chose ArgoCD for its straightforward approach and excellent UI. Here’s the high-level architecture:

# application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-service
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/org/k8s-manifests
    targetRevision: main
    path: services/my-service/production
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Results

After 3 months of running GitOps:

• Deployment frequency went from 2/week to 5/day
• Failed deployments dropped by 80%
• Mean time to recovery decreased from 45min to 5min (just git revert)
• On-call engineers actually sleep now

Lessons Learned

• Start with a single non-critical service. Don’t migrate everything at once.
• Invest in your git repository structure early — it’s harder to reorganize later.
• selfHeal: true is powerful but can be surprising. Make sure your team understands what it does.
• Secrets management needs its own solution (we use Sealed Secrets).

GitOps isn’t magic, but it’s the closest thing to it in the Kubernetes ecosystem.