Reconciliation Loop

One AppReconciler handles every App — the flow below tracks operator/internal/controller/app_controller.go.

The Loop

┌──────────────────────────────────────────────────────────┐
│                    Reconcile(app)                         │
│                                                           │
│  1. Fetch App — deleted → finalizer cleanup               │
│  2. Resolve source:                                       │
│     Git   → clone/pull, get sha + rendered object hash    │
│     Image → resolve digest, render Deployment+Service…    │
│  3. Decide to apply?                                      │
│     freeze            → skip sync, health only            │
│     pending operation → consume it                        │
│     automated + changed → apply                           │
│     requiresApproval    → stage as Held                   │
│  4. Apply (SSA, forced) + prune if enabled                │
│  5. Health check rendered set on destination              │
│  6. Status writeback                                      │
│     revision/objectHash/resources/conditions/opState      │
│  7. Notify (Slack) if configured                          │
│  8. Requeue — 60s normal / 30s error                      │
└──────────────────────────────────────────────────────────┘

Step-by-Step

1. Fetch the App

var app v1alpha1.App
if err := r.Get(ctx, req.NamespacedName, &app); err != nil {
    return ctrl.Result{}, client.IgnoreNotFound(err)
}

The finalizer prunes managed objects on deletion — a deleted App cleanly removes its rendered set.

2. Resolve the Source

The rendered set is hashed — status.objectHash tracks what was actually applied, so spec edits trigger a resync even when the upstream sha didn't move.

3. Gate Checks

In order:

• spec.freeze → skip everything but health checks (the CR still reconciles; nothing applies)
• Pending spec.operation → consume it (Sync/Deploy/Rollback/Restart/Approve), record outcome in status.operationState
• syncPolicy.requiresApproval → stage the move as Held with the staged revision; a human releases via POST /approve
• syncPolicy.automated → apply when observedRevision != status.revision or objectHash differs
• Otherwise → status stays OutOfSync and the loop just re-checks health

selfHeal is declared but never read — no live-vs-desired diff loop exists.

4. Apply

r.Applier.Apply(ctx, targetClient, rendered, app):

• Server-side apply with fieldManager: wardn-operator and force — Wardn's rendered fields are taken over even when another manager owns them
• Managed-by label cd.wardn.space/app: <name> stamped on every applied object
• Prune — with prune: true, objects in destination.namespace carrying that label but absent from the rendered set are deleted

destination.cluster resolution: empty → in-cluster client; a name → GET /internal/clusters/{name}/kubeconfig on the server (cached).

5. Health

r.Health.Check(ctx, targetClient, rendered):

• Real logic for Deployment/StatefulSet/DaemonSet/Job/Pod
• Everything else → Healthy if it exists, Missing if not
• Aggregate order: Degraded > Missing > Progressing > Unknown > Healthy

See Health Checks.

6. Status Writeback

app.Status = v1alpha1.AppStatus{
    SyncStatus:        syncStatus,
    HealthStatus:      healthStatus,
    HealthReason:      reason,
    Revision:          resolvedSha,      // applied
    ObservedRevision:  upstreamSha,      // newest seen
    PreviousRevisions: ledger,          // rollback history
    ObjectHash:        renderedHash,
    Resources:         perResource,
    URL:               ingressURL,       // image sources
    OperationState:    opResult,
    Conditions:        conditions,
}

There's no Redis pub/sub or SSE — status lives on the CR; the UI polls the API.

7. Notify

r.Notify.Dispatch(ctx, &app, success, err) — Slack POST on onSyncSuccess/onSyncFailure. onHealthDegraded 📋 declared, never read.

8. Requeue

RequeueAfter: 60s normal, 30s on error. spec.operation writes fire immediately via generation bump — no annotation watching.

Error Handling

Observing the Loop

# Live status
kubectl --kubeconfig k3s.yaml get app my-app -n org-my-org -o yaml | \
  yq '.status | {syncStatus, healthStatus, revision, operationState}'

# Operator logs
kubectl --kubeconfig k3s.yaml logs deployment/wardn-operator -n <ns> -f --tail=100

Metrics — 📋 planned

The --metrics-bind-address (:8080) binds controller-runtime's metrics; there are no wardn_* custom series today. Planned: sync duration, op outcome counters, health gauge.

Related

• Architecture — components
• Sync Policies — automation options
• CRD Reference — App types