One Chart, a Ladder of Dependencies¶
The same Helm chart deploys my orders service six different ways: in memory, on PostgreSQL with in-memory messaging, on PostgreSQL and RabbitMQ backed by local disk, the same pair on EBS, both pointed at external endpoints, or PostgreSQL with AWS SQS. Six values files, one chart, no forks.
I got there by being wrong about what portability means.
Try It Yourself
Three runnable scenarios in helmfile/, and every overlay under src/*/chart/.
Portability Is Not Two Configurations¶
My first assumption was that portability meant a laptop config and a cloud config. Write both, keep them in sync, done.
That model breaks the moment you ask an ordinary question: what is the fastest way to check that a chart renders and a pod becomes healthy, before any storage exists? Neither of two configurations answers it. One needs a local-path provisioner, the other needs an EBS CSI driver, and both need a database to come up first.
What the application actually has is a ladder. Every rung is a real place to stand, and which one you want depends on what you are testing rather than on where you are.
For the orders service the rungs look like this:
| Overlay | Persistence | Messaging | What it is for |
|---|---|---|---|
values-01-in-memory | H2 | in-memory | Does the chart render and the pod start, with zero infrastructure |
values-02-postgresql-ephemeral-msg-in-memory | PostgreSQL, no PVC | in-memory | Add a real database, still no storage layer |
values-03-postgresql-rabbitmq-pvc-baremetal | PostgreSQL on local-path | RabbitMQ | Full stack on a laptop cluster |
values-04-postgresql-rabbitmq-pvc-eks | PostgreSQL on gp3 | RabbitMQ | Same shape, cloud storage |
values-05-postgresql-rabbitmq-external | External endpoint | External broker | Nothing in cluster |
values-06-postgresql-pvc-eks-sqs | PostgreSQL on gp3 | AWS SQS | Managed messaging with IRSA |
The numbering is not decoration. Each file adds exactly one dependency to the one before it, so when rung four fails and rung three worked, the thing that broke is the storage class and not the application.
The bottom rung is worth defending specifically. values-01 has two keys in it. It is the only configuration that can tell you a chart is broken without any possibility that the cluster is broken instead, and that separation is worth a file.
Twenty Six Overlays, Not Three Environments¶
The ladder is per service, and the services do not have the same number of rungs because they do not have the same number of things to depend on:
| Service | Overlays | Axis |
|---|---|---|
| catalog | 5 | MySQL: in-memory, ephemeral, local-path, gp3, external |
| cart | 3 | DynamoDB: in-memory, DynamoDB Local in cluster, real AWS DynamoDB |
| orders | 6 | Database and messaging, varied independently |
| checkout | 4 | Redis: in-memory, local, ElastiCache, TLS |
| ui | 7 | Exposure and integrations, not persistence |
Twenty six files against five upstream charts that were never modified. Every overlay is a file added inside the chart directory, so helm pull on the upstream chart still produces the same thing it always did.
The UI is the odd one out and the reason is structural. It has no database, so its axis is not persistence at all: it is how the thing is exposed (clusterip, nodeport, loadbalancer, alb-ingress) plus two optional chat integrations. Same pattern, different question.
One overlay changes nothing at all, deliberately
catalog/values-in-memory.yaml sets provider: in-memory, which is already the chart default. Its own comment says so. I kept it because "the default happens to be what I want" and "I have chosen this" are different states, and only one of them survives an upstream chart bump. A no-op overlay is cheap insurance against a default moving underneath you.
Three Scenarios, Each Picking a Rung¶
The Helmfiles are thin. Each one selects a rung per service and nothing more:
baremetal-ephemeral | baremetal-persistent | eks | |
|---|---|---|---|
| catalog | mysql-ephemeral | mysql-pvc-baremetal | mysql-pvc-eks |
| cart | dynamodb-local | dynamodb-local | dynamodb-aws |
| orders | 02 ephemeral | 03 RabbitMQ, local-path | 06 SQS, gp3 |
| checkout | redis-local | redis-local | redis-local |
| ui | nodeport | nodeport | alb-ingress |
Two things stand out reading it as a grid rather than as three files.
Checkout never changes. The same local Redis in all three. That is not an oversight; a values-redis-aws-elasticache.yaml exists and is simply not wired into any scenario. It is a rung above where I stopped, and saying so is more useful than implying the cloud scenario is fully managed.
The EKS scenario still runs its databases in the cluster. MySQL and PostgreSQL both sit on gp3 PVCs. Only the cart's DynamoDB and the orders service's SQS messaging are genuinely AWS-managed. "Deployed to EKS" and "offloaded state to managed services" are separate claims, and only the first one is true here.
Release Order Is a Graph, Not a List¶
Five releases with a naive sequential ordering means catalog, then cart, then orders, then checkout, then ui, and you wait for each. But catalog and orders do not depend on each other at all.
Helmfile's needs: expresses the real shape:
catalogandordersstart togethercartwaits forcatalogcheckoutwaits fororders, because it needs the orders endpoint to existuiwaits for all four
The alternative is helmDefaults.wait at the top level, which makes everything wait for everything. That is correct and slower, and it hides which dependencies are real. Writing the graph out means the file documents the coupling, and a wrong edge shows up as a startup race rather than as a comment nobody reads.
The Overlay That Is Never Optional¶
The UI is a front door. It calls catalog, cart, orders and checkout over HTTP at startup, and without their addresses the pod comes up healthy and every page returns empty.
So values-endpoints.yaml is passed as a second -f in every scenario, alongside whichever exposure overlay is in play:
app:
endpoints:
catalog: http://catalog.catalog.svc.cluster.local:80
orders: http://orders.orders.svc.cluster.local:80
carts: http://cart-carts.cart.svc.cluster.local:80
checkout: http://checkout.checkout.svc.cluster.local:80
These are in-cluster DNS names, so they are identical in every environment and the file never needs a variant. That is the actual test for whether something deserves an overlay: does it change between scenarios. This does not, so it is one file used five times rather than five nearly identical files.
cart-carts is worth noticing. The service name carries the Helm release name as a prefix while the others do not, which is a chart-level inconsistency upstream. It is the kind of thing that costs twenty minutes if you assume the naming is uniform and type cart.cart.
When Not To Make a Values File¶
Checkout needs the orders endpoint, and it is passed inline instead:
One scalar, identical across all three scenarios. A dedicated values-orders-endpoint.yaml holding a single key would be a file to open, a file to keep in sync, and a file to wonder about later. set: says plainly that this is one value that never varies.
The line I ended up drawing: a values file when there is a choice between alternatives, set: when there is a constant that happens to be injected.
What Sits Above the Top Rung¶
The EKS rung uses real AWS services for two of the five, and the wiring for that lives outside the chart:
The chart does not create the service account and does not know about IAM. The account is created by eksctl create iamserviceaccount, which attaches a role with a policy scoped to one DynamoDB table and its indexes, and the chart simply uses it by name.
That split is deliberate and it is why the same chart still deploys on a laptop. A chart that owned the IRSA annotation would carry an AWS-shaped concept into an environment where it means nothing. The orders service gets the same treatment for SQS, and the queue feeds a Lambda that publishes to SNS, none of which the chart has any opinion about.
What the Ladder Actually Buys¶
Not portability in the abstract. Something narrower and more useful: when a deployment fails, the ladder tells you which rung introduced the failure.
A chart that only has a laptop mode and a cloud mode gives you one bisection step. Six rungs give you five, and each one isolates a single dependency. That turned out to matter more than any of the individual configurations, and it is the part I would build first next time rather than arriving at by accumulation.
Source¶
- The three scenarios, with the design notes on ordering
- Every chart and overlay, plus a per-service runbook in each chart directory
- Deployment runbook, including the IRSA bindings and the SQS to Lambda to SNS pipeline
Series: One Application, Four Ways (Part 4 of 4)
- Previous: An ACME Challenge That Could Not Reach Itself
- Full series index: Four Compute Models, One Artifact