Two cert-manager Solvers, and the Flag That Is Off By Default¶

cert-manager has two HTTP-01 solvers. Which one you need is not a preference, it is determined entirely by what is routing traffic in your cluster. Pick the wrong one and the challenge never gets served.

Pick the right one and it still may not work, because on a manifest install the Gateway API solver is compiled in and switched off.

What a Solver Actually Does¶

Both solvers exist to answer one question: when Let's Encrypt requests /.well-known/acme-challenge/<token>, how does that request reach a pod that knows the answer?

cert-manager's approach is the same either way. It creates a tiny Go HTTP server as a temporary pod, a Service in front of it on port 8089, and then one routing object appropriate to your ingress layer, all named cm-acme-http-solver-<hash>. When the challenge completes, all three are deleted.

The solver type only decides which routing object gets created.

Solver A: ingress¶

For ingress-nginx and anything else driven by Ingress resources.

solvers:
  - http01:
      ingress:
        ingressClassName: nginx

It creates a temporary Ingress with a rule matching the challenge path, pointing at the solver Service:

Ingress: cm-acme-http-solver-<hash>
  ingressClassName: nginx
  rules:
    - host: java-monolith.ibtisam-iq.com
      paths:
        - path: /.well-known/acme-challenge/<token>
          backend: cm-acme-http-solver-<hash>:8089

ingressClassName must match what your controller actually watches. If it does not, the Ingress is created, sits unclaimed, and routes nothing. The object exists and no controller has adopted it.

Solver B: gatewayHTTPRoute¶

For Gateway API implementations: NGINX Gateway Fabric, Envoy Gateway, and the rest.

solvers:
  - http01:
      gatewayHTTPRoute:
        parentRefs:
          - name: bankapp-gateway
            namespace: bankapp
            kind: Gateway
            group: gateway.networking.k8s.io

Same shape, different object: it creates a temporary HTTPRoute attached to the Gateway you name, rather than an Ingress.

group is easy to omit and worth not omitting. Left out, it defaults to the core API group, and Gateway does not live there. The parentRef then references a kind that does not exist where it is looking, and the route attaches to nothing.

The Flag That Is Off¶

This is the part that cost me time and it is not in the solver configuration at all.

Installing cert-manager from the raw manifest gives you the controller and every CRD, including the ones for the Gateway API solver. It does not enable Gateway API support. The gatewayHTTPRoute solver stays inert until the controller is started with an explicit flag.

So the sequence is: apply the manifest, write a perfectly correct ClusterIssuer with a gatewayHTTPRoute solver, apply it, and cert-manager accepts it. The ClusterIssuer is valid. It reports ready. Challenges are created and no HTTPRoute ever appears, because the controller responsible for that solver was never started.

kubectl patch deployment cert-manager -n cert-manager \
  --type=json \
  -p='[{
    "op": "add",
    "path": "/spec/template/spec/containers/0/args/-",
    "value": "--enable-gateway-api"
  }]'

Verify it took, on the deployment and in the logs:

kubectl get deployment cert-manager -n cert-manager \
  -o jsonpath='{.spec.template.spec.containers[*].args}' | tr ',' '\n'
# expect --enable-gateway-api

kubectl logs deploy/cert-manager -n cert-manager | grep -E "gateway-shim|Gateway API"
# "enabling the sig-network Gateway API certificate-shim and HTTP-01 solver"
# "starting controller" controller="gateway-shim"

The log line is the real confirmation. It tells you the shim started, which is the thing that creates HTTPRoutes.

Which One, and Why It Is Not a Choice¶

The solver has to match the routing layer, because it creates an object that only that layer understands. An Ingress in a cluster with no Ingress controller is inert. An HTTPRoute in a cluster with no Gateway controller is inert.

The last row is legal. A ClusterIssuer takes a list of solvers, and each can carry a selector, so a cluster mid-migration from Ingress to Gateway API can serve challenges through whichever layer owns a given hostname.

The Object Chain, and Where to Look When It Stalls¶

Both solvers sit inside a chain of five cert-manager objects, and knowing the order turns a vague "the certificate is not ready" into a specific question:

Certificate          <- you write this, or ingress-shim writes it for you
  -> CertificateRequest   <- one per issuance attempt
    -> Order              <- the ACME order with Let's Encrypt
      -> Challenge        <- one per domain on the certificate
        -> solver Pod + Service + Ingress/HTTPRoute

Each object is created by the one above it, so the lowest object that exists tells you how far the process got.

• No CertificateRequest: the Certificate is malformed, or the issuer it references does not exist.
• Order but no Challenge: cert-manager has not decided how to solve it. Usually a solver that does not match any configured selector.
• Challenge stuck at Presented: false: the solver could not create its routing object. This is the disabled-flag case, and the wrong-group case.
• Challenge at Presented: true, still pending: the object was created and the request cannot reach it. Now it is a networking problem, not a cert-manager one.

That last distinction is the one worth internalising. Presented: true means cert-manager has done everything it can do; the remaining failure is entirely outside it, which is why no amount of re-reading the ClusterIssuer helps once you are there.

kubectl get certificate,certificaterequest,order,challenge -n <namespace>

One command, and the answer is whichever row is missing.

The hostNetwork Escape¶

Both solvers assume the challenge request can physically reach the cluster. On bare metal with NodePort services, that assumption is often false, because nothing is bound to port 80 on the host.

The iptables answer redirects host ports to NodePorts. The other answer is to remove the translation entirely:

kubectl patch deployment ingress-nginx-controller -n ingress-nginx \
  --type=json \
  -p='[
    {"op": "add", "path": "/spec/template/spec/hostNetwork", "value": true},
    {"op": "add", "path": "/spec/template/spec/dnsPolicy",  "value": "ClusterFirstWithHostNet"}
  ]'

Now the proxy binds 0.0.0.0:80 and 0.0.0.0:443 directly. No NAT rules, nothing to persist across reboots, and the security group plus the process binding are the whole story.

Two conditions before doing it.

dnsPolicy: ClusterFirstWithHostNet is mandatory, not optional. With hostNetwork: true alone the pod inherits the node's /etc/resolv.conf and loses cluster DNS, so the proxy can no longer resolve the backend Services it is proxying to. The routing layer comes up healthy and cannot reach anything behind it.

Check the port is free first. Anything already bound to 80 makes the pod crash-loop on bind: address already in use:

sudo ss -tlnp | grep -E ':80|:443'

Between the two, hostNetwork is simpler and the better long-term answer on a node you intend to keep. iptables is reversible with one command and does not restart the data plane, which is what I wanted while still changing things.

The Order That Saves Time¶

1. Is the solver type right for the routing layer? An Ingress solver on a Gateway API cluster fails silently at the routing step.
2. Is the solver's controller actually enabled? Check the args and the logs, not the ClusterIssuer status.
3. Can the request physically arrive? curl the challenge path from outside; if it is refused, no solver configuration will help.
4. Then read the Challenge object, which is where most people start and where the least information is.

Three of the four are answered before touching cert-manager's own resources. That is usually the shape of it: the certificate is the last thing to fail and the last place to look.

Source¶

• The cert-manager overview, both solvers with the full object chain
• The Ingress solver walkthrough, including the hostNetwork path
• The Gateway solver debugging chain, when the request cannot arrive at all
• When HTTP-01 is not available, behind NAT