Skip to content

Your Server Has No Public IP: Serving a Custom Domain with TLS Anyway

I ran curl ifconfig.me on four different servers and got the same address back four times. That single result invalidated the entire approach I had spent three days validating on EC2. This is the full path from that discovery to three services running on my own subdomains with valid certificates and zero inbound ports.

Try It Yourself

This tunnel setup is live on all three service machines right now.

Launch the CI/CD Stack ↗ and watch cloudflared in action on Jenkins, SonarQube, and Nexus at once.

The Setup That Worked, and Why

Before moving anything to the platform, iximiuz Labs, I validated the architecture on three EC2 instances, one service each: Jenkins, SonarQube, Nexus. Every node got Nginx as a reverse proxy on port 80 and a Let's Encrypt certificate via Certbot.

That worked because EC2 gives every instance a public IP, which enabled a specific chain of assumptions:

  • A DNS A record could point at each instance's address
  • Certbot's HTTP-01 challenge could reach port 80 from outside to prove domain ownership
  • Let's Encrypt issued a certificate
  • Traffic flowed Browser → DNS → Public IP → Nginx :80 → service port

Three days of work produced a set of Nginx configs, systemd units and startup scripts that I expected to lift and drop somewhere cheaper.

The Discovery

I copied the same configuration onto a four-node playground. Services started. Nginx answered on port 80. Health checks passed. Everything looked identical from inside the machines.

DNS and TLS were completely dead.

The iximiuz Labs platform showed no public IP anywhere in its interface, which I initially read as a UI gap rather than a fact. So I checked directly, on all four nodes:

curl -s ifconfig.me
148.113.47.48
148.113.47.48
148.113.47.48
148.113.47.48

Four machines. One address.

What NAT Actually Does Here

The iximiuz Labs platform runs each playground as a group of Firecracker microVMs on a bare-metal host. Every VM in a playground attaches to a bridge network on that host, and bridges belonging to different playgrounds are isolated with network namespaces. All outbound traffic from every VM leaves through a single shared NAT gateway.

VM 1 (172.16.0.2) ─┐
VM 2 (172.16.0.3) ─┤──► NAT gateway ──► 148.113.47.48 ──► Internet
VM 3 (172.16.0.4) ─┘

This is not a limitation someone forgot to fix. It is how the isolation model works, and the same shape appears behind CGNAT at home, on most corporate networks, and on any platform where you do not own the edge.

From inside, each VM has a private address and can reach its neighbours by hostname. From outside, there is one address and no way to say which VM a request is meant for.

Five Things That Do Not Work

I tried or seriously evaluated all of these before accepting that the problem was structural.

Approach Why it fails
A record pointing at the shared NAT address All four VMs sit behind it. Nothing in the request distinguishes which one should receive it, and there is no port forwarding to disambiguate.
Port forwarding on the gateway Requires control of the NAT gateway. On a managed platform you do not have it.
Dynamic DNS Solves changing addresses, not missing ones. Still needs one routable address per machine.
Certbot HTTP-01 Validation requires Let's Encrypt to make an inbound request to port 80 on your host. Behind NAT that request never arrives.
Self-signed certificates Produces working TLS and a browser warning for every visitor. Fine for a scratch box, not for anything you link to.

The Certbot failure is worth showing because it misleads you into debugging DNS:

FAILED: Challenge did not complete successfully.
detail: DNS problem: SERVFAIL looking up A for your-domain.com

The message points at DNS. The actual cause is that there is no address to put in the A record in the first place.

The Assumption Underneath All Five

Every approach above assumes the same thing: a request originates on the internet, finds your server's address, arrives at a port, and your server answers.

Behind NAT, step two cannot happen. No amount of configuration on your side fixes a missing inbound path, because the missing piece is not on your side.

The fix is to stop waiting for inbound traffic and reverse the direction. Your server initiates an outbound connection to something that already has a public address, and that something holds the connection open. When a request arrives at the intermediary, it travels back down the connection your server opened.

Outbound works through NAT by default. That is what NAT is for.

Cloudflare Tunnel

cloudflared is a daemon that runs on your server and maintains a persistent outbound connection to Cloudflare's edge network. In this stack it ships preinstalled in every service image rather than being added by hand; how that image is built is a separate story, covered in Building an OCI Image That Boots as a microVM, Not a Container.

Browser ──► Cloudflare edge (jenkins.example.com, TLS terminated here)
                    │  persistent outbound connection, opened by the server
              cloudflared on jenkins-server
              Nginx :80 ──► Jenkins :8080

A request to jenkins.example.com resolves to Cloudflare. Cloudflare matches the hostname to the tunnel your server registered, pushes the request down that open connection, and cloudflared hands it to localhost:80.

TLS terminates at the edge with a Cloudflare-managed certificate, so Certbot never enters the picture and there is no renewal to automate.

Property How it is satisfied
NAT traversal The connection is outbound, which NAT permits by default
Certificate management Issued and renewed by Cloudflare at the edge
No HTTP-01 challenge Domain ownership is proven by the tunnel token, not by an inbound request
Custom subdomains Each tunnel maps to one hostname, and Cloudflare writes the CNAME

Setting It Up

Create the tunnel. In the Cloudflare dashboard, go to Networks then Connectors, create a tunnel, choose Cloudflared as the connector type, and name it. Cloudflare then shows an install command containing the tunnel token.

Install the connector. On the target server:

sudo cloudflared service install eyJhIjoiZXlK...

That installs cloudflared as a systemd service and starts it. The token is written into the unit file, which makes it a credential worth protecting:

sudo systemctl status cloudflared
# active (running)

sudo stat -c '%U %a' /etc/systemd/system/cloudflared.service
# root 600

Anyone holding that token can connect to Cloudflare's edge as your tunnel. Treat it the way you would treat an SSH private key.

Add the route. Back in the dashboard, under the tunnel, add a published application route:

Field Value
Subdomain jenkins
Domain example.com
Service HTTP
URL localhost:80

Cloudflare creates a CNAME automatically. No A record, no address anywhere.

Verify:

curl -I https://jenkins.example.com
# HTTP/2 200, valid Cloudflare TLS certificate

Why the Tunnel Points at Nginx, Not the Service

The route target is localhost:80, which is Nginx, rather than localhost:8080, which is Jenkins. That indirection is deliberate and it pays for itself four ways.

Nginx gives you request buffering, timeout control and access logging that the tunnel does not provide. It exposes a /health endpoint that answers 200 even when the backend is down, so you can distinguish a dead tunnel from a dead application. It normalises the entry point to port 80 across every server regardless of what the service behind it listens on. And when a service port changes, only the Nginx upstream changes, while the tunnel configuration stays untouched.

The Jenkins configuration also needs real work beyond a plain proxy_pass, because Jenkins uses WebSockets for its CLI and agent connections:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      "";
}

upstream jenkins {
    server 127.0.0.1:8080 fail_timeout=0;
    keepalive 32;
}

Alongside proxy_buffering off so console output streams rather than arriving in blocks, and dedicated locations for /cli and /jnlpJars/. The full configurations are in the repository: Jenkins, SonarQube, Nexus.

One Tunnel Per Service

I run three tunnels rather than one shared tunnel with three routes.

Tunnel Server Hostname
jenkins-lab jenkins-server jenkins.example.com
sonarqube-lab sonarqube-server sonar.example.com
nexus-lab nexus-server nexus.example.com

Restarting one connector affects one service. Each tunnel carries its own token, so a leaked credential exposes one service rather than three. And each service can be taken down or rebuilt without coordinating with the others. Cloudflare does not charge for tunnels, so the only cost of this split is three dashboard entries instead of one.

What the Security Posture Actually Is

This is the part I did not expect to like as much as I do.

Jenkins binds to 127.0.0.1:8080, so it is unreachable from another machine even on the private network. Nginx binds to 0.0.0.0:80, but the host has no public address, so that port is unreachable from the internet regardless. There are no inbound firewall rules to write because there is no inbound path to permit.

The consequence is that the origin cannot be scanned. There is no address to point a scanner at. Every request that reaches the application has passed through Cloudflare's edge first, which means rate limiting, geo-blocking and Zero Trust access policies are available as edge configuration rather than as something you install and maintain on the box.

Compared with the EC2 setup this replaced, where each instance had a public address, an open port 80, a security group to maintain and a certificate to renew, the tunnel version has a smaller attack surface and less to operate.

Where This Applies

Nothing above is specific to the iximiuz Labs platform. The same technique works for a home lab behind CGNAT, a Raspberry Pi on a residential connection, a machine on a corporate network where you cannot request a firewall change, or any environment where the thing you need is a routable address and the thing you have is not.

The general lesson is narrower than the tooling. When inbound connectivity is impossible, stop trying to fix inbound connectivity. Reverse the direction and let something with a public address hold the connection open on your behalf.


Series: Building a Self-Hosted CI/CD Stack from Scratch (Part 1 of 6)