Why HTTP-01 Fails You on Private Laravel Deployments

The first time HTTP-01 silently fails on a private-network deployment, you spend an embarrassing amount of time staring at Certbot output wondering what you did wrong. You didn't do anything wrong — the challenge type itself is the problem. DNS-01 is the fix, and once you wire it up properly it's actually more elegant than the HTTP approach anyway.

What the ACME Challenge Problem Actually Is

Let's back up. Let's Encrypt (and any ACME-compatible CA) has to prove you control the domain before handing you a certificate. HTTP-01 does this by placing a token file under /.well-known/acme-challenge/ on your web server and then having the CA's servers fetch it over the public internet.

That's the trap: the CA's servers fetch it. Not your browser. Not your CI runner. The Let's Encrypt validation infrastructure — which lives out on the internet — has to be able to reach your server on port 80.

If your server is behind a VPN, sitting on a private subnet, or is an internal staging host that deliberately has no public ingress, that fetch never succeeds. Certbot returns a vague authorization error, and you start questioning your nginx config when the issue has nothing to do with nginx.

I hit this exact situation last year running a Laravel app for a biotech client. The app lived on a private AWS subnet — no public IP, all traffic routed through a VPN gateway. Totally reasonable security posture for healthcare-adjacent work. HTTP-01 was dead on arrival. We needed wildcard certs for subdomains (think *.internal.example.com) anyway, and wildcards require DNS-01 regardless of network topology — HTTP-01 can't issue them at all.

How DNS-01 Actually Works

Instead of placing a file on your web server, DNS-01 tells you to place a specific TXT record at _acme-challenge.yourdomain.com. Let's Encrypt queries DNS — which is always public, by design — reads that TXT record, and issues the cert. Your server never needs to be reachable from the internet.

The flow:

1. Certbot requests an authorization from the ACME server
2. ACME returns a token and tells you to publish _acme-challenge.yourdomain.com as a TXT record containing a derived key
3. You (or a hook script) create that DNS record
4. You tell ACME to validate
5. ACME queries DNS, finds the record, issues the cert
6. You clean up the TXT record

Step 3 is where automation comes in. For a one-off cert you can do this manually. For renewals every 60-90 days, you need a Certbot DNS plugin or a hook script that talks to your DNS provider's API.

The Setup I Actually Use

Most of my clients use either Cloudflare or Route 53. Certbot has first-party plugins for both. Here's the Cloudflare path, which I reach for most often.

Install Certbot and the plugin:

sudo apt install certbot python3-certbot-dns-cloudflare

Create a Cloudflare API token. Give it the Zone:DNS:Edit permission scoped to the specific zone. Don't use your global API key — scope it tight.

Drop the credentials file somewhere only root reads:

sudo mkdir -p /etc/letsencrypt/secrets
sudo tee /etc/letsencrypt/secrets/cloudflare.ini > /dev/null <<EOF
dns_cloudflare_api_token = YOUR_TOKEN_HERE
EOF
sudo chmod 600 /etc/letsencrypt/secrets/cloudflare.ini

Issue the wildcard cert:

sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials /etc/letsencrypt/secrets/cloudflare.ini \
  --dns-cloudflare-propagation-seconds 30 \
  -d "example.com" \
  -d "*.example.com"

That --dns-cloudflare-propagation-seconds 30 flag tells Certbot how long to wait after creating the TXT record before telling ACME to validate. Default is 10 seconds, which is often not enough if your DNS provider has any propagation lag. I've bumped this to 60 on providers that are slower than Cloudflare.

When this succeeds you get a wildcard cert at /etc/letsencrypt/live/example.com/. Renewal is automatic via the systemd timer or cron job Certbot installs — the plugin handles the DNS dance on every renewal without you touching anything.

Wiring the Cert into Laravel / Nginx

The cert path is the same whether you used HTTP-01 or DNS-01, so your nginx config doesn't change much:

server {
    listen 443 ssl;
    server_name app.example.com;

    ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    root /var/www/app/public;
    index index.php;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ \.php$ {
        fastcgi_pass unix:/run/php/php8.3-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        include fastcgi_params;
    }
}

Because it's a wildcard cert, the same fullchain.pem / privkey.pem pair covers every subdomain. If you're running multiple Laravel apps on the same server under different subdomains — say, a client portal at portal.example.com and an admin panel at admin.example.com — they all share the one cert. One renewal, everything stays green.

In your Laravel .env you'll want:

APP_URL=https://app.example.com
SESSION_SECURE_COOKIE=true

If you're behind a load balancer or reverse proxy that terminates SSL, add the trusted proxies in app/Http/Middleware/TrustProxies.php:

protected $proxies = '*'; // or specify your proxy IPs
protected $headers =
    Request::HEADER_X_FORWARDED_FOR |
    Request::HEADER_X_FORWARDED_HOST |
    Request::HEADER_X_FORWARDED_PORT |
    Request::HEADER_X_FORWARDED_PROTO;

Without this, request()->secure() returns false even over HTTPS, which breaks mixed-content warnings and asset URL generation in production.

The Gotchas That Will Bite You

Propagation time is not optional. I've seen people set --dns-cloudflare-propagation-seconds 10 and watch it fail intermittently. The ACME servers are distributed and sometimes query a nameserver that hasn't gotten the update yet. 30 seconds works reliably for Cloudflare. For Route 53 I use 60. For slower registrar DNS (some clients inherit old setups), I've needed 120.

Rate limits apply to wildcard issuance too. Let's Encrypt allows 50 certificates per registered domain per week. If you're scripting cert issuance in a dev loop, you'll burn through that. Use the staging environment (--staging flag) until your automation is solid:

sudo certbot certonly \
  --staging \
  --dns-cloudflare \
  --dns-cloudflare-credentials /etc/letsencrypt/secrets/cloudflare.ini \
  -d "*.example.com"

Staging certs aren't trusted by browsers but the issuance flow is identical. Test there first.

CAA records can block issuance. If your DNS zone has a CAA record restricting issuance to a specific CA and it doesn't include letsencrypt.org, you'll get an authorization error that looks like a challenge failure. Check:

dig CAA example.com

If you're getting a cert from Let's Encrypt, you need either no CAA records or one that includes 0 issue "letsencrypt.org".

Certbot's renewal timer needs to reach the DNS API too. This sounds obvious but I've caught it on a locked-down box where outbound HTTP was filtered: the renewal cron runs fine but can't reach the Cloudflare API, so the cert silently goes stale. Make sure your firewall allows the server to call out to api.cloudflare.com (or Route 53's endpoints). Test it:

sudo certbot renew --dry-run

Run that right after setup. Don't wait until 30 days before expiry to find out renewal is broken.

The credentials file permissions matter. Certbot's Cloudflare plugin will refuse to run if the credentials file is world-readable. chmod 600 and owned by root. If you're running Certbot as a non-root user, adjust accordingly, but I generally don't — the cert key material is sensitive enough that I want root owning it.

When I'd Reach for DNS-01 (and When I Wouldn't)

I reach for DNS-01 in every one of these situations:

• The server has no public ingress (private subnet, VPN-only, firewalled)
• I need a wildcard cert (no choice — DNS-01 is mandatory)
• Multiple subdomains on one server and I don't want to manage per-subdomain certs
• CI/CD environments that spin up preview deployments on dynamic subdomains

The only time HTTP-01 is simpler is a single public-facing server with one domain, where Certbot's --nginx or --apache plugin can handle everything automatically without touching DNS. For that case, HTTP-01 is fine. But the moment you have any network complexity or need wildcards, DNS-01 is the right tool.

If your DNS provider doesn't have a Certbot plugin and doesn't have a usable API, you're looking at manual renewal or a hook script that uses their web interface through something like Selenium. I've done it. It's miserable. That's actually a reasonable signal to switch DNS providers — Cloudflare's free tier handles this perfectly and their API is one of the cleaner ones I work with regularly.

Closing

HTTP-01 is fine for the simple case, but "simple case" covers less ground than you'd think once you're running real infrastructure with any network boundaries. DNS-01 with a provider plugin takes maybe 20 minutes to set up correctly, then it runs unattended for years. I've had setups on client servers auto-renewing for three years without a single intervention. That's the kind of boring reliability I want from certificate management.