I recently migrated my blog from a small server hosted by a friend to a large server with 2TB of space. This was a technical necessity, I assure you.

While doing so, I decided to try migrating everything I ran to use Caddy, a reasonably performant nginx-like webserver written in Go. What do I mean by nginx-like? It's fast and sometimes very cryptic.

I use Caddy on my home network to proxy internal services to be world-accessible. It works pretty well for that! Turns out, probably 99% of what I use nginx for can also be done with Caddy.

Roadblocks

I extensively use Cloudflare as a reverse-proxy and CDN for my services. This is great, but has the downside of obscuring your visitor's IPs unless you take special care to correct it. Cloudflare provides the canonical client IP in the Cf-Connecting-Ip header, and also provides a list of ipv4 and ipv6 addresses that you can trust (their endpoints) - which is fantastic.

In nginx, I use the realip module in combination with a script that generates a proxy allow list to set the real IP globally. In Caddy, there were a lot of conflicting sources online saying either

1: Caddy doesn't support this and never will and you should instead figure it out in your application (lol good luck with that) or 2: Caddy does support this and even has a module that will do the heavy lifting.

In my real-world experience neither were completely true! Caddy 2.7 and above do support it (natively), and there is a module that will do this most of the way for you. You do have to either compile caddy yourself or use the caddy add-package command to download a modified caddy bin to add the module in, and for PHP you have to add env REMOTE_ADDR {client_ip} to the proxy/fastcgi configuration so that it doesn't send Cloudflare's IP to your script.

Not the most obvious solution, but oddly many folks online recommended just figuring it out in the client application... why is that something your application should even have to worry about? When I type a letter into a textbox, javascript doesn't have to figure out if your keyboard typed that letter or if it was pasted in when looking at the value of the textbox- that's an implementation detail that should be abstracted away before it gets to that point.

Anyway.

The error logging also leaves a lot to be desired, and I didn't see a good way to test the config before attempting to reload the service, but I'm sure those come down to configuration tweaks and scouring the docs.

Things I like better

Configuration is significantly easier.

• No more complex regex-looking (but not actually regex) filepaths everywhere and weird ssl blocks.
• Caddy auto-configures SSL - including fetching a new certificate.
• The cloudflare module automatically fetches the list of addresses CF uses, no more script required.
• You don't have to pass a million fastcgi parameters to PHP, the built in php_fastcgi directive (an alias of reverse_proxy with some default value set) handles it for you.
• Server blocks are defined by their hostnames or their listen addresses (e.x. andrewph.org { server config here } or :80, :8080 { default stuff here }) instead of being started off with server { every time
• Most configuration options are well-named and easy to find information on in their documentation.

Speed

I'm not benchmarking anything, but it seems pretty much the same. Memory usage seems around the same too (minimal). I've read online that out-of-the-box performance is around 4x faster for proxy workloads in Caddy than nginx, but I'm not sure if that's still true - my gut says that it probably is a bit faster, because it's a newer software written without years of code smell behind it.

Overall

It's a pain to have to hand-rewrite all of my site configs. It sucks having to learn new things.

But Caddy abstracts away a lot of things that I no longer will need to worry about in the foreseeable future. Things that would often be completely nonstandard in nginx, or require configuration hacks. That is a net decrease in time spent worrying about things that I shouldn't have to worry about. That's great.

Caddy has advanced very quickly over the last few years, and now supports many things developers claimed it would never support because "you shouldn't do it that way, fix it in the client application". This makes it hard to find solutions to more niche issues. However, overall, setting it up has been smoother than setting up nginx from scratch.

I rate Caddy a 4.8 out of 5. Works great.