Codeberg Lost Its Name, Not Its Servers
At 05:52 UTC on 18 November, codeberg.org stopped resolving. The servers kept running. No resolver on the internet had any reason to ask them a single question about the project, the repositories, or the releases inside them.
The public registry record for the domain carries a clientHold status, set by NameSilo, Codeberg’s registrar. That single EPP flag pulls the delegation out of the .org zone. Nothing else changes. The A record, the machine answering on 217.197.84.140, the Forgejo instance, the storage: all of it stays exactly where it was, and no resolver on the internet will ask about any of it. Depending on the resolver, clients get NXDOMAIN or SERVFAIL, and neither error points at a registrar.
Codeberg’s own note was blunt. Servers up, name missing, and adding 217.197.84.140 codeberg.org to /etc/hosts gets you back in. The follow-up says the name is back and the workaround is no longer needed, which is the closest thing to a root cause anyone got.
What they could rule out matters more than what they could confirm. Not a missed renewal. No suspicious logins in the registrar account’s history, and no entry in the domain’s own history that explains what happened to it or who ordered it.
That is the part I would design around.
Diagnosis order matters here, and most teams run it backwards. Curl the IP before you curl the name. If the IP answers on 443 with a valid certificate for the hostname, your platform is healthy and your namespace is not, and you have just skipped an hour of pointless container restarts. Then read the registry record. Then check whether your resolver sees a delegation at all. Logs come last, because the service never saw the request.
The name is the rented part
Self-hosting usually gets argued as an ownership claim. My box, my data, my rules. The box is yours. The name is not. It is a lease on a string, administered by a registrar, delegated by a registry, and either of those parties can end your reachability without touching one of your servers.
Short TTLs do not help. The record is absent rather than stale, so there is nothing to expire. DNSSEC is irrelevant for the same class of failure, since it protects answers against forgery, not against the parent zone declining to publish a delegation at all.
A clientHold is not a control you can audit. It is a registrar-side switch, and whoever flips it does not owe you a cause, a timestamp you can act on, or a ticket you can escalate inside your own incident window.
The hosts-file workaround deserves its own paragraph, because it looks like a fix and is not one. It works for the people who read the post and edit their own machine. It does not work for a fresh git clone on a CI runner, for mail, for ACME HTTP-01 renewal, or for anyone who arrives tomorrow. It is a diagnostic that proves the servers are fine, plus a stopgap for a handful of operators who already know the IP.
What actually shortens the outage
Uptime checks answer the wrong question. A monitor polling https://example.org from three regions tells you the site is down and nothing about why. A monitor polling the IP stays green through the entire incident, which is worse, because it hands you a false all-clear at the moment you need a signal. Poll the registry record instead, on a schedule, from a host that does not resolve through your own nameservers.
curl -s https://rdap.org/domain/example.org \
| jq -r '.status[]?, (.events[] | "\(.eventAction) \(.eventDate)")'
Alert on the diff, not the value. A new entry in status, a changed NS set, a registrar change, an expiry date that jumped. Those fields turn into outages, and all of them are readable over plain HTTPS with no credentials and no rate limit worth worrying about.
The second measure is a second name. A domain at a different registrar, pointing at the same frontend, printed in the README and in every release note as the fallback clone URL. One registration costs less than an hour of this incident, and it changes the failure from “we are gone” to “one of our two names is gone”.
Registrar hygiene is the boring half: hardware key on the account, no shared mailbox for recovery, a registry lock through the registrar where your TLD sells one. That blocks the failures caused by somebody logging in. It does not block a status change made by the registrar itself, which is the shape of what happened here. Which is why I would watch the record rather than trust the process.
Everything that resolves through the name goes with it. Clone URLs inside lockfiles, signing key fetches, advisory links your dependency scanner follows, CI badge endpoints, password reset mail. A name outage is not one service down, it is every reference to that service, and the count is always larger than the list you wrote down.
The test worth running on your own setup is cheap. Block your primary domain at the resolver and see what still works. Whatever breaks in those five minutes is the list you should be defending, and most of it will not be the thing you spent the last year hardening.