Notice to my future self: If you add zones to bind by rndc addzone please remember that those zones will be stored in /var/cache/bind/*.nzf. If you have to change your nameservers, you also need to adapt the also-notify list in all zones. If you forget one zone and there is one unused ip address in that list, all slaves will get the notification, start the transfer but the update won’t happen and the old data remain on the slave.
This sounds really crazy, but think about April 2018, when the challenge for your letsencrypt certificate was added to the master server but never reached the slaves. The log was full of
ERROR: Challenge is invalid! (returned: invalid) (result: {
"type": "dns-01",
"status": "invalid",
"error": {
"type": "urn:acme:error:unauthorized",
"detail": "Incorrect TXT record \"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\" found at _acme-challenge.xxxxxxxxx",
"status": 403
},