Posted on

Fun with Hetzner: How long does it take to fix a bug?

For some time I have a CX11 instance running at Hetzner. It is connected via vpn to my internal network. There was only IPv4 traffic sent over that connection and everything was fine. Until an idea occured to me: Why not use IPv6 over that tunnel as well? I mean, I did that on others servers, so why not on this one? Copy and paste of some lines of configuration and voila, I could “ping6” from the vpn server to the client. That was easy …

But suddenly my nagios went red because the external IPv6 connection was gone. Did I do some routing wrong? Stopping openvpn and everything was fine again. So I deactivated IPv6 and wondered why this tiny little server behaves different from any other.

After some time I found this article and I can confirm that Hetzner was not able to fix their bug after about two years of reporting it. Well done Hetzner!

Posted on

Let other devices use my own NTP server

I have these fine set-top boxes here, that try to synchronize their time with some external NTP servers.

The names of the NTP servers are coded into the firmware and can not be changed in the network settings menu. They are called ntp1.technibutler.de, ntp2.technibutler.de and ntp3.technibutler.de. Though they are already Stratum 2 servers, I would rather use my own, local DCF77 radio clock. Obviously it makes no sense to contact some server in the wide internet to get information that is already available locally.

Luckily those servers are just used for time synchronization and nobody wants to get web pages from them or wants to send emails to them. So all that needs to be done is to redefine their address resolution in DNS.

In a first step, I configure my own DNS server. The example below are config files for bind9. Any other DNS server should work as well, just pretend that you are authorized to answer queries for the technibutler NTP servers. As long as there is no DNSSEC or secure NTP involved, everything is fine.

First I need to define the different zones. As there might be other services within the technibutler.de zone, that I still want to use, I will define an extra zone for each hostname of the NTP servers. 

;
$TTL    86400
@       IN      SOA     ntp1.technibutler.de. redefined-dns.alteholz.de. (
                              1         ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                          86400 )       ; Negative Cache TTL
;
@       IN      NS      localhost.
@       IN      A       10.10.10.1

;
$TTL    86400
@       IN      SOA     ntp2.technibutler.de. redefined-dns.alteholz.de. (
                              1         ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                          86400 )       ; Negative Cache TTL
;
@       IN      NS      localhost.
@       IN      A       10.10.10.1

;
$TTL    86400
@       IN      SOA     ntp3.technibutler.de. redefined-dns.alteholz.de. (
                              1         ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                          86400 )       ; Negative Cache TTL
;
@       IN      NS      localhost.
@       IN      A       10.10.10.1

I store those configs in /etc/bind/redefined/db.ntp1.technibutler.de, /etc/bind/redefined/db.ntp3.technibutler.de and /etc/bind/redefined/db.ntp3.technibutler.de. The only IP address that is needed in these files are the actual IP address of my local NTP server. As I just have only one, all NTP servers from technibutler.de need to point to this address.

Now I have to tell bind that my zones are the master zone. This is done in /etc/bind/redefined/redefined-zones.conf:

zone "ntp1.technibutler.de" {
   type master;
   file "/etc/bind/redefined/db.ntp1.technibutler.de";
};

zone "ntp2.technibutler.de" {
   type master;
   file "/etc/bind/redefined/db.ntp2.technibutler.de";
};

zone "ntp3.technibutler.de" {
   type master;
   file "/etc/bind/redefined/db.ntp3.technibutler.de";
};

And last but not least I have to tell bind9 to load this config during startup. So I add a line:

include "/etc/bind/redefined/redefined-zones.conf";

at the beginning of /etc/bind/named.conf.local

And voila, before that configuration:

$ nslookup ntp1.technibutler.de
Server:         10.10.10.254
Address:        10.10.10.254#53

Non-authoritative answer:
Name:   ntp1.technibutler.de
Address: 62.138.2.9

and after that configuration:

$ nslookup ntp1.technibutler.de
Server:         10.10.10.254
Address:        10.10.10.254#53

Non-authoritative answer:
Name:   ntp1.technibutler.de
Address: 10.10.10.1

After the configuration of your DNS server is done, you just need to point the set-top boxes or any other device in your home network to your own DNS server. You can either deliver this information via “option domain-name-servers” with DHCP, or manually put your DNS server in the network settings of your device.

Posted on

APU and Debian

I just got an APU1D4 made by PC Engines. I bought it from a German retailer called VARIA System GmbH. They are also located in Chemnitz, so at least I could support the local economy. I purchased a bundle consisting of mainboard, case, power supply and 16GB SSD. The board has 4GB RAM and three network adapters and shall replace my old PC that I use as router to the internet.

As there is no VGA/HDMI output, the first hurdle was organizing a null-modem cable. Of course I could have prepared the SSD on another PC, but I wanted to try PXE. After finding the cable on the ground of a box, deeply buried under other boxes, I could start.

The DHCP server got an entry

host apu1d4 {
  hardware ethernet 00:0d:b9:42:a0:e8;
  fixed-address apu1d4;
  option broadcast-address 10.42.255.255;
  option routers 10.42.10.1;
  next-server 10.42.10.1;
  filename "pxelinux.0";
}

and the TFTP server got a file …/tftp/pxelinux.cfg/01-00-0d-b9-42-a0-e8

default install
label install
        menu label ^Install
        menu default
        kernel debian-installer/amd64/linux
        append initrd=debian-installer/amd64/initrd.gz --- vga=off console=ttyS0,115200n8

The files debian-installer/amd64/linux and debian-installer/amd64/initrd.gz are the normal debian installer files obtained from the official Debian servers.

That’s it, the installer starts, spits its output over the serial line and I can install the system. Great! Thanks DebianInstaller team. Why couldn’t everything be always so easy?

Posted on

Moving WordPress to another server

Today I moved this blog from a vServer to a dedicated server. The migration went surprisingly smooth. I just had to apt-get install the Debian packages apache2, mysql-server and wordpress. Afterwards only the following steps were necessary:

  • dumping the old database with basically just one command:

    mysqldump -u$DBUSER -p$DBPASS –lock-tables=false $DBNAME > $DBFILE

  • creating the database on the new host:

    CREATE DATABASE $DBNAME;
    \r $DBNAME
    GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER ON $DBNAME TO ‘$DBUSER’@’localhost’ IDENTIFIED BY ‘$DBPASS’;
    GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER ON $DBNAME.* TO ‘$DBUSER’@’localhost’ IDENTIFIED BY $DBPASS’;
    FLUSH PRIVILEGES;

  • importing the dump with something like:

    mysql –user=$DBUSER –password=$DBPASS $DBNAME < $DBFILE

and almost done …

Finally some fine tuning of /etc/wordpress/htaccess and access rights of a few directories to allow installation of plugins. As I wanted to clean up my wp-content-directory, I manually reinstalled all plugins instead of just copying them. Thankfully all of the important plugins store their data in the database and all settings survived the migration.

Posted on

What should be done to replace a faulty harddisk?

I am taking care of several dedicated servers hosted at different providers. As these servers are running 24/7 and have lots of things to write to and read from disk, from time to time a disk fails and has to be replaced. As there are RAIDs in these servers, this is no problem. Quite accidentally three disks at three different providers failed within a short time, and this is the story of their replacement:

  1. Server4You: I informed the support of the bad drive and asked what I need to do for a replacement. After a short time I was told to show part of the syslog, note the serial number of the faulty device and tell when the server might be switched of (the drives are not hot pluggable). At the given time nagios complained about a missing host. After about 15 minutes later everything was fine again and the RAID was syncing.
    downtime of host: 15min, total working time spent: 20min, only two people involved
    Great service!

  2. Hetzner: I informed the support of the bad drive and asked what I need to do for a replacement. After a short time I was told to show part of the syslog, note the serial number of the faulty device and tell when the server might be switched of (the drives are not hot pluggable). At the given time nagios complained about a missing host. After about 15 minutes later everything was fine again and the RAID was syncing.
    downtime of host: 15min, total working time spent: 20min, only two people involved
    Great service!

    (both are really almost identical)

  3. Strato: I informed the support of the bad drive and asked what I need to do for a replacement. After a short time employe1 told me to show part of the syslog and note the serial number of the faulty device. In response to those data employe2 told me that it is not possible to replace a single disk of the RAID. Instead the complete server(!!) needs to be replaced. I asked whether he was joking, but he confirmed that the answer of employe1 was wrong. I really need to click here and there on the customer service webpage to request a new installation of the server and activate a checkbox to request the exchange of the hardware.
    Ok, after thinking about my options I returned to the webpage and wanted to activate that checkbox. It was gone! My next email was answered by employe1: She is very sorry but she could not answer my email because I sent it from an unauthorized address. Btw. it was the same address that I used before and employe1 already sent an answer to!
    Anyway, maybe their webinterface can be used to send authenticated emails. Really, I got an answer from employe3 saying that I need to perform a hardware test to get my checkbox back. There are two versions, one lasting 2 hours and the second lasting up to 12 hours. During that time the server is not reachable. Ok, I needed that checkbox so I started the test. The next morning I was told that everything is fine with the hardware. Strange enough that checkbox appeared again. So I was finally able to use the new hardware and start to install the new system.
    downtime of host: about 12 hours, total working time spent: 6 hours, four people involved

    Maybe there are good reasons for such a procedure. From the customers point of view this is a total desaster. I think you can guess who will not rent out the next servers.