Posted on

HowTo build a kernel

I write this article mainly to be able to remember all steps in a few months.

– Get kernel: git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
This is the main kernel repository. Other repositories can be found at http://git.kernel.org/
For example the development of /dev/random takes place in git://git.kernel.org/pub/scm/linux/kernel/git/tytso/random.git

– As I don’t want to manually handle entries in grub, I want to use make-kpkg to build Debian packages.

– The package needs to be built as root, ‘–rootcmd fakeroot’ does not always work. Maybe there is some time to check this issue.

– Working with original sources instead of the Debian source package results in a plus sign added to the kernel version. This is done by one of the kernel scripts (scripts/setlocalversion)
For more info see comment at end of that script. I avoid this ‘+’ by doing something like: ‘export LOCALVERSION=”-ta-1″

– As /tmp nowadays is a bit small, you need to do somehting like ‘export TEMPDIR=/home/tmp’ or whatever suits your system

– Target ‘buildpackage’ calls ‘clean’ + ‘binary’
‘binary’ -> ‘binary-indep’ + ‘binary-arch’
‘binary-indep’ -> ‘kernel_source’, ‘kernel_manual’, ‘kernel_doc’
‘binary-arch’ -> ‘kernel_headers’, ‘kernel_image’

So for a normal builds, call ‘make-kpkg –initrd binary-arch’ or at least ‘make-kpkg –initrd kernel_image’

– In case of several cores call ‘make-kpkg -j 4 –initrd binary-arch’
(the blank between ‘j’ and ‘4’ is important)
The best results will be obtained if the given number equals the number of cores

– The old Laptop needs the following times to build the package:
make-kpkg –initrd binary-arch 91 minutes
make-kpkg -j 3 –initrd binary-arch 78 minutes
make-kpkg -j 4 –initrd binary-arch 59 minutes
make-kpkg -j 4 –initrd binary-arch &> log 57 minutes

– Check version of package in automatically build debian/changelog
The debian-directory can be rebuild by ‘make-kpkg debian’

– Move kernel package to Xen VM (grub-legacy should be installed, otherwise pygrub on wheezy dom0 is not able to start domU)
– dpkg -i
– edit /boot/grub/menu.lst (again a pygrub issue with some entries)

Posted on

Random numbers from Linux kernel

The Linux kernel provides two devices that create some random data:

  • /dev/random
  • /dev/urandom

In this article I would like to sched some light on their usability.

Getting data

In a first step I totally neglect their mode of operation but just concentrate on their output. Doing something like

dd if=/dev/random of=random.1k bs=1 count=1000

for 1k, 10k and 100k with kernel 2.6.25, 2.6.32 and 3.2.0 on /dev/random and /dev/urandom should result in 18 files with data.

Unfortunately creating these files takes some time:


------------------------------------------------------------------
Kernel 2.6.25
/dev/urandom
1000+0 records in
1000+0 records out
1000 bytes (1.0 kB) copied, 0.00850279 seconds, 118 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 0.0789053 seconds, 127 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 0.804752 seconds, 124 kB/s

/dev/random
1000+0 records in
1000+0 records out
1000 bytes (1,0 kB) copied, 188,884 seconds, 0,0 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 6109,87 seconds, 0,0 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 80509,7 seconds, 0,0 kB/s

------------------------------------------------------------------
Kernel 2.6.32
/dev/urandom
1000+0 records in
1000+0 records out
1000 bytes (1.0 kB) copied, 0.0110714 s, 90.3 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 0.111313 s, 89.8 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 1.10515 s, 90.5 kB/s
1000000+0 records in
1000000+0 records out
1000000 bytes (1.0 MB) copied, 11.1315 s, 89.8 kB/s

/dev/random
1000+0 records in
1000+0 records out
1000 bytes (1.0 kB) copied, 62.7861 s, 0.0 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 41183.7 s, 0.0 kB/s

------------------------------------------------------------------
Kernel 3.2.0
/dev/urandom
1000+0 records in
1000+0 records out
1000 bytes (1.0 kB) copied, 0.00949647 s, 105 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 0.127453 s, 78.5 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 0.988632 s, 101 kB/s
1000000+0 records in
1000000+0 records out
1000000 bytes (1.0 MB) copied, 10.3351 s, 96.8 kB/s

/dev/random
1000+0 records in
1000+0 records out
1000 bytes (1,0 kB) copied, 188,692 s, 0,0 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 1238,26 s, 0,0 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 49876,2 s, 0,0 kB/s

So as a first result, not every machine is able to create enough random numbers from /dev/random within a reasonable timeframe. /dev/urandom is able to deliver much more data.

Quality of data

In order to analyse the quality of all those data, I am using software from the Debian package ent. The output from this program is:

  • entropy (8 bits per byte is random)
  • compression rate (0% is random)
  • arithmetic mean value of data bytes (127.5 is random)
  • error of Monte Carlo value for Pi (0% is random)
  • serial correlation coefficient (0.0 is random)

I will compare the output of the device files with data extracted from a real random number generator. It is an entropy key from Simtec Electronics. This device is very well supported in Linux and has an own package in Debian.

filename bytes of random data entropy compression rate [%] mean value Pi error [%] serial correlation
kernel-2.6.25-random.1k 1000 7.805056 2 122.2400 1.82 -0.009945
kernel-2.6.25-random.10k 10000 7.982797 0 128.3138 0.19 0.019648
kernel-2.6.25-random.100k 100000 7.998234 0 127.4827 0.20 -0.003984
kernel-2.6.25-urandom.1k 1000 7.809595 2 127.0670 5.85 -0.014163
kernel-2.6.25-urandom.10k 10000 7.983436 0 126.2238 2.18 0.002686
kernel-2.6.25-urandom.100k 100000 7.998327

0 127.5812 0.42 0.000916
kernel-2.6.25-urandom.1m 1000000 7.999830 0 127.4333 0.02 -0.000883
kernel-2.6.32-random.1k 1000 7.793478 2 128.0320 2.01 -0.012834
kernel-2.6.32-random.10k 10000 7.981851 0 126.4048 0.04 -0.010927
kernel-2.6.32-urandom.1k 1000 7.816192 2 125.9590 4.31 0.018713
kernel-2.6.32-urandom.10k 10000 7.981499 0 127.1751 1.03 -0.004903
kernel-2.6.32-urandom.100k 100000 7.998210 0 127.8787 1.20 0.002980
kernel-2.6.32-urandom.1m 1000000 7.999809 0 127.4078 0.04 0.001283
kernel-3.2.0-random.1k 1000 7.821790 2 129.0400 3.36 0.039336
kernel-3.2.0-random.10k 10000 7.983251 0 127.3707 1.72 -0.008163
kernel-3.2.0-random.100k 100000 7.998081 0 127.1491 0.23 -0.005288
kernel-3.2.0-urandom.1k 1000 7.786879 2 127.8330 5.85 -0.067891
kernel-3.2.0-urandom.10k 10000 7.981188 0 127.8730 3.02 -0.006596
kernel-3.2.0-urandom.100k 100000 7.998690 0 127.6806 0.09 -0.001264
kernel-3.2.0-urandom.1m 1000000 7.999840 0 127.4574 0.17 -0.000390
ekg.1k 1000 7.829743 2 127.0100 4.89 -0.021692
ekg.10k 10000 7.982590 0 128.5926 0.96 -0.018205
ekg.100k 100000 7.998119 0 127.3629 0.52 0.000735
ekg.1m 1000000 7.999819 0 127.4686 0.22 -0.001522

So as a first conclusion, there is not much difference between /dev/urandom, /dev/random and an EKG. Up to now I would not mind to use /dev/urandom for example to generate short living session keys in an https-connection. But I am sure that there are more sophisticated tests to get the quality of randomness, so expect more to come in this blog. Of course any hints and tipps are always welcome.

Posted on

exim4 and catchall mailbox

Of course there are millions of posts with similar content. But instead of storing a bookmark in one browser, I prefer to collect such knowledge at a central place.

In case you are working with Debian Wheezy and exim4 and want to create a mailbox, that gets all emails to unknown addresses, the following has to be done:

  1. edit /etc/exim4/exim4.conf.template and look for router/400_exim4-config_system_aliases
  2. change line data = ${lookup{$local_part}lsearch{/etc/aliases}} to
    data = ${lookup{$local_part}lsearch*{/etc/aliases}}
  3. add all valid users to /etc/aliases (like heinz: heinz)
  4. call update-exim4.conf
  5. restart exim4 /etc/init.d/exim4 restart

Disadvantage:

  • You need to insert all local users to /etc/aliases, but this could be managed during user creation
  • This does not work in case you set dc_use_split_config to ‘true’
Posted on

What should be done to replace a faulty harddisk?

I am taking care of several dedicated servers hosted at different providers. As these servers are running 24/7 and have lots of things to write to and read from disk, from time to time a disk fails and has to be replaced. As there are RAIDs in these servers, this is no problem. Quite accidentally three disks at three different providers failed within a short time, and this is the story of their replacement:

  1. Server4You: I informed the support of the bad drive and asked what I need to do for a replacement. After a short time I was told to show part of the syslog, note the serial number of the faulty device and tell when the server might be switched of (the drives are not hot pluggable). At the given time nagios complained about a missing host. After about 15 minutes later everything was fine again and the RAID was syncing.
    downtime of host: 15min, total working time spent: 20min, only two people involved
    Great service!

  2. Hetzner: I informed the support of the bad drive and asked what I need to do for a replacement. After a short time I was told to show part of the syslog, note the serial number of the faulty device and tell when the server might be switched of (the drives are not hot pluggable). At the given time nagios complained about a missing host. After about 15 minutes later everything was fine again and the RAID was syncing.
    downtime of host: 15min, total working time spent: 20min, only two people involved
    Great service!

    (both are really almost identical)

  3. Strato: I informed the support of the bad drive and asked what I need to do for a replacement. After a short time employe1 told me to show part of the syslog and note the serial number of the faulty device. In response to those data employe2 told me that it is not possible to replace a single disk of the RAID. Instead the complete server(!!) needs to be replaced. I asked whether he was joking, but he confirmed that the answer of employe1 was wrong. I really need to click here and there on the customer service webpage to request a new installation of the server and activate a checkbox to request the exchange of the hardware.
    Ok, after thinking about my options I returned to the webpage and wanted to activate that checkbox. It was gone! My next email was answered by employe1: She is very sorry but she could not answer my email because I sent it from an unauthorized address. Btw. it was the same address that I used before and employe1 already sent an answer to!
    Anyway, maybe their webinterface can be used to send authenticated emails. Really, I got an answer from employe3 saying that I need to perform a hardware test to get my checkbox back. There are two versions, one lasting 2 hours and the second lasting up to 12 hours. During that time the server is not reachable. Ok, I needed that checkbox so I started the test. The next morning I was told that everything is fine with the hardware. Strange enough that checkbox appeared again. So I was finally able to use the new hardware and start to install the new system.
    downtime of host: about 12 hours, total working time spent: 6 hours, four people involved

    Maybe there are good reasons for such a procedure. From the customers point of view this is a total desaster. I think you can guess who will not rent out the next servers.

Posted on

Debian Med advent calendar

I would like to anounce the Debian Med advent calendar 2012. Just like last year the Debian Med team starts a bug squashing event from the December 1st to 24th. Every day at least one bug from the Debian BTS should be closed. Especially RC bugs for the oncoming Debian release (Wheezy) or bugs in one of the packages maintained by Debian Med shall be closed. Anyone shall be called upon to fix a bug or send a patch. Don’t hestitate, start to squash :-).

Posted on

DOPOM: a56 – Motorola DSP56001 assembler

Since I first looked at the list of orphaned Debian packages (available at http://www.debian.org/devel/wnpp/orphaned) some time ago, the package a56 has been the lonely leader of the list.

This package contains a freeware assembler for the 56000 architecture. These chips have been very popular in the 1980s (used in NeXT, Atari Falcon and SGI Indigo Workstations).
Updated versions are still used in today’s devices like some mobile phones (-> http://www.freescale.com/webapp/sps/site/homepage.jsp?code=563XXGPDSP)

So, being a bit nostalgic, I adopted this package and brought it to shape. There was even a small bug that I was able to close.

Posted on

BOM: overflow in ent

Recently I got a bug report for package ent. The internal counter of processed bytes has just type long. In case you feed enough bytes to ent, there will be an overflow after about half an hour (of course that depends on your type of CPU, the bug was reported on architecture i386).

As modern C (C99) introduced a new type long long, I changed the type of some variables from simple long to unsigned long long. The overflow disappeared for now, but it will reappear just some trillion bytes later.

So, are there any recommendations on how to handle such a situation better?

Posted on

New version of chktex in Debian unstable and experimental

Just recently I heard of a change of the maintainer of chktex. Apparently development goes on as there is already version 1.7.1 available. Even for the old 1.6 branch there have been some bugfixes.

So I uploaded 1.6.6 to unstable and 1.7.1 to experimental. Keep on testing :-).

Posted on

DCF77 computer clock

Some time ago I read about a lottery conducted by the German company Meinberg Funkuhren GmbH & Co. KG. The prizes should be seven DCF77 computer clocks and three GPS time receivers. After the drawing the winners must use their clocks to contribute to the NTP Project. In order to win one just had to give away an email address.

After having done so, I almost forgot the lottery and was really surprised to get a notification that I won a PZF180PEX together with an external antenna and accessories.

After some days everything arrived and I tried to install the hardware. Unfortunately the conditions for reception of the DCF77 signal were really bad at the originally planned location. So I had to install a spare computer at another place with better conditions.

As there is no Debian package available for the driver, I had to compile the software on my own. This raised some issues, which could be resolved rather quickly by the Meinberg support team. At the end I got a beta version of the new software and everything was fine.

Afterwards I reconfigured other external servers to use that internal clock as well. As a result the NTP pool got no new stratum 1 server, but in compensation three stratum 2 servers.

Anyway, thanks alot to the Meinberg team for their generous donor.