Posted on

Random numbers from Linux kernel

The Linux kernel provides two devices that create some random data:

  • /dev/random
  • /dev/urandom

In this article I would like to sched some light on their usability.

Getting data

In a first step I totally neglect their mode of operation but just concentrate on their output. Doing something like

dd if=/dev/random of=random.1k bs=1 count=1000

for 1k, 10k and 100k with kernel 2.6.25, 2.6.32 and 3.2.0 on /dev/random and /dev/urandom should result in 18 files with data.

Unfortunately creating these files takes some time:


------------------------------------------------------------------
Kernel 2.6.25
/dev/urandom
1000+0 records in
1000+0 records out
1000 bytes (1.0 kB) copied, 0.00850279 seconds, 118 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 0.0789053 seconds, 127 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 0.804752 seconds, 124 kB/s

/dev/random
1000+0 records in
1000+0 records out
1000 bytes (1,0 kB) copied, 188,884 seconds, 0,0 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 6109,87 seconds, 0,0 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 80509,7 seconds, 0,0 kB/s

------------------------------------------------------------------
Kernel 2.6.32
/dev/urandom
1000+0 records in
1000+0 records out
1000 bytes (1.0 kB) copied, 0.0110714 s, 90.3 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 0.111313 s, 89.8 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 1.10515 s, 90.5 kB/s
1000000+0 records in
1000000+0 records out
1000000 bytes (1.0 MB) copied, 11.1315 s, 89.8 kB/s

/dev/random
1000+0 records in
1000+0 records out
1000 bytes (1.0 kB) copied, 62.7861 s, 0.0 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 41183.7 s, 0.0 kB/s

------------------------------------------------------------------
Kernel 3.2.0
/dev/urandom
1000+0 records in
1000+0 records out
1000 bytes (1.0 kB) copied, 0.00949647 s, 105 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 0.127453 s, 78.5 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 0.988632 s, 101 kB/s
1000000+0 records in
1000000+0 records out
1000000 bytes (1.0 MB) copied, 10.3351 s, 96.8 kB/s

/dev/random
1000+0 records in
1000+0 records out
1000 bytes (1,0 kB) copied, 188,692 s, 0,0 kB/s
10000+0 records in
10000+0 records out
10000 bytes (10 kB) copied, 1238,26 s, 0,0 kB/s
100000+0 records in
100000+0 records out
100000 bytes (100 kB) copied, 49876,2 s, 0,0 kB/s

So as a first result, not every machine is able to create enough random numbers from /dev/random within a reasonable timeframe. /dev/urandom is able to deliver much more data.

Quality of data

In order to analyse the quality of all those data, I am using software from the Debian package ent. The output from this program is:

  • entropy (8 bits per byte is random)
  • compression rate (0% is random)
  • arithmetic mean value of data bytes (127.5 is random)
  • error of Monte Carlo value for Pi (0% is random)
  • serial correlation coefficient (0.0 is random)

I will compare the output of the device files with data extracted from a real random number generator. It is an entropy key from Simtec Electronics. This device is very well supported in Linux and has an own package in Debian.

filename bytes of random data entropy compression rate [%] mean value Pi error [%] serial correlation
kernel-2.6.25-random.1k 1000 7.805056 2 122.2400 1.82 -0.009945
kernel-2.6.25-random.10k 10000 7.982797 0 128.3138 0.19 0.019648
kernel-2.6.25-random.100k 100000 7.998234 0 127.4827 0.20 -0.003984
kernel-2.6.25-urandom.1k 1000 7.809595 2 127.0670 5.85 -0.014163
kernel-2.6.25-urandom.10k 10000 7.983436 0 126.2238 2.18 0.002686
kernel-2.6.25-urandom.100k 100000 7.998327

0 127.5812 0.42 0.000916
kernel-2.6.25-urandom.1m 1000000 7.999830 0 127.4333 0.02 -0.000883
kernel-2.6.32-random.1k 1000 7.793478 2 128.0320 2.01 -0.012834
kernel-2.6.32-random.10k 10000 7.981851 0 126.4048 0.04 -0.010927
kernel-2.6.32-urandom.1k 1000 7.816192 2 125.9590 4.31 0.018713
kernel-2.6.32-urandom.10k 10000 7.981499 0 127.1751 1.03 -0.004903
kernel-2.6.32-urandom.100k 100000 7.998210 0 127.8787 1.20 0.002980
kernel-2.6.32-urandom.1m 1000000 7.999809 0 127.4078 0.04 0.001283
kernel-3.2.0-random.1k 1000 7.821790 2 129.0400 3.36 0.039336
kernel-3.2.0-random.10k 10000 7.983251 0 127.3707 1.72 -0.008163
kernel-3.2.0-random.100k 100000 7.998081 0 127.1491 0.23 -0.005288
kernel-3.2.0-urandom.1k 1000 7.786879 2 127.8330 5.85 -0.067891
kernel-3.2.0-urandom.10k 10000 7.981188 0 127.8730 3.02 -0.006596
kernel-3.2.0-urandom.100k 100000 7.998690 0 127.6806 0.09 -0.001264
kernel-3.2.0-urandom.1m 1000000 7.999840 0 127.4574 0.17 -0.000390
ekg.1k 1000 7.829743 2 127.0100 4.89 -0.021692
ekg.10k 10000 7.982590 0 128.5926 0.96 -0.018205
ekg.100k 100000 7.998119 0 127.3629 0.52 0.000735
ekg.1m 1000000 7.999819 0 127.4686 0.22 -0.001522

So as a first conclusion, there is not much difference between /dev/urandom, /dev/random and an EKG. Up to now I would not mind to use /dev/urandom for example to generate short living session keys in an https-connection. But I am sure that there are more sophisticated tests to get the quality of randomness, so expect more to come in this blog. Of course any hints and tipps are always welcome.

Posted on

exim4 and catchall mailbox

Of course there are millions of posts with similar content. But instead of storing a bookmark in one browser, I prefer to collect such knowledge at a central place.

In case you are working with Debian Wheezy and exim4 and want to create a mailbox, that gets all emails to unknown addresses, the following has to be done:

  1. edit /etc/exim4/exim4.conf.template and look for router/400_exim4-config_system_aliases
  2. change line data = ${lookup{$local_part}lsearch{/etc/aliases}} to
    data = ${lookup{$local_part}lsearch*{/etc/aliases}}
  3. add all valid users to /etc/aliases (like heinz: heinz)
  4. call update-exim4.conf
  5. restart exim4 /etc/init.d/exim4 restart

Disadvantage:

  • You need to insert all local users to /etc/aliases, but this could be managed during user creation
  • This does not work in case you set dc_use_split_config to ‘true’
Posted on

What should be done to replace a faulty harddisk?

I am taking care of several dedicated servers hosted at different providers. As these servers are running 24/7 and have lots of things to write to and read from disk, from time to time a disk fails and has to be replaced. As there are RAIDs in these servers, this is no problem. Quite accidentally three disks at three different providers failed within a short time, and this is the story of their replacement:

  1. Server4You: I informed the support of the bad drive and asked what I need to do for a replacement. After a short time I was told to show part of the syslog, note the serial number of the faulty device and tell when the server might be switched of (the drives are not hot pluggable). At the given time nagios complained about a missing host. After about 15 minutes later everything was fine again and the RAID was syncing.
    downtime of host: 15min, total working time spent: 20min, only two people involved
    Great service!

  2. Hetzner: I informed the support of the bad drive and asked what I need to do for a replacement. After a short time I was told to show part of the syslog, note the serial number of the faulty device and tell when the server might be switched of (the drives are not hot pluggable). At the given time nagios complained about a missing host. After about 15 minutes later everything was fine again and the RAID was syncing.
    downtime of host: 15min, total working time spent: 20min, only two people involved
    Great service!

    (both are really almost identical)

  3. Strato: I informed the support of the bad drive and asked what I need to do for a replacement. After a short time employe1 told me to show part of the syslog and note the serial number of the faulty device. In response to those data employe2 told me that it is not possible to replace a single disk of the RAID. Instead the complete server(!!) needs to be replaced. I asked whether he was joking, but he confirmed that the answer of employe1 was wrong. I really need to click here and there on the customer service webpage to request a new installation of the server and activate a checkbox to request the exchange of the hardware.
    Ok, after thinking about my options I returned to the webpage and wanted to activate that checkbox. It was gone! My next email was answered by employe1: She is very sorry but she could not answer my email because I sent it from an unauthorized address. Btw. it was the same address that I used before and employe1 already sent an answer to!
    Anyway, maybe their webinterface can be used to send authenticated emails. Really, I got an answer from employe3 saying that I need to perform a hardware test to get my checkbox back. There are two versions, one lasting 2 hours and the second lasting up to 12 hours. During that time the server is not reachable. Ok, I needed that checkbox so I started the test. The next morning I was told that everything is fine with the hardware. Strange enough that checkbox appeared again. So I was finally able to use the new hardware and start to install the new system.
    downtime of host: about 12 hours, total working time spent: 6 hours, four people involved

    Maybe there are good reasons for such a procedure. From the customers point of view this is a total desaster. I think you can guess who will not rent out the next servers.

Posted on

Debian Med advent calendar

I would like to anounce the Debian Med advent calendar 2012. Just like last year the Debian Med team starts a bug squashing event from the December 1st to 24th. Every day at least one bug from the Debian BTS should be closed. Especially RC bugs for the oncoming Debian release (Wheezy) or bugs in one of the packages maintained by Debian Med shall be closed. Anyone shall be called upon to fix a bug or send a patch. Don’t hestitate, start to squash :-).

Posted on

DOPOM: a56 – Motorola DSP56001 assembler

Since I first looked at the list of orphaned Debian packages (available at http://www.debian.org/devel/wnpp/orphaned) some time ago, the package a56 has been the lonely leader of the list.

This package contains a freeware assembler for the 56000 architecture. These chips have been very popular in the 1980s (used in NeXT, Atari Falcon and SGI Indigo Workstations).
Updated versions are still used in today’s devices like some mobile phones (-> http://www.freescale.com/webapp/sps/site/homepage.jsp?code=563XXGPDSP)

So, being a bit nostalgic, I adopted this package and brought it to shape. There was even a small bug that I was able to close.

Posted on

BOM: overflow in ent

Recently I got a bug report for package ent. The internal counter of processed bytes has just type long. In case you feed enough bytes to ent, there will be an overflow after about half an hour (of course that depends on your type of CPU, the bug was reported on architecture i386).

As modern C (C99) introduced a new type long long, I changed the type of some variables from simple long to unsigned long long. The overflow disappeared for now, but it will reappear just some trillion bytes later.

So, are there any recommendations on how to handle such a situation better?

Posted on

DSOM: vera++ – Programmable verification and analysis tool for C++

This software is a programmable tool for verification, analysis and transformation of C++ source code.
It is mainly an engine that parses C++ source files and presents the result of this parsing to scripts in the form of various collections. Afterwards the scripts are actually performing the requested tasks.

This is a link to the Debian PTS page.

Posted on

DOPOM: greylistd – Greylisting daemon for use with Exim 4

I was really amazed that a package like greylistd does not have an active  maintainer anymore.

Further looking at popcon, this package is only installed on just a few computers. The reason might be that it is only installed on servers that do not take part in popcon.

Anyway, this package needs a maintainer and here I am.

My first upload to ‘experimental’ mainly takes care of lintian warnings and closes a few bugs:

  • #375504: don’t expire entries every check:  the fix was contributed by Steven A. Reisman and verified by Jason  Cormie.
  • #585231: do not use Python strings exceptions anymore

The next step will be to take care of bugs with patches and than reduce the number of other bugs.

Please feel free to help fixing bugs or volunteer for becoming a comaintainer :-).

Here you can find the PTS page.

Posted on

BOM: libctl

During my adoption of setserial, I was able to close 9 bugs for that package. So the compulsory exercise is already finished and the free program can start.

As a new version of libctl and meep unexpectedly appeared, the BOM of July shall be the new upload of libctl. Unfortunately the soname changed and the library package needs to go through the new-queue. So the bug is resolved but not yet closed.