I’ve been struggling for over a week now to get my DE10-Nano board to boot a minimum Linux kernel with no dependencies to the onboard SD Card.

This post is how to get an embedded initial RAM disk or initramfs to boot and provide minimum application services.

Here’s my embedded “INITRD – How-To Guide”

After much teeth gnashing I finally worked out how to embed a RAM disk into the Linux kernel image, have it boot and run a static GLIB executable with no dependencies. It was a difficult journey that wasn’t helped by obscure, misleading and sometimes no kernel error messages. I’ve documented this process, not just for me, but for anyone else who missed the fine print in the ‘man’ pages.

The objective

I was trying to create a management executable that ran on the embedded ARM processor on my Terasic DE10-Nano. This process would listen for commands from the network and send signals appropriately across the FPGA general purpose bus, such as the reset signal. However, I couldn’t boot Linux from the SD Card, because I needed that to be accessible directly from the FPGA fabric. Linux didn’t take kindly to ripping out the SD card signals while the kernel was running, throwing a hard ‘Bus Error’ that was not trappable. I needed a way to boot Linux from U-Boot using only the kernel image and not requiring a SD card-based root file system. An initramfs seemed like the best option, but I struggled to get this to work. Here’s how I did it.

In Brief:

1. Download and install arm-gcc and arm-glibc for your build platform, if you’re cross-compiling. This how-to assumes we’re cross compiling.
2. Download a new kernel image from kernel.org
3. Create the default configuration for your hardware:
   1. make ARCH=arm socfpga_defconfig
      or just
   2. make defconfig (if you’re building for the build platform)
4. make ARCH=arm xconfig (or menuconfig)
5. Update the important settings:
   1. Under General→Initial RAM filesystem and RAM disk support, tick this option
   2. Fill in the new field “initramfs source file(s)” folder path – see below for structure.
   3. Make sure the compression you’re planning to use is supported.
   4. Under ‘Boot Options’ or sometimes ‘Process Type and Features’ check the ‘Built-in Kernel Command Line’ and add your start-up program:
      rdinit=/linuxrc
      This is only needed if the start program is not /init. The kernel will look for and execute /init if no options are provided. This caused me days of pain, see “Error Messages” below.
   5. Check the man page for initrd “man initrd”. This says:
      ….support for both “RAM disk” and “Initial RAM disk” (e.g., CONFIG_BLK_DEV_RAM=y and CONFIG_BLK_DEV_INITRD=y) must be compiled directly into the Linux kernel to use /dev/initrd.
      CONFIG_BLK_DEV_INITRD is managed by step (a) above.
      CONFIG_BLK_DEV_RAM is managed under “Device Drivers→Generic→Block Devices→RAM Block Device Support”. You can also elect how many default RAM disks to create. A value of 4 gives ram0/ram1/ram2/ram3.
   6. Finally, thank you to www.linux.com for their article on the early initramfs start up:
      https://www.linux.com/learn/kernel-newbie-corner-initrd-and-initramfs-some-unfinished-business . This is where I found the startup program /init was missing (or rdinit=). Here’s the kernel code that runs:

     /*
      * check if there is an early userspace init.  If yes, let it do all
      * the work
      */
      if (!ramdisk_execute_command)
          ramdisk_execute_command = "/init";
      if (sys_access((const char __user *) ramdisk_execute_command, 0) != 0) 
      {
          ramdisk_execute_command = NULL;
          prepare_namespace();
      }

6. Once the kernel config has been created, save the config file and exit xconfig/menuconfig.
7. Next, make -j 8 ARCH=arm CROSS_COMPILE=arm-linux-gnu- zImage

The resulting zImage contains the Linux kernel, the embedded initramfs including the system routines you want to run. Bear in mind that if the ‘init’ application exits, a kernel panic results.

Error Messages:

If the early user-space processes cannot find /init and you don’t provide an rdinit= parameter, then it looks as though the kernel can’t see the initramfs. This is the snippet of the error:

[    1.169024] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -2
[    1.170082] Please append a correct "root=" boot option; here are the available partitions:
[    1.171249] 0100            4096 ram0
[    1.171249]  (driver?)
[    1.172120] 0101            4096 ram1
[    1.172120]  (driver?)
[    1.172986] 0102            4096 ram2
[    1.172986]  (driver?)
[    1.173854] 0103            4096 ram3
[    1.173854]  (driver?)
[    1.174726] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    1.175876] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.19.9 #18
[    1.176714] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS ?-20180531_142017-buildhw-08.phx2.fedoraproject.org-1.fc28 04/01/2014
[    1.178471] Call Trace:
[    1.178823]  dump_stack+0x46/0x60
[    1.179297]  panic+0xdc/0x227
[    1.179718]  mount_block_root+0x191/0x23b
[    1.180285]  ? do_early_param+0x8c/0x8c
[    1.180823]  mount_root+0x10a/0x128
[    1.181317]  ? do_early_param+0x8c/0x8c
[    1.181853]  prepare_namespace+0x160/0x196
[    1.182427]  kernel_init_freeable+0x1e2/0x1f2
[    1.183035]  ? rest_init+0x9a/0x9a
[    1.183515]  kernel_init+0x5/0x100
[    1.183993]  ret_from_fork+0x35/0x40
[    1.184909] Kernel Offset: 0xbc00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[    1.186397] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---

In the first line, the kernel error shows VFS cannot open root device null, unknown block (0,0). Even adding root=/dev/ram0 or root=/dev/initrd or root=0x0100 did not fix the problem, and simply changed the block that it was trying to mount. What this error really means is that it can’t find an executable to run on the initramfs and so drops out of the early user stage to normal run-time stage. Unfortunately, because no root device was mounted and the initramfs has already been discarded, there is no longer any root device and the kernel panics. Either add rdinit=<executable> or place an executable in the root of the ramdisk image called /init and make sure it’s executable, with no dependencies.

Note on the /init process

The /init process or whatever you decide to call your management executable must be stand alone and never exit. While it’s possible to create dynamic linked executables, a static linked executable is preferred as it does not depend on any shared libraries like ld-linux.so or libc.so

However compiling a C program with -static will result in an error unless the static GLIBC library is installed. On Fedora, I installed:

glibc-static-2.28-23.fc29.x86_64

Then using the following to create a stand-alone, static init file:

gcc -o init -static helloworld.c

Note that this is an x86_64 glibc, not ARM. There are still some issues building static ARM linux executables on Fedora 29. Bare metal newlib builds fine as static.

A good candidate for “init” is a statically-linked busybox. Using the hash-bang magic header, we can execute a script to do basic system set up operations:

#!/sbin/busybox sh

/sbin/busybox mount -t proc none /proc
# Don’t exit, exec instead
exec <your process>

Note that this busybox script runs as process 1, so you can do pivot_root and exec the real distribution init as process 1. Remember, this script should never exit.

Typical File Tree:

drwxrwxr-x. 5 root root    4096 Dec 17 20:55 .
drwxrwxr-x. 2 root root    4096 Dec 17 11:22 ./dev
crw-------. 1 root  root     5, 1 Dec 17 11:18 ./dev/console
-rwxrwxr-x. 1 root root      66 Dec 17 16:29 ./linuxrc
drwxrwxr-x. 2 root root    4096 Dec 17 14:01 ./proc
drwxrwxr-x. 2 root root    4096 Dec 17 14:01 ./sbin
-rwxr-xr-x. 1 root root 1192728 Dec 17 11:25 ./sbin/busybox
lrwxrwxrwx. 1 root root       7 Dec 17 14:01 ./sbin/df -> busybox
lrwxrwxrwx. 1 root root       7 Dec 17 13:46 ./sbin/ln -> busybox
lrwxrwxrwx. 1 root root       7 Dec 17 13:46 ./sbin/ls -> busybox
lrwxrwxrwx. 1 root root       7 Dec 17 14:00 ./sbin/mkdir -> busybox
lrwxrwxrwx. 1 root root       7 Dec 17 13:46 ./sbin/mount -> busybox
lrwxrwxrwx. 1 root root       7 Dec 17 14:00 ./sbin/rm -> busybox
lrwxrwxrwx. 1 root root       7 Dec 17 14:00 ./sbin/rmdir -> busybox
lrwxrwxrwx. 1 root root       7 Dec 17 11:25 ./sbin/sh -> busybox
lrwxrwxrwx. 1 root root       7 Dec 17 13:46 ./sbin/umount -> busybox

Basic linuxrc:

#!/sbin/busybox sh

mount -t proc none /proc
mount -t devtmpfs none /dev
exec /sbin/setsid -c /sbin/sh

I found a great write-up here: https://wiki.gentoo.org/wiki/Custom_Initramfs

As always, this is a necessary diversion for my CPC2 project.