Tuesday 13 February 2018

Compile linux kernel from source & write a custom system call.

Stage 1: Compile Linux kernel from source


1.     A virtual machine with Linux base operating system, I am using Ubuntu 16.4 LTS as my base OS.
2.     Virtual Box or VMware Fusion
3.     Disk apace > 30GB because kernel takes more than 18Gb disk space.


1.     Use at-least 2 cores for your VM
2.     Use either Virtual box or VMware Fusion application to compile kernel, to avoid corruption your base operating system.

Steps followed to compile new kernel:

The main benefit of using custom Linux kernel over Distribution is that whenever a latest kernel is out need not wait for a long time for a distribution to catch up and that way we can use latest kernel that we want.

I used following steps to compile new kernel.

1.     Update GRUB configuration file (on Host OS i.e my Ubuntu 16.4 VM):

This is important step, this won’t remove the old configuration file but it will add it to the list of available kernels and will allow us to select the menu when we are booting up to the old kernel.

Command: sudo vim /etc/default/grub
This should give us enough time to select the option if we want.

2.     Install pre-requisite packages

apt-get update

      apt-get install git fakeroot build-essential ncurses-dev xz-utils libssl-dev bc

3.     Download STABLE version of kernel

I used stable version of kernel from kernel.org optionally one can use latest version as well. 

wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.13.12.tar.xz
now, extract the packages,
            tar xvf linux-4.13.12.tar.xz  // this will extract the files in current directory
and now change your directory,
            cd linux-4.13.12

This folder will contain all files and folders that re necessary for a kernel. And the same folder will act as a source for our kernel compilation project. I used same folder to write new system calls as well as test programs.

4.     Install & Configure Modules for a new kernel

cp /boot/config-$(uname -r) .config

There are many ways to configure and install whichever packages we want such as ‘make config’ adding entry to boot loader using command line[4][5], but easiest way to achieve this is copy existing configuration file but this is very lengthy process, so I used default configuration file(of my Ubuntu 16.4 LTS) to the new kernel source folder.

For comparison sake,

uname –r  // below, as shown my generic kernel (Ubuntu 16.4 LTS)

            make menuconfig  // to make necessary changes in driver’s and some other extra configurations, I kept all default options.

5.  Now, compiling the kernel:

nproc // to check available processing units to the system, as I was allocated 2 processors to my VM I had 2 PSU’s available

make –j 2  // to compile kernel
make modules_install  -j 2// to install modules
make install –j 2 // copies configuration files and kernel files to /boot folder and creates system.map file

system.map file is a symbol table used by kernel which again I used to add an index of my system function.
            Now, s
            shutdown –r now  // restart the system

Compiling in progress,

On boot up, I was on new kernel that is 4.13.12

Stage 2: Adding a system call “Hello World.!!”

Once new kernel is up and running, I used below steps to implement a simple Hello World! Program written in ‘c’ program.

1.     Create Directory and C program (Hello World!)

mkdir linux-4.13.12 /helloworld
gedit helloworld.c // I used gedit editor  
#include <linux/kernel.h>
#include <stdio.h>

asmlinkage long sys_helloworld(void)
            printk("Hellow World!\n");
            return 0;

// for compiling systems standard c library 
// un-necessary but I was getting ‘asm’ error, adding this helped
// asmlinkage helps compiler to look at cpu’s stack for function arguments 
// long used as a return type in kernel spaces

2.     Create Makefile

gedit linux4.12.13/helloworld/Makefile  // to ensure program compiles and added to kernel source code
 added below line into Makefile,

## Code Begin ##

obj-y := helloworld.o

## Code End ##                   

3.     Link system call with kernel

To achieve this, add helloworld directory that we created in step 1. in kernel’s Makefile

gedit /root/linux-4.13.12/Makefile   // and made change as below

Compiler will refer to our new system call in helloworld directory

4. Modify System Table for Indexing

gedit /linux-4.13.12/arch/x86/entry/syscalls/syscall_64.tbl

append line to the end of the table like below,

333      64        helloworld                   sys_helloworld

333 is index of our system call

5.  Modify System Call Header File

gedit include/linux/syscalls.h

and append following line at the end of the file, basically this is to define prototype of function of our system call.

and now done, we re-compiled the system.

6. Re-compile the kernel

make menuconfig
make oldconfig
make –j 2
make modules_install  -j 2
make install –j 2
shutdown –r now

7. Test your system call

We created new directory inside kernel source called test and below is test c program,

#include <stdio.h>
#include <linux/kernel.h>
#include <sys/syscall.h>
#include <unistd.h>
int main()
            long int s=syscall(333);
            printf("Message:Hello World executed sucessfully! %d \n",s);
            return 0;

Now compiled,

gcc test.c


As program returned 0, that means function executed successfully and kernel should make a log of this execution. (which can be seen using command dmesg | tail -5). 

Few References:

[1]       “Welcome to Linux From Scratch!” [Online]. Available: http://www.linuxfromscratch.org/. 
[2]       “KernelBuild - Linux Kernel Newbies.” [Online]. Available: https://kernelnewbies.org/KernelBuild
[3]       “The Linux Kernel Archives.” [Online]. Available: https://www.kernel.org/.