Linux Kernel Development II - ARP replicating network driver - part I

Here is the second article in Linux Kernel development series. Majority of this article is from my old blog. I was complaining about scarcity of information on Linux networking in the old article but that was before I read Understanding Linux network internals. I still think similar since some parts of the book is outdated but there are other books I haven't read yet like Rami Rosen's book which may be filling the gap. Also there seems to be a lot more blog posts and wiki articles, like mine, talking about various topics. So if you know how everything works in general, it is quite possible to find information on a specific mechanism or implementation of a concept nowadays.

I will be talking about the fakeARP driver which you can access from GitHub [source]. It is a network driver which replies incoming ARP requests with fake ARP replies. (The link directs you to tutorial tag in Git, development continues on development and master branches. If you are interested you can check out current status too.)

Thread cancelling caveats in pthreads

This is one of the posts I moved from my previous blog. I am preparing a long article about network drivers in Linux but it seems that it will take some more time. So I am posting this article I wrote a few years ago to keep the blog alive.

Today I will talk about two things I encountered while working with conditional mutexes in pthreads which are somewhat weird.

1. If a thread gets cancelled while being blocked by pthread_cond_wait(), cancelled thread decides to lock the condition mutex before exiting. (as a result no one else can unlock it)
2. pthread_cleanup_push() macro doesn't compile if another pthread_cleanup_pop() macro is not included in the same block of code.

If you read the man pages or some mailing list discussions they will convince you that this is the way things should work. But I think they are caveats one should be beware of.

Preparing a test VM - Videos

Hello,

I streamed some streams in livecoding.tv. They are of poor quality because of a number of reasons.

1) It was first time I ever streamed something, I learned as I went.
2) I was very tired or inconvenient in most of them since I tried to stream in late hours or in my very limited spare time.
3) My upload bandwidth is not enough to stream in a better quality. Unfortunately sometimes I have frame drops and connection is lost between some streams.

Link to my channel.

Linux Kernel Development I - "Hello, World!" char driver

So here is my first post about Linux kernel module development!

I published this article in my old blog before. We will write a "Hello, world!" char device driver. I also released the code long time ago [source].

Here is the reason why I start with this article: When you google printk you get so many results, tutorials, stackoverflow.com questions, .doc files, .pdf files, stuff about formatting and console log levels etc. But when you search for how to write a simple working driver (I mean a driver which does something ''besides'' printing text to dmesg) you get only a handful of helpful results, some of which are fairly out-dated.

What I want to give you is a working char device driver with minimal kernel bureaucracy. You will still need to read a good book on drivers and write lots of code later. But you will get your own working little kernel code. As they say, "Seeing is believing" and I think there is a good number of people who put off just because they don't see their code do something meaningful (or study a working module/driver source but can't see any piece code which makes sense :). I aim to help people who are curious about kernel development but lose interest thinking it is too much stuff to learn or too much code to write even for the simplest working driver. So if you are stuck while reading a book or taking a course on developing device drivers for Linux you probably came to the right place.

Preparing a test VM - Extras

Hi,

In this extra part I want to talk a little bit about accessing the Internet from within the VM and installing packages in Gentoo. I will also talk about practically uploading your files into the VM.

Preparing a test VM - Compile and run a module

Hi,

This is the final post in preparing our test environment. We will write a code piece and run it in the VM. You must have finished previous sections (filesystem, kernel and QEMU) to be able to carry out the instructions in this one. After finishing this you should be able to compile and run your own kernel level code.

Preparing a test VM - QEMU

We have our kernel and our filesystem. Now we need to utilize virtualization to run our OS.

I will not talk about how virtualization or emulation works. However you need to know a few details.

When IT people think what virtualization technology is all about, they often think how host CPUs are shared between VMs and how various memory tricks work, like ballooning or taking memory snapshots. After that storage or network infrastructure and configuration comes to mind. And of course, live migration, which looks like pure magic the first time you see it work (if you have a cluster of hosts)

For kernel development we are interested in different kind of stuff. Most of the time we need only one machine with fixed amount of memory, fixed amount of disk space and just one virtual CPU.

Preparing a test VM - Kernel

In previous post I talked about preparing a filesystem with Gentoo. Today I will talk about compiling the Linux kernel.

In normal Gentoo setup you emerge (install like "apt-get install" in Ubuntu) the kernel from Gentoo repo and compile it on the machine you are installing Gentoo.

We will be doing something different. Using virtualization, we will feed the kernel binary files from the host to the VM. The main reason is we need the compiled kernel files in our development environment to compile modules for that kernel. Kernel source and the module files should be located in the same machine.

Preparing a test VM - Filesystem

Hello,

This will be the first of a five part how-to on using virtualization to create a testing environment. We need a separate test machine because we will be crashing the kernel most of the time and rebooting your computer each time will be hazardous for your computer and frustrating for you.

We will be using QEMU for virtualization and Gentoo for the guest operating system where our kernel code will run. I assume development will be done in the same machine which is the virtualization host. I will not cover the virt host/development environment, you can use any distro capable of running QEMU (and preferably kvm).