Linux Kernel Internals on Linux-Kernel-Internals

Char Driver

Mon, 01 Jan 0001 00:00:00 +0000

`dev_t`#

32 bit unsigned integer where first 20 used to store minor and rest 12 bit used to store major.

see:

include/linux/kdev_t.h
include/linux/types.h

Char Devices#

There are 2 ways to create a char device node:

Automatically, via class_create() / device_create() (udev creates the /dev entry).

Manually, via the mknod command.

We’ll go through both.

Note: you must populate struct file_operations before binding it to struct cdev (i.e. before calling cdev_init()). See example

unsigned :\ 0 Bit-Fields in c

Mon, 01 Jan 0001 00:00:00 +0000

`unsigned : 0` in C Bit-Fields#

Sometimes We have particular case, where a field must begin at the next word boundary, so a zero-width bit-field is used.

Consider the following structure:

struct test {
 unsigned a : 3;
 unsigned b : 5;
 unsigned c : 6;
};

member of this struct uses only 14 bits in total (3 + 5 + 6). However, on most systems, sizeof(struct test) gives 4 bytes, not 14 bits. Because unsigned bit-fields are allocated inside an unsigned int storage unit. If unsigned int is 32 bits, all 14 bits fit into a single 32-bit storage unit.

Completion-Variable

Mon, 01 Jan 0001 00:00:00 +0000

Completion-Variable#

Completions are a simple synchronization mechanism that is preferable to sleeping and waking up in some situations. If you have a task that must simply sleep until some process has run its course, completions can do it easily and without race conditions

#include <linux/completion.h>

struct completion_dev {
 struct cdev cdev;
 struct completion completion; 
};

struct completion_dev completion_dev;

static int completion_open(struct inode *inode, struct file *filp)
{
	filp->private_data =
		container_of(inode->i_cdev, struct completion_dev, cdev);
	return 0;
}

static ssize_t completion_read(struct file *filp, char __user *buf,
			 size_t count, loff_t *pos)
{
	struct completion_dev *dev = filp->private_data;
 
	pr_debug("process %d(%s) going to sleep\n", current->pid, current->comm);
	wait_for_completion(&dev->completion);
	pr_debug("awoken %d(%s)\n", current->pid, current->comm);

	return 0;
}

static ssize_t completion_write(struct file *filp, const char __user *buf,
				size_t count, loff_t *pos)
{
	struct completion_dev *dev = filp->private_data;
	pr_debug("process %d(%s) awakening the readers...\n", current->pid, current->comm);

	complete(&dev->completion);
	return count;
}

static const struct file_operations completion_fops = {
	.owner = THIS_MODULE,
	.open = completion_open,
	.read = completion_read,
	.write = completion_write,
};

static int __init m_init(void)
{
	int ret;

	init_completion(&completion_dev.completion);

	ret = alloc_chrdev_region(&dev__t, 0, 1, MODULE_NAME);
	if (ret) {
		pr_debug("Error: %d -Cant't get major\n", ret);
		return ret;
	}
	cdev_init(&completion_dev.cdev, &completion_fops);

	ret = cdev_add(&completion_dev.cdev, dev__t, 1);
	[...],
}

References:

container_of()

Mon, 01 Jan 0001 00:00:00 +0000

`container_of()`#

Accessing a custom struct from `struct inode` via `struct cdev`#

container_of() lets you walk backward from an embedded member (here, struct cdev) to the containing struct. This is how a driver recovers its private device context inside file_operations callbacks.

#include <linux/kernel.h> /* container_of() */
#include <linux/fs.h> /* struct inode, struct file */
#include <linux/cdev.h> /* struct cdev */

struct scull_dev {
	/* ... */
	struct cdev cdev;
};

int scull_open(struct inode *inode, struct file *filp)
{
	struct scull_dev *dev;

	dev = container_of(inode->i_cdev, struct scull_dev, cdev);
	filp->private_data = dev;

	/* ... */
}

ssize_t scull_read(struct file *filp, char __user *buff, size_t count,
		 loff_t *f_pos)
{
	struct scull_dev *dev = filp->private_data;

	/* ... */
}

Pattern: stash the recovered pointer in filp->private_data on open() so every later callback (read, write, ioctl, release, …) can grab it back in one line.

do { ... } while (0)

Mon, 01 Jan 0001 00:00:00 +0000

The Structural Wrapper: `do { ... } while (0)`#

Standard macro design pattern. Forces the macro to behave like a single C statement, so it can’t break syntax when used inside an if/else without braces.

#define LOG_DEBUG(msg) do { \
 printk(KERN_DEBUG msg); \
 debug_count++; \
} while (0)

if (tmp)
 LOG_DEBUG("checking condition\n");
else
 do_something();

Two statements is where this trick stops being a nice-to-have and becomes necessary. Without the wrapper:

Error Handling with Pointers

Mon, 01 Jan 0001 00:00:00 +0000

Error Handling with Pointers in the Kernel#

Many kernel functions return either a valid pointer on success or an encoded error code on failure. this lets a single return value carry both success and failure info, keeping the function signature clean.

In include/linux/err.h:

#define MAX_ERRNO 4095

This is the largest error number the kernel uses. No kernel pointer will ever legitimately point into the very top of the address space, that range is reserved to encode error codes instead, Because kernel reserves a small range of addresses at the very top of the virtual address space and these addresses are invalid as actual memory locations.

Ioctl Interface

Mon, 01 Jan 0001 00:00:00 +0000

ioctl#

The ioctl() function manipulates the underlying device parameters of special files. Most devices can perform operations beyond simple data transfers; user space must often be able to request, for example, that the device lock its door, eject its media, report error information, change a baud rate, or self destruct. These operations are usually supported via the ioctl method.

lwn.net have wonderful explaination on ioctl, i suggest you read it, here to know the internals.

Kbuild Makefile

Mon, 01 Jan 0001 00:00:00 +0000

Kbuild Makefile#

These aren’t plain GNU Make, Kbuild (the kernel’s build system) reads them with special meaning when you run make -C /lib/modules/$(uname -r)/build M=$(pwd) modules.

CFLAGS_main.o := -DDEBUG

ccflags-y := -std=gnu99 -DENABLE_DEBUG

proc_fs_basic-objs := main.o

obj-m := proc_fs_basic.o

CFLAGS_main.o - per-object compiler flags. Only main.o gets -DDEBUG; if you had main.o and helper.o, only the first sees the macro. Useful for debug-gating one file without recompiling the whole module noisily.

Module Parameters

Mon, 01 Jan 0001 00:00:00 +0000

Module Parameters#

#include <linux/module.h>
#include <linux/moduleparam.h> /* not strictly needed, linux/module.h already pulls it in */
#include <linux/kernel.h>

module_param(name, type, perm);
MODULE_PARM_DESC(myarr, "this is my array of int");

Pass the value at load time:

insmod module.ko param=value

pr_fmt()

Mon, 01 Jan 0001 00:00:00 +0000

`pr_fmt()`#

Prefixing the `pr_*()` calls#

pr_fmt() is a macro that rewrites the format string of every pr_*() call (pr_info, pr_warn, pr_err, …) in the file. It has to be defined before any header that pulls in <linux/printk.h>, so it always sits at the very top, above your includes.

#define pr_fmt(fmt) KBUILD_MODNAME ":%s: " fmt, __func__

#include <linux/kernel.h>
#include <linux/module.h>

static int func(void)
{
	pr_warn("hello-world\n");
	return -1;
}

pr_fmt() line auto-adjusts to whichever function it’s expanded in.

Stringification

Mon, 01 Jan 0001 00:00:00 +0000

Stringification#

include/linux/stringify.h defines:

#define __stringify_1(x...) #x
#define __stringify(x...) __stringify_1(x)

Both use # to turn an argument into a string literal. They differ only when the argument is a macro.

# stringifies tokens before expanding any macros in them. So __stringify_1(x) gives you the literal text you passed, macro name included, unexpanded.

__stringify(x) adds one layer of indirection. Since x isn’t touched directly by # inside __stringify, the preprocessor expands x first when it’s passed down to __stringify_1, which then stringifies the expanded result.

Error Handling

Mon, 01 Jan 0001 00:00:00 +0000

Error Handling for complex cases#

Standard kernel pattern for unwinding partial allocations when a loop fails mid-way.

int ret;
int i;

for (i = 0; i < count; i++)
{
 arr1[i] = kmalloc(size1, GFP_KERNEL);
 if (!arr1[i]) {
 ret = -ENOMEM;
 goto err;
 }
 arr2[i] = kmalloc(size2, GFP_KERNEL);
 if (!arr2[i]) {
		/* error handling in mid-way loop */
 kfree(arr1[i]);
 arr1[i] = NULL;
 ret = -ENOMEM;
 goto err;
 }
}

	return 0;

err:
	while (--i >= 0)
	{
		kfree(arr2[i]);
		kfree(arr1[i]);
	}
	return ret;

Proc-Interface

Mon, 01 Jan 0001 00:00:00 +0000

Proc Interface#

/proc is the classic way a driver hands kernel-side information to userspace without going through a real block device. There are three ways to implement the read side of a proc entry, depends on whether you’re exposing a single fixed value or walking a list.

Keep a note that this method is depreciated for device driver.

1. Raw `file_operations` - the old way#

This is the original method: you wire up .read, .write, .open, .llseek, and .release yourself, exactly like a normal character device.

programming concepts

Mon, 01 Jan 0001 00:00:00 +0000

In this section, I’ll try to cover the programming concepts.

‘Thundering Herd problem’#

where hundreds of processes are woken up at once to fight for the same lock.

We can observe this situation when wait_queue_head is waked up by wake_up_all; multiple list element trying to access the same lock/resources. To Handle these situation its better to try exclusive wake up - wake_up{,_nr}.

Build & Install Linux Kernel for BBB

Mon, 01 Jan 0001 00:00:00 +0000

Build and Install Linux Kernel For Beaglebone Black#

Overview#

The BeagleBone Black (BBB) uses an ARM Cortex-A8 32 bit processor, so the kernel must be cross-compiled on an x86 machine and then deployed to the board. This blog will assume that you already have a os[debian/ubuntu] installed on your sd card.

We’ll:

set up toolchain
fetch kernel source
configure for BBB
build kernel + modules
deploy to SD card / board

1. Prerequisites#

Make sure you have:

Build & Install Linux Kernel for BBB

Mon, 01 Jan 0001 00:00:00 +0000

Build and Install Linux Kernel For Beaglebone Black#

Overview#

We’ll:

set up toolchain
fetch kernel source
configure for BBB
build kernel + modules
deploy to SD card / board

1. Prerequisites#

Make sure you have:

C Memory Layout

Mon, 01 Jan 0001 00:00:00 +0000

C Memory Layout: Structs, Unions, and a Linux Kernel Patch#

Overview#

C memory layout looks simple in isolation: structs store fields sequentially, unions overlap memory, and flexible array members allow variable-sized trailing storage. But combining these features can create subtle layout issues.

While working on a Linux kernel selftest, I encountered a case where a union contained multiple structures, each ending with a flexible array member at different offsets. Although the code was functionally correct, this confused the compiler into treating the union as effectively variable-sized, triggering a layout warning.

C Memory Layout

Mon, 01 Jan 0001 00:00:00 +0000

C Memory Layout: Structs, Unions, and a Linux Kernel Patch#

Overview#

Configure U-boot for BBB

Mon, 01 Jan 0001 00:00:00 +0000

Configure U-boot for BeagleBone Black#

Overview#

U-Boot (Universal Boot Loader) is the go-to bootloader for embedded Linux systems. If you’re working with the BeagleBone Black (BBB), building U-Boot from source gives you full control over the boot process, from initializing hardware to loading your kernel. This guide walks you through cross-compiling and installing U-Boot on the BBB from scratch.

This blog will explain how to compile and install U-Boot, and then either manually boot from the U-Boot prompt or use uEnv.txt to autoboot the kernel. You can also use extlinux/extlinux.conf to boot the kernel.

Configure U-boot for BBB

Mon, 01 Jan 0001 00:00:00 +0000

Configure U-boot for BeagleBone Black#

Overview#

Debugging Macros

Mon, 01 Jan 0001 00:00:00 +0000

Debugging Macros#

This section will contain all macros related to kernel debugging.

`might_sleep`#

configs

CONFIG_DEBUG_ATOMIC_SLEEP

this macros is useful to detect sleeping function called in Non sleeping code-paths, and implies that code path should not be executed in atomic context, since it can sleep.

/**
 * might_sleep - annotation for functions that can sleep
 *
 * this macro will print a stack trace if it is executed in an atomic
 * context (spinlock, irq-handler, ...). Additional sections where blocking is
 * not allowed can be annotated with non_block_start() and non_block_end()
 * pairs.
 *
 * This is a useful debugging help to be able to catch problems early and not
 * be bitten later when the calling function happens to sleep when it is not
 * supposed to.
 */
# define might_sleep() \
 do { __might_sleep(__FILE__, __LINE__); might_resched(); } while (0)

Forward Declarations

Mon, 01 Jan 0001 00:00:00 +0000

Understanding Forward Declarations in C#

One of the best ways to learn systems programming is by reading real-world code. Recently, while reading the Linux kernel source, I noticed something interesting inside drivers/pinctrl/core.h:

#include <linux/kref.h>
#include <linux/list.h>
#include <linux/mutex.h>
#include <linux/radix-tree.h>
#include <linux/types.h>

#include <linux/pinctrl/machine.h>

struct dentry;
struct device;
struct device_node;
struct module;

At first glance, these lines seem incomplete. We see declarations like struct device; but no actual definition. Where is the complete structure with all its members? Why didn’t file defining this structure isn’t included?

Install Ubuntu Server on Qemu

Mon, 01 Jan 0001 00:00:00 +0000

Install Ubuntu Server on Qemu#

Installation of ubuntu server on qemu-x86_64#

At First I want to run the custom modules on linux. But this is something that is not good to do on host machine. So i searched a lot of stuff. Tinkering with buildroot, after some days i was able to run the build on qemu.

But i found out build actually didn’t have my basic tool like gnu make. Everytime i try to do something i need to find particular tool on menuconfig then make the build again. Learning was so slow.

Linux Kernel Mentorship Program

Mon, 01 Jan 0001 00:00:00 +0000

Linux Kernel Mentorship Program#

Introduction#

I have been using Linux for the last two years and gradually developed an interest in systems programming and C. The mentorship prerequisites includes tasks like building and booting the Linux kernel, writing a basic kernel module, decoding stack traces, and modifying and booting a custom kernel build.

Somehow, my application was accepted ;-).

At first, the Linux Kernel seemed quite difficult to understand what was going on. We were initially told to choose two subsystems to work on.

Linux Kernel Mentorship Program

Mon, 01 Jan 0001 00:00:00 +0000

Linux Kernel Mentorship Program#

Introduction#

Somehow, my application was accepted ;-).

At first, the Linux Kernel seemed quite difficult to understand what was going on. We were initially told to choose two subsystems to work on.

Return value by Macro

Mon, 01 Jan 0001 00:00:00 +0000

code example of macro which returns a value.

#include <stdio.h>

#define check(condition) \
 ({ \
 int __ret = 0; \
 if (!(condition)) \
 { \
 printf("Condition failed!\n"); \
 __ret = -1; \
 } \
 __ret; \
 })

int main()
{
 int x = 10;
 int result = check(x > 15);

 printf("result: %d", result);
 return 0;
}

SLUB debug

Mon, 01 Jan 0001 00:00:00 +0000

Basics of debugging slab memory corruption via SLUB debug#

Introduction#

Memory corruption can occur due to various bugs or defects: Uninitialized Memory Reads (UMR), Use After Free (UAF), Use After Return (UAR), double-free, memory leakage, or illegal Out Of Bounds (OOB) accesses that attempt to work upon (read/write/ execute) illegal memory regions.

Since memory is dynamically allocated and freed via the kernel’s engine – the page allocator. This can lead to serious wastage (internal fragmentation) of memory. To mitigate this, the slab allocator (or slab cache) is layered upon it, serving two primary tasks – providing fragments of pages efficiently (within the kernel, allocation requests for small pieces of memory, from a few bytes to a couple of kilobytes), and serving as a cache for commonly used data structures.

Variadic Macro

Mon, 01 Jan 0001 00:00:00 +0000

In kernel you’ll see two type of variaidic macro

#define __stringify(x...)	 __stringify_1(x)

this is same as

#define __stringify(...)	 __stringify_1(__VA_ARGS__)

In standard C99 and later, a variadic macro must look like above one, However, GCC introduced an extension that allows you to give a name to the variable arguments by placing the ellipsis immediately after an identifier (x...). When you write x..., the compiler treats x as a named parameter that bundles everything passed into the macro (including any commas separating multiple arguments).

Linux Kernel Internals on Linux-Kernel-Internals

Char Driver

dev_t#

Char Devices#

unsigned :\ 0 Bit-Fields in c

unsigned : 0 in C Bit-Fields#

Completion-Variable

Completion-Variable#

container_of()

container_of()#

Accessing a custom struct from struct inode via struct cdev#

do { ... } while (0)

The Structural Wrapper: do { ... } while (0)#

Error Handling with Pointers

Error Handling with Pointers in the Kernel#

Ioctl Interface

ioctl#

Kbuild Makefile

Kbuild Makefile#

Module Parameters

Module Parameters#

pr_fmt()

pr_fmt()#

Prefixing the pr_*() calls#

Stringification

Stringification#

Error Handling

Error Handling for complex cases#

Proc-Interface

Proc Interface#

1. Raw file_operations - the old way#

programming concepts

‘Thundering Herd problem’#

Build & Install Linux Kernel for BBB

Build and Install Linux Kernel For Beaglebone Black#

Overview#

1. Prerequisites#

Build & Install Linux Kernel for BBB

Build and Install Linux Kernel For Beaglebone Black#

Overview#

1. Prerequisites#

C Memory Layout

C Memory Layout: Structs, Unions, and a Linux Kernel Patch#

Overview#

C Memory Layout

C Memory Layout: Structs, Unions, and a Linux Kernel Patch#

Overview#

Configure U-boot for BBB

Configure U-boot for BeagleBone Black#

Overview#

Configure U-boot for BBB

Configure U-boot for BeagleBone Black#

Overview#

Debugging Macros

Debugging Macros#

might_sleep#

Forward Declarations

Understanding Forward Declarations in C#

Install Ubuntu Server on Qemu

Install Ubuntu Server on Qemu#

Installation of ubuntu server on qemu-x86_64#

Linux Kernel Mentorship Program

Linux Kernel Mentorship Program#

Introduction#

Linux Kernel Mentorship Program

Linux Kernel Mentorship Program#

Introduction#

Return value by Macro

SLUB debug

Basics of debugging slab memory corruption via SLUB debug#

Introduction#

Variadic Macro

`dev_t`#

`unsigned : 0` in C Bit-Fields#

`container_of()`#

Accessing a custom struct from `struct inode` via `struct cdev`#

The Structural Wrapper: `do { ... } while (0)`#

`pr_fmt()`#

Prefixing the `pr_*()` calls#

1. Raw `file_operations` - the old way#

`might_sleep`#