lkb-mq -- linux kernel-based concurrent message queue

sudo insmod ./driver/scull.ko scull_fifo_size=<int> scull_fifo_elemsz=<int>

producer <concurrent processes>

consumer <concurrent processes>

[See the Source Code!]

LKB-MQ extends a barebones "scull" character device driver into a bounded FIFO message queue. At module load time, the driver allocates the entire queue as a single flat character array holding scull_fifo_size elements. Each element is a length header followed by scull_fifo_elemsz bytes of data, and both parameters are configurable through module parameters. Because the element size is only known at runtime, the queue is managed as a circular buffer with start and end pointers, advanced by manual pointer arithmetic and wrapped back to the beginning of the array when they pass its end. The module checks that allocation succeeds, and frees the buffer on unload. scull_write copies at most one element's worth of data from user space, truncating anything longer, and stores the actual message length alongside it. scull_read copies out one message, truncating to the caller's buffer size if needed, and returns the number of bytes delivered.

To make this safe for many concurrent producers and consumers, reads and writes are synchronized using kernel mutexs and semaphores; consumers block while the queue is empty and producers block while it is full, and each wakes the other side as space or data becomes available -- in line with your typical readers-writers or producers-consumers implementation. In this case however, the blocking must be interruptible such that a process can safely be killed with Ctrl-C, with the driver returning an appropriate error, as opposed to leaving a lock held, which inevitably would lead kernel memory corruption.

First, the character device driver itself has to be inserted as a kernel module:

$ cd driver
# Build the kernel module / device driver
$ make

# Register the kernel module
$ sudo insmod scull.ko

# Get the dynamically assigned device ID
$ export LKB_MQ_ID=$(cat /proc/devices | grep scull | awk '{print $1}')

# Create the device node using the assigned device ID
$ sudo mknod /dev/scull c $LKB_MQ_ID 0

# Give read/write permissions so the producer can access it without root
$ sudo chmod o+rw /dev/scull

As for the consumer and producer programs:

# From the driver directory
$ cd ../src

# Build the user-space producer and consumer programs
$ make

# Produce and consume some data, blocking when necessary
$ ./producer p 3
Device (/dev/scull) opened
write: Tristan Shabbick
write: Tristan Shabbick
write: Tristan Shabbick
Device (/dev/scull) closed
$ ./consumer p 5
Device (/dev/scull) opened
read: Tristan Shabbick
read: Tristan Shabbick
read: Tristan Shabbick

# *blocks until more data can be read*

This was originally an assignment during the Spring 2024 semester for CS 492 - Operating Systems @ Stevens Institute of Technology; however, due to a registrar error and for the one and only time in at least a decade, this course's prerequisites only required CS 382 - Computer Architecture and Organization, as opposed to CS 392 - Systems Programming, which translated to being able to (wrongfully) take this course an entire year early, before any extended exposure to the C programming language, among other necessary system-level concepts. As a result, myself, and many other students, had more difficulty with some of these assignments than the upperclassmen. As such, in September of 2026, I went back and revised much of this project's code to work as intended in light of this unfortunate mistake by the registrar.