Lesson 25-Inline Assembly in Driver Development

Introduction: Where Assembly Meets the Kernel

If you’ve made it this far in the series, you know that assembly language is the language of the CPU. But most of the code you’ve written so far has been in user space—programs that run with limited privileges, accessing only their own memory, and relying on the operating system to provide services via system calls.

Drivers are different. A device driver runs in kernel space (ring 0 on x86), which means it has unrestricted access to the hardware: CPU control registers, I/O ports, device memory, and more. And because drivers need to do things that C simply can’t express—like reading the current CR3 register (page table base), performing an atomic operation with the LOCK prefix, or reading the CPU’s timestamp counter—they often need inline assembly.

In this article, we’re going to explore the use of inline assembly in driver development. We’ll cover why drivers need assembly, how to write inline assembly in both GCC and MSVC, the special registers and instructions that only drivers can use, atomic operations and memory barriers (crucial for SMP systems), and we’ll walk through a complete example of a Linux character device driver that uses inline assembly to read the CPU’s timestamp counter.

This is advanced material. I’m assuming you’re comfortable with x86 assembly, you understand the concept of privilege levels (rings), you know what a device driver is and roughly how it interfaces with the kernel, and you’ve at least heard of concepts like SMP (symmetric multiprocessing) and memory ordering. If you’re not there yet, you might want to go back and read the earlier articles in this series—I’ll wait.

Alright, let’s dive into the deep end of the pool: writing assembly in the kernel.

1. Why Drivers Need Assembly? (Accessing CPU-Specific Registers, I/O Ports, Atomic Operations)

Let’s start with the obvious question: why would a driver—which is typically written in C—need to use assembly at all? C is a high-level language (well, relatively speaking). It doesn’t give you direct access to CPU registers like CR0, CR3, or CR8. It doesn’t let you perform I/O port accesses (IN/OUT instructions). And it doesn’t have built-in support for atomic read-modify-write operations (like CMPXCHG with LOCK prefix) or memory barriers (like MFENCE).

Drivers, on the other hand, need to do all of these things. Let’s break down the main reasons:

1.1 Accessing CPU Control Registers (CR0-CR8)

The x86 CPU has a set of control registers (CR0 through CR8) that control important aspects of the CPU’s operation: paging, interrupt handling, task switching, and more. These registers can only be accessed from ring 0 (kernel mode). User-space code trying to read or write them will trigger a #GP (general protection fault).

Here’s what some of the control registers do:

  • CRO: Controls the operating mode of the CPU. Key bits:
    • PE (bit 0): Protected mode enable.
    • PG (bit 31): Paging enable.
    • WP (bit 16): Write protect (prevents kernel from writing to read-only pages).
    • AM (bit 18): Alignment mask.
    • NE (bit 5): Numeric error (floating-point error handling).
    • ET (bit 4): Extension type (always 1 on 486+).
    • TS (bit 3): Task switched (used for lazy FPU context switching).
    • EM (bit 2): Emulation (if set, floating-point instructions cause #NM).
    • MP (bit 1): Monitor coprocessor.
  • CR2: Contains the linear address that caused the last page fault. The page fault handler reads CR2 to determine which address needs to be paged in.
  • CR3: Contains the physical address of the top-level page table (PDPT for 32-bit PAE, PML4 for x86-64). This is how the CPU knows where to look up virtual-to-physical address translations. Context switching between processes involves changing CR3 to point to the new process’s page tables.
  • CR4: Controls features like:
    • PAE (bit 5): Physical Address Extension (allows 36-bit physical addresses on 32-bit CPUs).
    • PCIDE (bit 17): PCID (Process Context Identifier) enable (x86-64).
    • OSFXSR (bit 9): OS supports FXSAVE/FXRSTOR.
    • OSXMMEXCPT (bit 10): OS supports unmasked SIMD floating-point exceptions.
  • CR8: (x86-64 only) Contains the Task Priority Register (TPR) for the APIC. Controls which interrupt vectors can be delivered to the CPU.

To access these registers, you need to use assembly instructions: MOV to/from CR registers (MOV EAX, CR0, MOV CR3, EBX, etc.). C simply doesn’t have syntax for this.

1.2 Accessing Model-Specific Registers (MSRs)

In addition to the standard control registers, x86 CPUs have a large number of Model-Specific Registers (MSRs). These are registers that control CPU features that vary by CPU model (hence “model-specific”). Examples:

  • IA32_TSC: The timestamp counter (MSR 0x10). Holds the number of CPU cycles since reset. Can be read with RDTSC (a simpler instruction that doesn’t require using RDMSR).
  • IA32_APIC_BASE: The base address of the local APIC (MSR 0x1B).
  • IA32_EFER: Extended Feature Enable Register (MSR 0xC0000080). Controls long mode (for x86-64) and other features.
  • Performance monitoring MSRs: Used to configure hardware performance counters (for profiling).

To read or write MSRs, you use the RDMSR and WRMSR instructions. These instructions are privileged (ring 0 only), and they take the MSR number in ECX and return the value in EDX:EAX (for RDMSR).

Again, C doesn’t have built-in support for RDMSR/WRMSR, so you need inline assembly.

1.3 I/O Port Access

x86 CPUs have a separate I/O address space (distinct from memory) for communicating with legacy devices. Devices like the keyboard controller, PIC, PIT (timer), and COM ports (serial ports) are accessed through I/O ports using the IN and OUT instructions.

While many modern devices use memory-mapped I/O (MMIO)—where device registers are mapped into the physical address space and can be accessed with normal MOV instructions—some still require port I/O. And for those, you need IN/OUT.

The Linux kernel provides helper functions (inb, outb, inw, outw, etc.) that wrap the inline assembly for you. But if you’re writing a driver for a new OS, or if you need to do complex I/O sequences, you might need to write the inline assembly yourself.

1.4 Atomic Operations

In a multi-core (SMP) system, multiple CPUs might be accessing the same memory location concurrently. If one CPU is incrementing a counter while another CPU is also incrementing it, you could lose an update (a “race condition”).

To prevent this, x86 provides atomic operations—operations that are guaranteed to complete without being interrupted by another CPU’s memory access. The key mechanism is the LOCK prefix, which locks the memory bus (or the cache line) for the duration of the instruction.

Atomic instructions:

  • LOCK INC, LOCK DEC, LOCK ADD, LOCK SUB, etc.: Atomic read-modify-write.
  • LOCK CMPXCHG: Atomic compare-and-swap (CAS). This is the building block of lock-free algorithms.
  • XCHG (with memory operand): Always locked (implicit LOCK prefix).

C supports some atomic operations through compiler builtins (like __sync_fetch_and_add in GCC, or stdatomic.h in C11). But for complex atomic algorithms, or for writing your own locking primitives (spinlocks, mutexes), you need to use inline assembly to ensure the correct instructions are generated.

1.5 Memory Barriers

Modern CPUs are out-of-order: they execute instructions in a different order than the program specified, as long as the architectural state (register values, memory values as seen by the program) is correct. This is normally fine—the CPU ensures that the result is as if the instructions were executed in order.

But in a multi-core system with memory-mapped I/O, out-of-order execution can cause problems. For example:

  • You write to a device register (memory-mapped I/O).
  • Then you write to memory (a flag saying “device operation complete”).
  • The CPU might reorder these writes, so the flag is written BEFORE the device register write.
  • Another CPU (or the device) might see the flag and think the operation is done, when it’s not.

To prevent this, you use memory barrier instructions:

  • MFENCE: Memory fence (all loads and stores before the fence must complete before any loads/stores after the fence).
  • LFENCE: Load fence (orders loads).
  • SFENCE: Store fence (orders stores).

C doesn’t have memory barriers (though C11’s atomic_thread_fence maps to them). For drivers, you often need to insert memory barriers explicitly using inline assembly.

2. Inline Assembly in C (GCC): asm() Syntax — Constraints, Clobbers, Volatile

Alright, now that we know why drivers need assembly, let’s talk about how to write inline assembly in C. We’ll start with GCC’s inline assembly syntax, which is the more complex (and more powerful) of the two main syntaxes.

2.1 Basic Syntax

GCC’s inline assembly uses the asm() (or __asm__()) keyword. The basic syntax:

asm("assembly instruction");

For example:

asm("cli");   // Clear interrupt flag (disable interrupts)
asm("sti");   // Set interrupt flag (enable interrupts)

But this is the “simple” form, and it’s rarely used in drivers because you usually need to interface with C variables (input operands, output operands). For that, you use the “extended” inline assembly syntax:

asm volatile (
    "assembly template"
    : output operands       /* optional */
    : input operands        /* optional */
    : clobbered registers   /* optional */
);

Let’s break this down.

2.2 The Assembly Template

The first argument is a string containing the assembly instructions. You can use placeholders (%0, %1, etc.) for the operands. For example:

int value = 10;
int result;

asm ("mov %1, %0"   // Move value into result
     : "=r" (result)  // Output: result (in a register)
     : "r" (value)     // Input: value (in a register)
);

Here, %0 refers to the first operand (output result), and %1 refers to the second operand (input value). The "r" constraint means “put this operand in a general-purpose register.”

2.3 Operand Constraints

Operand constraints tell GCC how to place the operands (in registers, in memory, etc.). Common constraints:

  • "r": General-purpose register (EAX, EBX, ECX, EDX, ESI, EDI, ESP, EBP).
  • "a", "b", "c", "d": Specific register (AL/AX/EAX, BL/BX/EBX, CL/CX/ECX, DL/DX/EDX).
  • "m": Memory operand (the variable’s address).
  • "i": Immediate operand (constant).
  • "=r": Output operand (write-only) in a register.
  • "+r": Input-output operand (read-write) in a register.

Example: reading the current value of the EFLAGS register:

unsigned long eflags;

asm volatile (
    "pushf\n\t"        // Push EFLAGS onto stack
    "pop %0"            // Pop into eflags
    : "=r" (eflags)    // Output: eflags
    :                    // No inputs
    :                    // No clobbered registers (we used stack, but GCC knows about stack)
);

2.4 Clobbered Registers

When you write inline assembly, GCC needs to know which registers your assembly code modifies (beyond the output operands). This allows GCC to save/restore those registers if needed.

For example, if your assembly uses ECX as a temporary register, you need to list "ecx" in the clobber list:

asm volatile (
    "mov $5, %0\n\t"
    "mov $10, %1"
    : "=r" (a), "=r" (b)
    : 
    : "%eax", "%ebx"   // We modify EAX and EBX (as temporaries)
);

Actually, the correct way to specify clobbered registers is with "%reg" syntax (for general-purpose registers). There are also special clobbers:

  • "memory": Tells GCC that the assembly reads or writes memory that isn’t accounted for by the operands. This prevents GCC from reordering loads/stores across the assembly.
  • "cc": Tells GCC that the assembly modifies the condition codes (flags).

2.5 Volatile

The volatile keyword (in asm volatile) tells GCC not to optimize away the assembly instruction, even if it seems to have no effect. This is crucial for driver code—you might write to a device register (which has side effects), and GCC might think “oh, that store has no effect, I’ll optimize it away.” volatile prevents that.

Even if the output operand is used, GCC might still optimize the assembly in ways you don’t expect (like moving it across other instructions). If the order of assembly instructions matters (for example, you’re writing to device registers in a specific order), use volatile.

2.6 A Complete Example: Reading CR0

Let’s write a function to read the CRO register:

unsigned long read_cr0(void) {
    unsigned long cr0;
    asm volatile (
        "mov %%cr0, %0"   // Move CR0 to cr0 (output)
        : "=r" (cr0)       // Output operand: cr0 (in a register)
        :                    // No inputs
        :                    // No clobbers (we only read CR0)
    );
    return cr0;
}

Note the double percent signs (%%cr0). In extended inline assembly, single % is used for operands (%0, %1), so to refer to a register, you need %%.

3. Inline Assembly in C (MSVC): __asm Blocks — Simpler Syntax but Less Portable

Microsoft Visual C++ (MSVC) has a different inline assembly syntax. It’s simpler (you don’t need to learn about constraints and clobbers), but it’s also less powerful and less portable (it only works on x86, not x86-64—Microsoft removed inline assembly support for x86-64).

3.1 Basic Syntax

MSVC uses __asm blocks:

__asm {
    instruction1
    instruction2
    ...
}

Or single-line:

__asm instruction

Example: disabling interrupts:

__asm {
    cli
}

3.2 Accessing C Variables

In MSVC inline assembly, you can refer to C variables directly by name. The assembler figures out the addressing:

int a = 10;
int b;

__asm {
    mov eax, a     // Load C variable 'a' into EAX
    add eax, 5
    mov b, eax     // Store EAX into C variable 'b'
}

This is simpler than GCC’s constraint-based approach. But it’s also less flexible—you can’t tell the compiler “put this variable in a specific register” (well, you can with __declspec(register), but it’s not as flexible).

3.3 Limitations of MSVC Inline Assembly

  1. No support for x86-64. Microsoft removed inline assembly support for x86-64. If you need assembly on x86-64 with MSVC, you have to write a separate .asm file and assemble it with MASM, or use intrinsics.
  2. Less optimization. GCC’s inline assembly with constraints allows the compiler to optimize better (it knows which registers are used, so it can allocate other registers more efficiently). MSVC’s inline assembly is essentially a “black box” to the compiler—it assumes you clobber all registers (except EBX, ESI, EDI, EBP, which are preserved).
  3. No instruction scheduling. GCC can schedule inline assembly instructions together with surrounding C code (if you don’t use volatile). MSVC can’t.

For these reasons, if you’re writing portable driver code, GCC’s inline assembly is preferred. But if you’re writing Windows drivers for x86-32, MSVC’s __asm blocks are simpler to use.

3.4 A Complete Example: Reading the Timestamp Counter (RDTSC)

Let’s write a function to read the CPU’s timestamp counter (a 64-bit value that increments every CPU cycle):

unsigned __int64 read_tsc(void) {
    unsigned __int64 tsc;
    unsigned int low, high;
    
    __asm {
        rdtsc           // EDX:EAX = timestamp counter
        mov low, eax    // Store low 32 bits
        mov high, edx   // Store high 32 bits
    }
    
    tsc = ((unsigned __int64)high << 32) | low;
    return tsc;
}

Actually, MSVC has an intrinsic for this: __rdtsc(). So you don’t need inline assembly. But this example illustrates how to use __asm blocks.

4. Reading/Writing CPU Control Registers (CR0-CR8)

Now let’s dive into the actual assembly instructions for reading and writing control registers. These instructions are privileged (can only run in ring 0), so they’re perfect for driver code.

4.1 Reading/Writing CR0 (Control Register 0)

CRO controls the CPU’s operating mode. Let’s say we want to disable interrupts and then enable them (using CLI/STI), but we also want to check the PE (protected mode enable) bit in CRO to make sure we’re actually in protected mode.

int is_protected_mode(void) {
    unsigned long cr0;
    asm volatile ("mov %%cr0, %0" : "=r" (cr0));
    return (cr0 & 0x1);   // PE bit is bit 0
}

To toggle the WP (write protect) bit (bit 16):

void disable_write_protect(void) {
    unsigned long cr0;
    asm volatile ("mov %%cr0, %0" : "=r" (cr0));
    cr0 &= ~(1 << 16);   // Clear WP bit
    asm volatile ("mov %0, %%cr0" : : "r" (cr0));
}

void enable_write_protect(void) {
    unsigned long cr0;
    asm volatile ("mov %%cr0, %0" : "=r" (cr0));
    cr0 |= (1 << 16);    // Set WP bit
    asm volatile ("mov %0, %%cr0" : : "r" (cr0));
}

4.2 Reading CR2 (Page Fault Address)

The CR2 register holds the linear address that caused the last page fault. The page fault handler in the OS reads CR2 to determine which page to load.

void *get_page_fault_address(void) {
    unsigned long address;
    asm volatile ("mov %%cr2, %0" : "=r" (address));
    return (void *)address;
}

4.3 Reading/Writing CR3 (Page Table Base)

CR3 holds the physical address of the top-level page table. In a context switch, the OS changes CR3 to point to the new process’s page tables.

unsigned long get_cr3(void) {
    unsigned long cr3;
    asm volatile ("mov %%cr3, %0" : "=r" (cr3));
    return cr3;
}

void set_cr3(unsigned long cr3_value) {
    asm volatile ("mov %0, %%cr3" : : "r" (cr3_value));
}

Note: CR3 contains the physical address. If you’re working with virtual addresses (as you usually are in C), you need to convert between virtual and physical addresses.

5. Reading/Writing Model-Specific Registers (MSRs)

As mentioned earlier, MSRs are model-specific registers that control CPU features. To read or write them, you use the RDMSR and WRMSR instructions.

5.1 RDMSR and WRMSR Instructions

The RDMSR instruction reads the MSR specified in ECX. The result is returned in EDX:EAX (64-bit value, with high 32 bits in EDX, low 32 bits in EAX).

The WRMSR instruction writes to the MSR specified in ECX. The value to write is in EDX:EAX.

These instructions are privileged (ring 0 only).

5.2 Reading the Timestamp Counter MSR (0x10)

The timestamp counter (TSC) can be read with RDTSC (which is simpler and doesn’t require setting up ECX). But for illustration, here’s how to read it with RDMSR:

unsigned __int64 read_tsc_msr(void) {
    unsigned int low, high;
    asm volatile (
        "mov $0x10, %%ecx\n\t"   // ECX = MSR number (0x10 = IA32_TSC)
        "rdmsr\n\t"                  // EDX:EAX = TSC
        "mov %%eax, %0\n\t"
        "mov %%edx, %1"
        : "=r" (low), "=r" (high)
        : 
        : "%eax", "%edx", "%ecx"
    );
    return ((unsigned __int64)high << 32) | low;
}

Actually, RDTSC is easier:

unsigned __int64 read_tsc(void) {
    unsigned int low, high;
    asm volatile (
        "rdtsc"
        : "=a" (low), "=d" (high)
    );
    return ((unsigned __int64)high << 32) | low;
}

Note the "=a" and "=d" constraints—they tell GCC to use EAX and EDX for the output operands (which is where RDTSC puts the result).

5.3 Writing an MSR (Example: Setting the APIC Base)

Let’s say we want to write to the IA32_APIC_BASE MSR (0x1B) to change the APIC base address. This is something a kernel might do during initialization.

void write_apic_base(unsigned long long value) {
    unsigned int low = (unsigned int)value;
    unsigned int high = (unsigned int)(value >> 32);
    asm volatile (
        "mov $0x1B, %%ecx\n\t"   // ECX = MSR number (0x1B)
        "mov %0, %%eax\n\t"       // EAX = low 32 bits
        "mov %1, %%edx\n\t"       // EDX = high 32 bits
        "wrmsr"
        : 
        : "r" (low), "r" (high)
        : "%eax", "%edx", "%ecx"
    );
}

6. I/O Port Access in Drivers

Many devices (especially legacy ones) use I/O ports for communication. The x86 architecture has a separate I/O address space (64KB, from 0x0000 to 0xFFFF). To read or write I/O ports, you use the IN and OUT instructions.

6.1 IN and OUT Instructions

  • IN AL, DX: Read byte from I/O port DX into AL.
  • IN AX, DX: Read word from I/O port DX into AX.
  • IN EAX, DX: Read dword from I/O port DX into EAX.
  • OUT DX, AL: Write byte AL to I/O port DX.
  • OUT DX, AX: Write word AX to I/O port DX.
  • OUT DX, EAX: Write dword EAX to I/O port DX.

There are also versions with an immediate port number: IN AL, 0x60 (read from port 0x60, the keyboard controller data port).

6.2 I/O Port Access in Linux Drivers

The Linux kernel provides helper functions for I/O port access:

#include <linux/ioport.h>
#include <linux/io.h>

// Read byte from port
unsigned char inb(unsigned short port);

// Write byte to port
void outb(unsigned char value, unsigned short port);

// Read word from port
unsigned short inw(unsigned short port);

// Write word to port
void outw(unsigned short value, unsigned short port);

// Read dword from port
unsigned int inl(unsigned short port);

// Write dword to port
void outl(unsigned int value, unsigned short port);

These functions are implemented with inline assembly. For example, inb() might be implemented as:

static inline unsigned char inb(unsigned short port) {
    unsigned char result;
    asm volatile ("inb %1, %0" : "=a" (result) : "Nd" (port));
    return result;
}

The "Nd" constraint means “immediate operand (constant) or DX register.” For IN/OUT with a constant port number, the assembler uses the IN imm8, AL form; for a variable port number, it uses IN DX, AL.

6.3 Example: Reading the Keyboard Controller

The keyboard controller (i8042) uses I/O ports 0x60 (data) and 0x64 (status/command). Let’s read a scancode from the keyboard:

unsigned char read_keyboard_scancode(void) {
    unsigned char scancode;
    asm volatile (
        "inb $0x60, %0"   // Read byte from port 0x60 into scancode
        : "=a" (scancode)
    );
    return scancode;
}

Actually, you should check the keyboard controller’s status register (port 0x64) to see if there’s a scancode available. But this example illustrates the IN instruction.

7. Atomic Operations in Drivers

In a multi-core system, atomic operations are essential for synchronization. Let’s talk about the x86 instructions that provide atomicity, and how to use them in drivers.

7.1 The LOCK Prefix

The LOCK prefix (applied to certain instructions) locks the memory bus (or the cache line) for the duration of the instruction. This ensures that no other CPU can access the memory operand while the instruction is executing.

Instructions that can be prefixed with LOCK:

  • ADD, OR, ADC, SBB, AND, SUB, XOR: These do a read-modify-write atomically.
  • NOT: Also atomic with LOCK (but NOT doesn’t modify flags, so it’s less useful).
  • NEG: Also atomic with LOCK.
  • INC, DEC: Also atomic with LOCK.
  • XCHG (with memory operand): Always locked (implicit LOCK prefix).
  • CMPXCHG: Compare-and-swap, atomic with LOCK.

7.2 Atomic Increment with LOCK INC

Let’s say we have a global counter that multiple CPUs might increment concurrently. We need to increment it atomically:

volatile int counter = 0;

void atomic_increment(void) {
    asm volatile (
        "lock incl %0"   // Increment counter atomically
        : "=m" (counter)
        : "m" (counter)
    );
}

Actually, GCC provides a builtin for this: __sync_fetch_and_add(). But if you’re writing a driver that needs to run on a compiler without builtins, you’d use inline assembly.

7.3 Compare-and-Swap with LOCK CMPXCHG

The compare-and-swap (CAS) operation is the building block of many lock-free algorithms. It atomically compares a memory location with a expected value, and if they’re equal, it swaps in a new value.

The CMPXCHG instruction does CAS. With LOCK prefix, it’s atomic.

Syntax: LOCK CMPXCHG r/m32, r32

  • Compares EAX (or AL for byte, AX for word) with the destination operand (r/m32).
  • If equal, sets ZF (zero flag) and loads the source operand (r32) into the destination.
  • If not equal, clears ZF and loads the destination into EAX.

Example: implementing an atomic CAS function:

int atomic_cas(volatile int *ptr, int old_val, int new_val) {
    unsigned char result;
    asm volatile (
        "lock cmpxchg %3, %1\n\t"   // Compare [ptr] with EAX, swap with new_val if equal
        "setz %0"                      // Set result to 1 if ZF set (swap happened)
        : "=r" (result), "=m" (*ptr)
        : "a" (old_val), "r" (new_val), "m" (*ptr)
        : "cc"                          // We modify condition codes (flags)
    );
    return result;
}

This function returns 1 if the swap happened (the old value matched), 0 otherwise.

7.4 Atomic Operation Timing

Let me show you a timeline of atomic vs. non-atomic operations:

sequenceDiagram
    participant CPU1
    participant Memory
    participant CPU2
    
    Note over CPU1,CPU2: Non-Atomic Increment (Race Condition)
    CPU1->>Memory: Read counter (value=5)
    CPU2->>Memory: Read counter (value=5)
    CPU1->>Memory: Write counter (value=6)
    CPU2->>Memory: Write counter (value=6)
    Note over CPU1,CPU2: Both incremented, but counter is 6 (should be 7)!
    
    Note over CPU1,CPU2: Atomic Increment (LOCK INC)
    CPU1->>Memory: LOCK INC counter (value=5 -> 6)
    Note over CPU2: Waits for LOCK to release
    CPU1-->>CPU2: LOCK released
    CPU2->>Memory: LOCK INC counter (value=6 -> 7)
    Note over CPU1,CPU2: Correct: counter is 7

This sequence diagram shows the problem with non-atomic operations (race condition) and how the LOCK prefix solves it by ensuring exclusive access to the memory location.

8. Memory Barriers: MFENCE, LFENCE, SFENCE

As mentioned earlier, modern CPUs are out-of-order. This can cause problems in drivers, especially when dealing with memory-mapped I/O (MMIO) or communicating with other CPUs.

8.1 Why Memory Barriers Matter

Consider this code in a driver:

// Tell the device to start a DMA transfer
device_reg->command = DMA_START;

// Wait for the device to set the "done" bit
while (!(device_reg->status & DMA_DONE)) {
    cpu_relax();
}

Seems fine, right? But the CPU might reorder the write to device_reg->command to AFTER the read of device_reg->status. That would be a disaster—the CPU would read the “done” bit (which might be set from a previous operation) before starting the new DMA transfer.

To prevent this, you use a memory barrier (also called a “fence”):

// Tell the device to start a DMA transfer
device_reg->command = DMA_START;

// Memory barrier: ensure the write above completes before the read below
wmb();

// Wait for the device to set the "done" bit
while (!(device_reg->status & DMA_DONE)) {
    cpu_relax();
}

8.2 The x86 Memory Barrier Instructions

x86 has three memory barrier instructions:

  • MFENCE: Memory fence. Ensures that all loads and stores before the MFENCE complete before any loads/stores after the MFENCE.
  • LFENCE: Load fence. Ensures that all loads before the LFENCE complete before any loads after the LFENCE. (Doesn’t order stores.)
  • SFENCE: Store fence. Ensures that all stores before the SFENCE complete before any stores after the SFENCE. (Doesn’t order loads.)

On x86, MFENCE is a full barrier (orders both loads and stores). LFENCE and SFENCE are more specialized.

8.3 Using Memory Barriers in Linux Drivers

The Linux kernel provides portable memory barrier macros:

#include <linux/compiler.h>

// Write memory barrier (orders stores)
wmb();

// Read memory barrier (orders loads)
rmb();

// Full memory barrier (orders both loads and stores)
mb();

These macros expand to the appropriate inline assembly for the architecture. On x86, wmb() might be barrier() (a compiler barrier only, because x86 has strong memory ordering for stores), while mb() expands to asm volatile ("mfence" ::: "memory").

8.4 Memory Barrier Effect Visualization

Let me show you how memory barriers affect instruction ordering:

graph LR
    A[Without Memory Barrier] --> B[Store A]
    B --> C[Load X]
    C --> D[Store B]
    D --> E[CPU may reorder: Load X before Store A]
    
    F[With Memory Barrier] --> G[Store A]
    G --> H[MFENCE]
    H --> I[Load X]
    I --> J[Store B]
    J --> K[CPU guaranteed: Store A completes before Load X]
    
    style A fill:#ffcdd2,stroke:#c62828,stroke-width:2px
    style F fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px
    style H fill:#fff3e0,stroke:#e65100,stroke-width:2px

This diagram shows the effect of a memory barrier. Without it, the CPU can reorder a load before a store. With MFENCE, the store is guaranteed to complete before the load.

9. Writing a Simple Linux Character Device Driver with Inline Assembly

Alright, let’s put everything together and write a simple Linux character device driver that uses inline assembly. The driver will have a "read" operation that returns the current value of the CPU’s timestamp counter (RDTSC).

9.1 Character Device Driver Basics

A character device driver is a driver that handles byte-stream devices (like serial ports, keyboards, mice). In Linux, you register a character device with register_chrdev() (or the newer cdev_init()/cdev_add() interface), and you implement file_operations callbacks (read, write, open, release, etc.).

For this example, we’ll use the older register_chrdev() interface (simpler for illustration).

9.2 The Driver Code (With Inline Assembly)

Here’s the complete driver code:

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/uaccess.h>

#define DEVICE_NAME "tsc_device"
#define CLASS_NAME "tsc_class"

static int major_number;
static struct class *tsc_class = NULL;
static struct device *tsc_device = NULL;

// Read TSC using inline assembly
static inline unsigned long long read_tsc(void) {
    unsigned long low, high;
    asm volatile ("rdtsc" : "=a" (low), "=d" (high));
    return ((unsigned long long)high << 32) | low;
}

// File operations: read
static ssize_t tsc_read(struct file *filp, char *buffer, size_t len, loff_t *offset) {
    unsigned long long tsc = read_tsc();
    
    // Copy TSC value to user space
    if (copy_to_user(buffer, &tsc, sizeof(tsc))) {
        return -EFAULT;
    }
    
    return sizeof(tsc);
}

// File operations structure
static struct file_operations fops = {
    .owner = THIS_MODULE,
    .read = tsc_read,
};

// Module init function
static int __init tsc_init(void) {
    // Register character device
    major_number = register_chrdev(0, DEVICE_NAME, &fops);
    if (major_number < 0) {
        printk(KERN_ALERT "TSC device: failed to register major number\n");
        return major_number;
    }
    
    // Create device class
    tsc_class = class_create(THIS_MODULE, CLASS_NAME);
    if (IS_ERR(tsc_class)) {
        unregister_chrdev(major_number, DEVICE_NAME);
        printk(KERN_ALERT "TSC device: failed to register device class\n");
        return PTR_ERR(tsc_class);
    }
    
    // Create device
    tsc_device = device_create(tsc_class, NULL, MKDEV(major_number, 0), NULL, DEVICE_NAME);
    if (IS_ERR(tsc_device)) {
        class_destroy(tsc_class);
        unregister_chrdev(major_number, DEVICE_NAME);
        printk(KERN_ALERT "TSC device: failed to create device\n");
        return PTR_ERR(tsc_device);
    }
    
    printk(KERN_INFO "TSC device: device created successfully\n");
    return 0;
}

// Module exit function
static void __exit tsc_exit(void) {
    device_destroy(tsc_class, MKDEV(major_number, 0));
    class_unregister(tsc_class);
    class_destroy(tsc_class);
    unregister_chrdev(major_number, DEVICE_NAME);
    printk(KERN_INFO "TSC device: device unregistered\n");
}

module_init(tsc_init);
module_exit(tsc_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Your Name");
MODULE_DESCRIPTION("TSC Character Device Driver");
MODULE_VERSION("0.1");

9.3 Compiling and Testing the Driver

To compile this driver, you need a Linux kernel build system (kbuild). Create a Makefile:

obj-m += tsc_driver.o

all:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Then compile:

$ make

Load the driver:

$ sudo insmod tsc_driver.ko
$ sudo chmod 666 /dev/tsc_device

Test it:

// test_tsc.c
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>

int main() {
    int fd = open("/dev/tsc_device", O_RDONLY);
    if (fd < 0) {
        perror("open");
        return 1;
    }
    
    unsigned long long tsc;
    read(fd, &tsc, sizeof(tsc));
    printf("TSC: %llu\n", tsc);
    
    close(fd);
    return 0;
}

$ gcc -o test_tsc test_tsc.c
$ ./test_tsc
TSC: 12345678901234

10. Complete Example: A Driver That Uses RDMSR to Read the CPU’s Timestamp Counter

Wait, we already did that in Section 9. But let’s do a different example: a driver that uses RDMSR to read an MSR that isn’t accessible via a simpler instruction (like RDTSC).

Let’s read the IA32_PLATFORM_ID MSR (0x17), which contains the platform ID (a value that identifies the CPU stepping, model, family, etc.).

10.1 The Driver Code (Using RDMSR)

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/uaccess.h>

#define DEVICE_NAME "msr_device"
#define CLASS_NAME "msr_class"
#define MSR_PLATFORM_ID 0x17

static int major_number;
static struct class *msr_class = NULL;
static struct device *msr_device = NULL;

// Read MSR using inline assembly (RDMSR)
static inline unsigned long long read_msr(unsigned int msr) {
    unsigned int low, high;
    asm volatile (
        "mov %2, %%ecx\n\t"   // ECX = MSR number
        "rdmsr\n\t"             // EDX:EAX = MSR value
        "mov %%eax, %0\n\t"
        "mov %%edx, %1"
        : "=r" (low), "=r" (high)
        : "r" (msr)
        : "%eax", "%edx", "%ecx"
    );
    return ((unsigned long long)high << 32) | low;
}

// File operations: read
static ssize_t msr_read(struct file *filp, char *buffer, size_t len, loff_t *offset) {
    unsigned long long msr_val = read_msr(MSR_PLATFORM_ID);
    
    if (copy_to_user(buffer, &msr_val, sizeof(msr_val))) {
        return -EFAULT;
    }
    
    return sizeof(msr_val);
}

static struct file_operations fops = {
    .owner = THIS_MODULE,
    .read = msr_read,
};

// (init and exit functions same as before, with device name "msr_device")

This driver reads the IA32_PLATFORM_ID MSR and returns it to user space. It uses RDMSR (via inline assembly) to read the MSR.

10.2 Inline Assembly in Driver Stack

Let me show you where inline assembly fits in the driver stack:

graph TD
    A[User Space Application] -->|"read()" system call| B[Virtual File System VFS]
    B --> C[Character Device Driver]
    C --> D[file_operations.read callback]
    D --> E[Inline Assembly: RDTSC / RDMSR]
    E --> F[CPU Hardware]
    
    F -->|"Returns TSC or MSR value"| E
    E -->|"Returns value to"| D
    D -->|"copy_to_user()"| B
    B -->|"Returns to user space"| A
    
    style C fill:#e3f2fd,stroke:#1565c0,stroke-width:2px
    style E fill:#fff3e0,stroke:#e65100,stroke-width:2px
    style F fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px

This diagram shows the flow of a read() system call from a user-space application, through the VFS, to the character device driver, to the inline assembly that reads the TSC or MSR from the CPU hardware.

Conclusion: The Power and Responsibility of Kernel-Level Assembly

We’ve covered a lot of ground in this article. From the reasons why drivers need assembly (accessing control registers, MSRs, I/O ports, atomic operations, memory barriers), to the syntax of inline assembly in GCC and MSVC, to complete examples of Linux character device drivers that use inline assembly.

Writing assembly in drivers is powerful—it lets you do things that C simply can’t express. But it’s also dangerous. A bug in a driver (especially in inline assembly) can crash the kernel (cause a kernel panic), corrupt memory, or introduce security vulnerabilities.

If you’re going to write inline assembly in drivers, here are my recommendations:

  1. Use helper functions when possible. Linux provides inb(), outb(), rdtsc(), mb(), etc. Use them instead of writing inline assembly yourself.
  2. Be careful with memory barriers. If your driver communicates with a device or another CPU, use memory barriers to ensure correct ordering.
  3. Test thoroughly. Kernel code is hard to debug. Use printk() (the kernel equivalent of printf) to log debug messages. Use a virtual machine for testing (so a kernel panic doesn’t crash your development system).
  4. Read the documentation. The Linux kernel documentation (in Documentation/) has guides on writing device drivers, using inline assembly, and memory barriers.
  5. Read other drivers. The Linux kernel source (available at kernel.org) has thousands of example drivers. Read them to learn the conventions and best practices.

And with that, we’ve reached the end of this article—and the end of the series on assembly language programming. I hope you’ve enjoyed the journey from basic assembly concepts to the depths of interrupt handling, performance optimization, reverse engineering, shellcode, and driver development. Assembly language is a beautiful, terrifying, and incredibly powerful tool. Use it wisely.

Course: Assembly Language Programming From Low-Level Logic to System Kernel
Share your love

Trending now

Exploring Web 3D Core Technologies Development
Assembly Language Programming From Low-Level Logic to System Kernel
WebRTC Advanced Guide：Media, Transport & Architecture
WebSocket Deep Guide：Building High-Performance Realtime Systems