Lesson 25-Advanced Memory Management

Memory Leak Detection Tools

Valgrind

Valgrind is a powerful suite of memory debugging and analysis tools, primarily used to detect memory leaks, illegal memory accesses, and other issues.

Installation and Usage

# Install Valgrind (Linux)
sudo apt-get install valgrind  # Ubuntu/Debian
sudo yum install valgrind      # CentOS/RHEL

# Use Memcheck to detect memory leaks
valgrind --leak-check=full ./your_program

Common Output Analysis

==12345== Memcheck, a memory error detector
==12345== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==12345== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==12345== Command: ./memory_leak_example
==12345== 
==12345== 
==12345== HEAP SUMMARY:
==12345==     in use at exit: 40 bytes in 1 blocks
==12345==   total heap usage: 3 allocs, 2 frees, 72,704 bytes allocated
==12345== 
==12345== 40 bytes in 1 blocks are definitely lost in loss record 1 of 1
==12345==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==12345==    by 0x4005F4: main (memory_leak_example.c:5)
==12345== 
==12345== LEAK SUMMARY:
==12345==    definitely lost: 40 bytes in 1 blocks
==12345==    indirectly lost: 0 bytes in 0 blocks
==12345==      possibly lost: 0 bytes in 0 blocks
==12345==    still reachable: 0 bytes in 0 blocks
==12345==         suppressed: 0 bytes in 0 blocks
==12345== 
==12345== For counts of detected and suppressed errors, rerun with: -v
==12345== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Key Metrics Explained

  • definitely lost: Confirmed memory leak – allocated memory that was not freed and is no longer accessible
  • indirectly lost: Indirect memory leak – memory that becomes inaccessible due to other leaks
  • possibly lost: Possible memory leak – pointer still exists but no longer points to the start of the allocated block
  • still reachable: Still reachable memory – memory accessible at program exit but not freed

AddressSanitizer (ASan)

AddressSanitizer is a memory error detection tool built into GCC and Clang, faster than Valgrind.

Enable at Compile Time

gcc -fsanitize=address -g your_program.c -o your_program

Common Error Types

  1. Heap Buffer Overflow:
int *arr = malloc(10 * sizeof(int));
arr[10] = 42;  // Out-of-bounds write
  1. Stack Buffer Overflow:
char buffer[10];
strcpy(buffer, "This is a very long string");  // Buffer overflow
  1. Use After Free:
int *p = malloc(sizeof(int));
free(p);
*p = 42;  // Use after free
  1. Memory Leak:
void leak() {
	int *p = malloc(sizeof(int));
	// Forgot to free(p)
}

ASan Output Example

=================================================================
==12345==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000eff4 at pc 0x000000400a3b bp 0x7ffd12345678 sp 0x7ffd12345670
WRITE of size 4 at 0x60200000eff4 thread T0
    #0 0x400a3a in main /path/to/your_program.c:5
    #1 0x7f8e1b2b82e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)
    #2 0x4008d8 in _start (/path/to/your_program+0x4008d8)

Memory Pools and Custom Allocators

Memory Pool Basic Concepts

A memory pool is a technique that pre-allocates a large block of memory and allocates smaller chunks from it when needed, reducing frequent system calls and memory fragmentation.

Simple Memory Pool Implementation

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

typedef struct MemoryBlock {
    size_t size;
    int free;
    struct MemoryBlock* next;
} MemoryBlock;

#define POOL_SIZE 1024 * 1024  // 1MB memory pool

static char memory_pool[POOL_SIZE];
static MemoryBlock* free_list = NULL;

void initialize_memory_pool() {
    free_list = (MemoryBlock*)memory_pool;
    free_list->size = POOL_SIZE - sizeof(MemoryBlock);
    free_list->free = 1;
    free_list->next = NULL;
}

void* my_malloc(size_t size) {
    if (!free_list) {
        initialize_memory_pool();
    }

    MemoryBlock* current = free_list;
    MemoryBlock* previous = NULL;

    while (current) {
        if (current->free && current->size >= size) {
            // Found a sufficiently large free block
            if (current->size > size + sizeof(MemoryBlock)) {
                // Split the block
                MemoryBlock* new_block = (MemoryBlock*)((char*)current + sizeof(MemoryBlock) + size);
                new_block->size = current->size - size - sizeof(MemoryBlock);
                new_block->free = 1;
                new_block->next = current->next;

                current->size = size;
                current->free = 0;
                current->next = new_block;
            } else {
                current->free = 0;
            }

            return (void*)(current + 1);
        }
        previous = current;
        current = current->next;
    }

    return NULL;  // No suitable block found
}

void my_free(void* ptr) {
    if (!ptr) return;

    MemoryBlock* block = (MemoryBlock*)ptr - 1;
    block->free = 1;

    // Merge adjacent free blocks
    MemoryBlock* current = free_list;
    while (current && current->next) {
        if (current->free && current->next->free) {
            current->size += sizeof(MemoryBlock) + current->next->size;
            current->next = current->next->next;
        } else {
            current = current->next;
        }
    }
}

// Test code
int main() {
    int* arr = (int*)my_malloc(10 * sizeof(int));
    for (int i = 0; i < 10; i++) {
        arr[i] = i;
    }
    
    for (int i = 0; i < 10; i++) {
        printf("%d ", arr[i]);
    }
    printf("\n");
    
    my_free(arr);
    return 0;
}

Memory Pool Optimization Techniques

  1. Multi-Size Block Management:
  • Maintain linked lists of different-sized memory blocks
  • Quickly find appropriately sized blocks
  1. Thread Safety:
#include <pthread.h>

pthread_mutex_t pool_mutex = PTHREAD_MUTEX_INITIALIZER;

void* thread_safe_malloc(size_t size) {
	pthread_mutex_lock(&pool_mutex);
	void* ptr = my_malloc(size);
	pthread_mutex_unlock(&pool_mutex);
	return ptr;
}
  1. Memory Alignment:
  • Ensure allocated memory addresses meet specific alignment requirements
  • Common alignment values: 4, 8, 16, 32, 64 bytes

Garbage Collection Mechanisms

Reference Counting

Reference counting is a simple garbage collection technique where each object maintains a reference counter.

Implementation Example

#include <stdio.h>
#include <stdlib.h>

typedef struct RefCounted {
    int ref_count;
    // Other data members
} RefCounted;

RefCounted* create_ref_counted() {
    RefCounted* obj = (RefCounted*)malloc(sizeof(RefCounted));
    if (obj) {
        obj->ref_count = 1;
    }
    return obj;
}

void retain(RefCounted* obj) {
    if (obj) {
        obj->ref_count++;
    }
}

void release(RefCounted* obj) {
    if (obj) {
        obj->ref_count--;
        if (obj->ref_count == 0) {
            free(obj);
        }
    }
}

// Test code
int main() {
    RefCounted* obj = create_ref_counted();
    retain(obj);  // ref_count = 2
    release(obj); // ref_count = 1
    release(obj); // ref_count = 0, object is freed
    return 0;
}

Reference Counting Drawbacks

  1. Circular Reference Problem:
  • Two objects referencing each other, preventing deallocation
  • Requires weak references to resolve
  1. Frequent reference count updates impact performance

Mark-and-Sweep Algorithm

Mark-and-sweep is a more complex garbage collection algorithm divided into two phases:

  1. Mark Phase: Starting from root objects, mark all reachable objects
  2. Sweep Phase: Reclaim all unmarked objects

Simple Implementation

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>

typedef struct GCObject {
    bool marked;
    struct GCObject* next;
    // Other data members
} GCObject;

GCObject* gc_root = NULL;
GCObject* gc_objects = NULL;

void gc_register(GCObject* obj) {
    obj->next = gc_objects;
    gc_objects = obj;
    obj->marked = false;
}

void gc_mark(GCObject* obj) {
    if (!obj || obj->marked) return;
    
    obj->marked = true;
    
    // Assume object has pointers to other objects
    // Recursive marking needs to be implemented based on actual object structure
}

void gc_collect() {
    // Mark phase
    GCObject* current = gc_root;
    while (current) {
        gc_mark(current);
        current = current->next;
    }
    
    // Sweep phase
    GCObject** prev = &gc_objects;
    current = gc_objects;
    
    while (current) {
        if (!current->marked) {
            *prev = current->next;
            free(current);
            current = *prev;
        } else {
            current->marked = false;  // Reset mark bit
            prev = ¤t->next;
            current = current->next;
        }
    }
}

// Test code
int main() {
    GCObject* obj1 = (GCObject*)malloc(sizeof(GCObject));
    GCObject* obj2 = (GCObject*)malloc(sizeof(GCObject));
    
    gc_register(obj1);
    gc_register(obj2);
    
    // Establish reference relationships
    // Needs to be implemented based on actual object structure
    
    gc_collect();
    return 0;
}

Memory Alignment and Optimization

Memory Alignment Concept

Memory alignment means that data in memory must be located at addresses that are integer multiples of its size. For example:

  • int typically requires 4-byte alignment
  • double typically requires 8-byte alignment

Using alignas to Specify Alignment

#include <stdalign.h>

struct alignas(16) AlignedStruct {
    char c;
    int i;
    double d;
};

int main() {
    printf("Alignment of AlignedStruct: %zu\n", alignof(AlignedStruct));
    return 0;
}

Cache-Friendly Design

  1. Data Locality:
    • Place frequently accessed data together in adjacent memory locations
    • Use array of structs instead of struct of arrays
// Cache-friendly array of structs
struct Point {
    float x, y, z;
};

struct Point points[1000];  // All x coordinates are contiguous, then y, then z

// Less cache-friendly struct of arrays
struct PointsArray {
    struct Point points[1000];
};
  1. Avoid False Sharing:
    • In multi-threaded environments, ensure variables modified by different threads are in different cache lines
#include <stdatomic.h>

struct PaddedCounter {
    atomic_int counter;
    char padding[64 - sizeof(atomic_int)];  // Assume cache line size is 64 bytes
};

struct PaddedCounter counters[4];  // Each counter has its own cache line

Stack Overflow Protection

Buffer Overflow Protection

  1. Use Safe Functions:
// Unsafe strcpy
// strcpy(dest, src);

// Safe version
strncpy(dest, src, dest_size - 1);
dest[dest_size - 1] = '\0';
  1. Boundary Checking:
void safe_copy(char* dest, const char* src, size_t dest_size) {
	if (dest_size == 0) return;
	
	size_t src_len = strlen(src);
	size_t copy_len = src_len < dest_size - 1 ? src_len : dest_size - 1;
	
	memcpy(dest, src, copy_len);
	dest[copy_len] = '\0';
}

Stack Protection Mechanisms

  1. Canary Value:
  • Insert a special value (canary) in the stack frame
  • Check if the value is modified before function return
// Enable stack protection at compile time (-fstack-protector-strong)
// GCC automatically inserts canary check code

void vulnerable_function() {
    char buffer[8];
    // If an attacker overflows buffer, it will corrupt the canary value
    // Causing program termination
}
  1. Stack Overflow Detection Tools:
  • AddressSanitizer: Detects stack buffer overflows
  • Stack Canary: As described above
  • Guard Pages: Set inaccessible memory pages between stack and heap

Dynamic Stack Growth Detection

Some systems support dynamic stack growth detection:

#include <signal.h>
#include <stdio.h>
#include <stdlib.h>

void stack_overflow_handler(int sig) {
    fprintf(stderr, "Stack overflow detected!\n");
    exit(1);
}

void setup_stack_protection() {
    // Set signal handler to catch stack overflow
    signal(SIGSEGV, stack_overflow_handler);
    
    // On some systems, smaller stack size can be set for testing
    // This is just an example; actual implementation is more complex
}

int recursive_function(int n) {
    char buffer[1024];  // Large local variable
    if (n <= 0) return 0;
    return recursive_function(n - 1);
}

int main() {
    setup_stack_protection();
    recursive_function(10000);  // May cause stack overflow
    return 0;
}

Summary

This article provides a detailed introduction to memory management techniques in C, including:

  1. Memory Leak Detection:
    • Valgrind: Comprehensive but slower
    • AddressSanitizer: Integrated in compiler, fast
  2. Custom Memory Allocators:
    • Memory pool design
    • Multi-size block management
    • Thread safety considerations
  3. Garbage Collection Mechanisms:
    • Reference counting and its limitations
    • Mark-and-sweep algorithm principles
  4. Memory Alignment and Optimization:
    • alignas/alignof usage
    • Cache-friendly design principles
  5. Stack Overflow Protection:
    • Canary value protection
    • Buffer overflow checking
    • Safe function usage

These techniques are crucial for writing safe and efficient C programs. In actual development, appropriate combinations of techniques should be selected based on specific needs, and good memory management practices should be followed.

Course: Mastering C Programming From Basics to Advanced Techniques
Share your love

Trending now

Exploring Web 3D Core Technologies Development
Assembly Language Programming From Low-Level Logic to System Kernel
WebRTC Advanced Guide：Media, Transport & Architecture
WebSocket Deep Guide：Building High-Performance Realtime Systems