Dynamic Analysis Introduction
Dynamic analysis involves running a program in a controlled environment and observing its runtime behavior (including instruction execution, memory access, and system interactions) to analyze the code. Compared to static analysis, dynamic analysis can reveal runtime logic, decrypt dynamically generated content, and detect hidden behaviors. This tutorial covers the following core topics:
- Debugging Fundamentals: Master breakpoints, register/memory monitoring, and single-step execution to control program execution.
- Dynamic Behavior Analysis: Monitor API calls, network activity, file system, and registry operations to understand program-system interactions.
This tutorial will explain these techniques step by step, providing Python code (combined with debugger APIs and monitoring tools) and practical cases using tools like Radare2, GDB, x64dbg, and Wireshark.
Preparation
Before starting, ensure the following tools are installed:
- Python 3.x: For writing debugging and monitoring scripts.
- Capstone: Disassembly engine (
pip install capstone). - pywin32 (Windows): For Windows debugging APIs (
pip install pywin32). - GDB: Linux/macOS debugger (included with the system or installed via package manager).
- x64dbg: Windows debugger (download from the official website).
- Radare2: Open-source reverse engineering framework (install from the official website).
- API Monitor: Windows API call monitoring tool.
- Procmon: Windows system monitoring tool (Sysinternals suite).
- Wireshark: Network traffic analysis tool.
- Fiddler: HTTP/HTTPS proxy tool.
- Sample Binary Files: PE, ELF, or Mach-O files for x86/x64/ARM architectures (obtainable from open-source projects or CTF challenges).
- Virtual Machine: Such as VMware or VirtualBox, for safely running malicious samples.
Debugging Fundamentals
Debugging is the process of controlling program execution, inspecting state, and modifying behavior to analyze code. Debuggers (such as GDB, x64dbg) provide features like breakpoints, register/memory monitoring, and single-step execution, forming the foundation of dynamic analysis.
Breakpoint Types
Breakpoints pause program execution, allowing the analyst to inspect the state. Common breakpoint types include:
- Software Breakpoints:
- Principle: Replace the target instruction with an interrupt instruction (e.g., x86
int 3,0xCC). - Advantages: Easy to set, unlimited quantity.
- Disadvantages: Easily detected (anti-debugging techniques may check for
0xCC). - Example: Set a software breakpoint at address
0x401000.
- Principle: Replace the target instruction with an interrupt instruction (e.g., x86
- Hardware Breakpoints:
- Principle: Use processor debug registers (e.g., x86 DR0-DR3) to monitor specific addresses.
- Advantages: Hard to detect, suitable for monitoring specific instructions or memory.
- Disadvantages: Limited quantity (usually 4).
- Example: Monitor execution at function entry
0x401000.
- Conditional Breakpoints:
- Principle: Pause when a specified condition (e.g., register value or memory content) is met.
- Advantages: Precise control, reduces irrelevant interruptions.
- Disadvantages: High performance overhead.
- Example: Pause when
eax == 0x1234.
Example: Setting Breakpoints in x64dbg
Set breakpoints in x64dbg:
- Open the binary, navigate to the target address (e.g.,
0x401000). - Press
F2to set a software breakpoint; the icon turns red. - Right-click and select “Hardware Breakpoint” to set a hardware breakpoint.
- In the “Breakpoints” window, add a conditional breakpoint (e.g.,
eax == 0x1234).
Example: Setting Breakpoints in GDB
Set breakpoints in GDB:
gdb sample.elf
(gdb) break *0x401000 # Software breakpoint
(gdb) hbreak *0x401000 # Hardware breakpoint
(gdb) condition 1 eax==0x1234 # Conditional breakpoint (assuming breakpoint number 1)
(gdb) run
Register and Memory Monitoring
Registers and memory store the runtime state of a program; monitoring them tracks data flow and control flow.
- Register Monitoring:
- Key Registers:
- x86/x64:
eax/rax(return value),esp/rsp(stack pointer),eip/rip(instruction pointer). - ARM:
r0-r3(parameters/return values),sp(stack pointer),lr(return address),pc(instruction pointer).
- x86/x64:
- Tools: Debugger register windows (e.g., x64dbg “Registers” panel, GDB
info registers).
- Key Registers:
- Memory Monitoring:
- Methods: Set memory breakpoints (read/write/execute) or directly view memory contents.
- Tools: x64dbg “Memory” panel, GDB
xcommand.
Example: GDB Register and Memory Monitoring
gdb sample.elf
(gdb) break main
(gdb) run
(gdb) info registers # View registers
(gdb) x/10x $esp # View top 10 words of the stack
(gdb) watch *$esp # Set memory write breakpoint
(gdb) continue
Output Example:
rax 0x0 0
rbx 0x7fffffffde48 140737488346696
rsp 0x7fffffffde38 0x7fffffffde38
rip 0x401000 0x401000 <main>
...
0x7fffffffde38: 0x00401234 0x00000000 0x7fffffffdf08 0x00000001
Example: x64dbg Memory Monitoring
- Open x64dbg, load the binary.
- In the “Memory” panel, right-click the target address (e.g.,
0x401000), select “Set Breakpoint” -> “Read/Write”. - Run the program; check memory contents when the breakpoint triggers.
Single-Step Execution and Tracing
Single-step execution and tracing control instruction-by-instruction or function-by-function execution, ideal for detailed logic analysis.
- Single-Step Execution:
- Step Into: Execute the next instruction, including function calls (e.g., GDB
stepi, x64dbgF7). - Step Over: Skip function calls, execute the next instruction (e.g., GDB
nexti, x64dbgF8). - Step Out: Execute until the current function returns (e.g., GDB
finish, x64dbgShift+F8).
- Step Into: Execute the next instruction, including function calls (e.g., GDB
- Tracing:
- Record instruction execution path, register changes, or memory access.
- Tools: x64dbg “Trace” function, GDB
tracecommand.
Example: GDB Single-Step Execution
gdb sample.elf
(gdb) break main
(gdb) run
(gdb) disassemble main
(gdb) si # Step into
(gdb) ni # Step over
(gdb) info registers
Example: x64dbg Tracing
- Load the binary in x64dbg, set a breakpoint.
- Press
F7to step into, observe register and memory changes. - Enable tracing in the “Trace” panel to record the execution path.
Code Example: Python Debugger (Windows)
The following code uses pywin32 to implement a simple Windows debugger, setting breakpoints and monitoring registers:
import win32api
import win32con
import win32process
import struct
def debug_process(executable):
# Start the process
startup_info = win32process.STARTUPINFO()
process_info = win32process.CreateProcess(None, executable, None, None, False,
win32con.DEBUG_PROCESS, None, None, startup_info)
h_process = process_info[0]
# Set software breakpoint (0xCC)
target_address = 0x401000
old_byte = win32api.ReadProcessMemory(h_process, target_address, 1)
win32api.WriteProcessMemory(h_process, target_address, b"\xCC")
# Debug loop
while True:
debug_event = win32process.DebugEvent()
if not win32process.GetDebugEvent(debug_event):
break
if debug_event.dwDebugEventCode == win32con.EXCEPTION_DEBUG_EVENT:
if debug_event.u.Exception.ExceptionRecord.ExceptionCode == win32con.EXCEPTION_BREAKPOINT:
print(f"Breakpoint hit at 0x{debug_event.u.Exception.ExceptionRecord.ExceptionAddress:08x}")
# Read registers
context = win32process.GetThreadContext(process_info[1])
print(f"EAX: 0x{context.Eax:08x}, EIP: 0x{context.Eip:08x}")
# Restore original instruction
win32api.WriteProcessMemory(h_process, target_address, old_byte)
context.Eip -= 1 # Rewind EIP
win32process.SetThreadContext(process_info[1], context)
win32process.ContinueDebugEvent(debug_event.dwProcessId, debug_event.dwThreadId, win32con.DBG_CONTINUE)
debug_process("sample.exe")
Output Example:
Breakpoint hit at 0x00401000
EAX: 0x00000000, EIP: 0x00401000
Debugging Fundamentals Notes
- Anti-Debugging Techniques: Malware may detect breakpoints or debuggers; use stealth debugging (e.g., hardware breakpoints).
- Performance Overhead: Conditional breakpoints and tracing may slow execution; optimize conditions.
- Environment Isolation: Debug in a virtual machine to prevent host infection by malicious code.
- Tool Selection: x64dbg is suitable for Windows GUI debugging, GDB for Linux command-line debugging.
Dynamic Behavior Analysis
Dynamic behavior analysis monitors a program’s runtime behavior (such as API calls, network activity, file system, and registry operations) to understand its functionality and intent. This is critical for analyzing malware, detecting vulnerabilities, or verifying program behavior.
API Call Monitoring
API calls reflect program interactions with the operating system, such as file operations, network requests, or process management. Monitoring API calls reveals key logic (e.g., malware C2 communication).
- Tools:
- API Monitor: Windows tool, captures API calls and parameters.
- Procmon: Sysinternals tool, monitors system calls.
- Frida: Cross-platform dynamic instrumentation framework, ideal for scripted monitoring.
Example: Using API Monitor
- Open API Monitor, load the target process (e.g.,
sample.exe). - Select APIs to monitor (e.g.,
kernel32.dll’sCreateFileA). - Run the program, view the call log.
Output Example:
CreateFileA("C:\\test.txt", GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL) -> 0x1234
WriteFile(0x1234, "Hello", 5, 0x12345678, NULL) -> TRUE
Example: Frida API Call Monitoring
The following Frida script monitors kernel32.dll’s CreateFileA:
// create_file_hook.js
const CreateFileA = Module.findExportByName("kernel32.dll", "CreateFileA");
Interceptor.attach(CreateFileA, {
onEnter(args) {
console.log("CreateFileA called with file: " + Memory.readUtf8String(args[0]));
},
onLeave(retval) {
console.log("CreateFileA returned: " + retval);
}
});
Run:
frida -f sample.exe -l create_file_hook.js
Output Example:
CreateFileA called with file: C:\test.txt
CreateFileA returned: 0x1234
Code Example: Python API Call Monitoring
The following code uses pywin32 to intercept CreateFileA:
import win32api
import win32process
import ctypes
def hook_api(process_handle, api_name, dll_name):
# Get API address
dll = ctypes.WinDLL(dll_name)
api_addr = ctypes.cast(getattr(dll, api_name), ctypes.c_void_p).value
# Inject breakpoint
old_byte = win32api.ReadProcessMemory(process_handle, api_addr, 1)
win32api.WriteProcessMemory(process_handle, api_addr, b"\xCC")
return old_byte, api_addr
def monitor_api(executable):
process_info = win32process.CreateProcess(None, executable, None, None, False,
win32con.DEBUG_PROCESS, None, None,
win32process.STARTUPINFO())
h_process = process_info[0]
# Hook CreateFileA
old_byte, api_addr = hook_api(h_process, "CreateFileA", "kernel32.dll")
# Debug loop
while True:
debug_event = win32process.DebugEvent()
if not win32process.GetDebugEvent(debug_event):
break
if debug_event.dwDebugEventCode == win32con.EXCEPTION_DEBUG_EVENT:
if debug_event.u.Exception.ExceptionRecord.ExceptionAddress == api_addr:
print("CreateFileA called")
context = win32process.GetThreadContext(process_info[1])
# Read filename (first parameter on stack)
filename_ptr = win32api.ReadProcessMemory(h_process, context.Esp + 4, 4)
filename = win32api.ReadProcessMemory(h_process, struct.unpack("I", filename_ptr)[0], 100).decode("ascii")
print(f"Filename: {filename}")
# Restore original instruction
win32api.WriteProcessMemory(h_process, api_addr, old_byte)
context.Eip -= 1
win32process.SetThreadContext(process_info[1], context)
win32process.ContinueDebugEvent(debug_event.dwProcessId, debug_event.dwThreadId, win32con.DBG_CONTINUE)
monitor_api("sample.exe")
Output Example:
CreateFileA called
Filename: C:\test.txt
Network Behavior Analysis
Network behavior analysis monitors a program’s network activity, such as HTTP requests, DNS queries, or C2 communication. Common tools include:
- Wireshark: Captures and analyzes network packets.
- Fiddler: Proxy tool, focused on HTTP/HTTPS traffic.
- tcpdump: Linux command-line tool, captures raw packets.
Example: Wireshark Analysis
- Start Wireshark, select a network interface (e.g., Wi-Fi).
- Set a filter (e.g.,
httportcp.port == 80). - Run the target program, observe traffic.
Output Example:
GET /api/data HTTP/1.1
Host: example.com
User-Agent: SampleApp/1.0
Example: Fiddler HTTPS Analysis
- Open Fiddler, enable HTTPS decryption (Tools -> Options -> HTTPS).
- Run the program, view requests and responses.
Code Example: Python Network Traffic Capture
Use scapy to capture HTTP traffic:
from scapy.all import *
def packet_callback(packet):
if packet.haslayer(TCP) and packet.haslayer(Raw):
if packet[TCP].dport == 80 or packet[TCP].sport == 80:
payload = packet[Raw].load.decode("ascii", errors="ignore")
if "GET" in payload or "POST" in payload:
print("=== HTTP Request ===")
print(payload)
sniff(filter="tcp port 80", prn=packet_callback, count=10)
Output Example:
=== HTTP Request ===
GET /index.html HTTP/1.1
Host: example.com
File System and Registry Tracking
File system and registry operations reflect a program’s persistence behavior (e.g., creating files, modifying configurations). Monitoring these reveals malicious actions (e.g., ransomware encrypting files).
- Tools:
- Procmon: Windows tool, monitors file and registry operations.
- strace: Linux tool, traces system calls.
- lsof: Lists open files.
Example: Procmon Monitoring
- Open Procmon, set a filter (e.g.,
Process Name is sample.exe). - Run the program, view file and registry operations.
Output Example:
sample.exe: CreateFile C:\test.txt
sample.exe: RegSetValue HKLM\Software\Sample
Example: strace File Operation Tracking
On Linux, use strace:
strace -e openat,write ./sample.elf
Output Example:
openat(AT_FDCWD, "test.txt", O_WRONLY|O_CREAT|O_TRUNC, 0644) = 3
write(3, "Hello", 5) = 5
Code Example: Python File Operation Monitoring
The following code uses watchdog to monitor file system changes:
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
class FileMonitor(FileSystemEventHandler):
def on_created(self, event):
print(f"File created: {event.src_path}")
def on_modified(self, event):
print(f"File modified: {event.src_path}")
observer = Observer()
observer.schedule(FileMonitor(), path=".", recursive=False)
observer.start()
try:
while True:
pass
except KeyboardInterrupt:
observer.stop()
observer.join()
Output Example:
File created: test.txt
File modified: test.txt
Dynamic Behavior Analysis Notes
- Sandbox Evasion: Malware may detect virtual machines or sandboxes; use stealth environments.
- Encrypted Traffic: HTTPS traffic requires decryption (e.g., Fiddler HTTPS decryption).
- Performance Overhead: Monitoring tools may impact performance; optimize filters.
- Tool Combination: Combine Procmon, Wireshark, and Frida for comprehensive behavior capture.
Comprehensive Example: Dynamic Malware Analysis
The following code combines debugging and behavior monitoring to analyze a Windows PE file:
import win32api
import win32process
import pefile
from scapy.all import *
def analyze_malware(file_path):
# Load PE file, check import table
pe = pefile.PE(file_path)
print("=== Imported APIs ===")
for entry in pe.DIRECTORY_ENTRY_IMPORT:
for imp in entry.imports:
print(f"{entry.dll.decode()}: {imp.name.decode()}")
# Debug and set breakpoint
startup_info = win32process.STARTUPINFO()
process_info = win32process.CreateProcess(None, file_path, None, None, False,
win32con.DEBUG_PROCESS, None, None, startup_info)
h_process = process_info[0]
# Set breakpoint at CreateFileA
dll = ctypes.WinDLL("kernel32.dll")
api_addr = ctypes.cast(getattr(dll, "CreateFileA"), ctypes.c_void_p).value
old_byte = win32api.ReadProcessMemory(h_process, api_addr, 1)
win32api.WriteProcessMemory(h_process, api_addr, b"\xCC")
# Monitor network
def packet_callback(packet):
if packet.haslayer(Raw) and (packet[TCP].dport == 80 or packet[TCP].sport == 80):
print("HTTP Packet:", packet[Raw].load.decode("ascii", errors="ignore"))
sniff(filter="tcp port 80", prn=packet_callback, count=5, store=False, timeout=10)
# Debug loop
while True:
debug_event = win32process.DebugEvent()
if not win32process.GetDebugEvent(debug_event):
break
if debug_event.dwDebugEventCode == win32con.EXCEPTION_DEBUG_EVENT:
if debug_event.u.Exception.ExceptionRecord.ExceptionAddress == api_addr:
print("CreateFileA called")
context = win32process.GetThreadContext(process_info[1])
filename_ptr = win32api.ReadProcessMemory(h_process, context.Esp + 4, 4)
filename = win32api.ReadProcessMemory(h_process, struct.unpack("I", filename_ptr)[0], 100).decode("ascii")
print(f"Filename: {filename}")
win32api.WriteProcessMemory(h_process, api_addr, old_byte)
context.Eip -= 1
win32process.SetThreadContext(process_info[1], context)
win32process.ContinueDebugEvent(debug_event.dwProcessId, debug_event.dwThreadId, win32con.DBG_CONTINUE)
analyze_malware("sample.exe")
Output Example:
=== Imported APIs ===
kernel32.dll: CreateFileA
kernel32.dll: WriteFile
wininet.dll: InternetOpenA
CreateFileA called
Filename: C:\malware.txt
HTTP Packet: GET /data HTTP/1.1
Using x64dbg and Wireshark for Interactive Analysis
- Debugging:
- Open x64dbg, load
sample.exe. - Set a breakpoint at
CreateFileA(F2). - Run the program, check parameters and return values.
- Open x64dbg, load
- Network Monitoring:
- Start Wireshark, filter
httptraffic. - Observe the program’s network requests.
- Start Wireshark, filter
- File Monitoring:
- Open Procmon, filter
sample.exe. - View file creation and write operations.
- Open Procmon, filter
Dynamic Analysis Practice
- Debugging Analysis: Set breakpoints at key APIs (e.g.,
CreateFileA,InternetOpenA), track parameters. - Behavior Monitoring: Use Procmon and Wireshark to capture file, network, and registry activity.
- Malware Analysis: Combine debugging and behavior analysis to identify C2 communication and persistence mechanisms.
- Automated Scripting: Write Frida or Python scripts to automate monitoring and data extraction.
Advanced Debugging
Advanced debugging techniques address complex protection mechanisms (e.g., anti-debugging) and low-level system analysis (e.g., kernel debugging). This section explains how to identify and bypass anti-debugging techniques and introduces the basics of kernel debugging.
Anti-Debugging Technique Identification
Anti-debugging techniques are mechanisms used by malware or protected software to detect debuggers. Common methods include:
- IsDebuggerPresent:
- Windows API, checks the
BeingDebuggedflag in the process’s PEB (Process Environment Block). - Code Example:
call IsDebuggerPresent test eax, eax jnz debugger_detected
- Windows API, checks the
- Timing Checks:
- Measure code execution time to detect delays caused by debuggers (e.g., single-stepping).
- Common APIs:
GetTickCount,QueryPerformanceCounter. - Code Example:
call GetTickCount mov ebx, eax ; Some operations call GetTickCount sub eax, ebx cmp eax, 1000 ; If time difference is too large, suspect debugging ja debugger_detected
- Other Techniques:
- Check for debugger processes (e.g.,
OllyDbg.exe,x64dbg.exe). - Use
NtQueryInformationProcessto check debug status. - Detect breakpoints (e.g., scan for
0xCCor hardware breakpoint registers).
- Check for debugger processes (e.g.,
Example: Identifying IsDebuggerPresent
The following code checks for a debugger:
call IsDebuggerPresent
test eax, eax
jnz debugger_detected
Disassembly:
from capstone import *
CODE = b"\xFF\x15\x00\x20\x40\x00\x85\xC0\x75\x02"
md = Cs(CS_ARCH_X86, CS_MODE_32)
for instr in md.disasm(CODE, 0x401000):
print(f"0x{instr.address:08x}: {instr.mnemonic} {instr.op_str}")
if instr.mnemonic == "call" and "IsDebuggerPresent" in instr.op_str:
print(" -> Anti-Debug: IsDebuggerPresent")
elif instr.mnemonic == "jnz":
print(" -> Anti-Debug: Debugger Check Branch")
Output:
0x00401000: call 0x402000
-> Anti-Debug: IsDebuggerPresent
0x00401006: test eax, eax
0x00401008: jnz 0x40100c
-> Anti-Debug: Debugger Check Branch
Example: Identifying Timing Checks
call GetTickCount
mov ebx, eax
call GetTickCount
sub eax, ebx
cmp eax, 1000
ja debugger_detected
Disassembly:
CODE = b"\xFF\x15\x04\x20\x40\x00\x8B\xD8\xFF\x15\x04\x20\x40\x00\x2B\xC3\x3D\xE8\x03\x00\x00\x77\x02"
md = Cs(CS_ARCH_X86, CS_MODE_32)
for instr in md.disasm(CODE, 0x401000):
print(f"0x{instr.address:08x}: {instr.mnemonic} {instr.op_str}")
if instr.mnemonic == "call" and "GetTickCount" in instr.op_str:
print(" -> Anti-Debug: Time Check")
elif instr.mnemonic == "ja":
print(" -> Anti-Debug: Time Check Branch")
Output:
0x00401000: call 0x402004
-> Anti-Debug: Time Check
0x00401006: mov ebx, eax
0x00401008: call 0x402004
-> Anti-Debug: Time Check
0x0040100e: sub eax, ebx
0x00401010: cmp eax, 0x3e8
0x00401015: ja 0x401019
-> Anti-Debug: Time Check Branch
Bypassing Anti-Debugging
Bypassing anti-debugging techniques requires modifying program behavior, typically through:
- Modifying Registers:
- Directly set
IsDebuggerPresentreturn value (e.g.,eax = 0). - Modify PEB’s
BeingDebuggedflag to 0.
- Directly set
- Patching Code:
- Replace conditional jumps (e.g.,
jnz) withnopor unconditional jumps (jmp). - Modify timing check thresholds.
- Replace conditional jumps (e.g.,
- Dynamic Instrumentation:
- Use Frida or debugger APIs to intercept anti-debugging functions.
Example: Bypassing IsDebuggerPresent
Method 1: Patch Code
Replace jnz debugger_detected with nop:
import win32api
def patch_binary(file_path, offset, new_bytes):
with open(file_path, "r+b") as f:
f.seek(offset)
f.write(new_bytes)
# Replace jnz (0x75 0x02) with nop (0x90 0x90)
patch_binary("sample.exe", 0x401008, b"\x90\x90")
Verification (Radare2):
r2 sample.exe
> s 0x401008
> pd 1
0x00401008 nop
Method 2: Frida Dynamic Instrumentation
// bypass_isdebuggerpresent.js
const IsDebuggerPresent = Module.findExportByName("kernel32.dll", "IsDebuggerPresent");
Interceptor.attach(IsDebuggerPresent, {
onEnter(args) {
console.log("IsDebuggerPresent called");
},
onLeave(retval) {
console.log("Forcing IsDebuggerPresent to return 0");
retval.replace(0);
}
});
Run:
frida -f sample.exe -l bypass_isdebuggerpresent.js
Output:
IsDebuggerPresent called
Forcing IsDebuggerPresent to return 0
Example: Bypassing Timing Checks
Method 1: Modify Threshold
Change cmp eax, 1000 immediate to 0xFFFF:
# Replace cmp eax, 0x3e8 with cmp eax, 0xffff
patch_binary("sample.exe", 0x401011, b"\x3D\xFF\xFF\x00\x00")
Method 2: Frida Intercept GetTickCount
// bypass_gettickcount.js
const GetTickCount = Module.findExportByName("kernel32.dll", "GetTickCount");
let lastTime = 0;
Interceptor.attach(GetTickCount, {
onEnter(args) {
console.log("GetTickCount called");
},
onLeave(retval) {
if (lastTime == 0) {
lastTime = retval.toInt32();
} else {
retval.replace(lastTime + 10); // Simulate small time difference
lastTime = 0;
}
}
});
Run:
frida -f sample.exe -l bypass_gettickcount.js
Kernel Debugging
Kernel debugging is used to analyze the operating system kernel, drivers, or low-level behavior. Common tools include WinDbg and SoftICE (now obsolete, WinDbg is mainstream).
- WinDbg:
- Supports user-mode and kernel-mode debugging.
- Commonly used for driver analysis, blue screen debugging, and rootkit detection.
- Configuration: Requires target and host systems connected via serial, USB, or network.
- SoftICE:
- Early kernel debugger, runs on the target system.
- Largely replaced by WinDbg.
Example: WinDbg Kernel Debugging Setup
- Configure Target System (Windows):
- Enable debug mode:
bcdedit /debug on bcdedit /dbgsettings serial baudrate:115200 - Restart the system.
- Enable debug mode:
- Start WinDbg on Host System:
- Install WinDbg (from Microsoft SDK).
- Launch:
windbg -k com:port=COM1,baud=115200
- Basic Commands:
- Set breakpoint:
bp nt!NtCreateFile(at kernel functionNtCreateFile). - View registers:
r. - Single-step:
t(step into),p(step over). - View call stack:
k.
- Set breakpoint:
Example: WinDbg Driver Debugging
Assume analyzing a malicious driver:
windbg -k com:port=COM1,baud=115200
In WinDbg:
!process 0 0 ; List processes
bp nt!NtCreateFile ; Set breakpoint at file creation
g ; Continue execution
When breakpoint triggers:
r ; View registers
k ; View call stack
dv ; View local variables
Code Example: Python Calling WinDbg
Use pykd (WinDbg Python extension) for automated debugging:
import pykd
def kernel_debug():
pykd.attachKernel() # Connect to kernel
print("Attached to kernel")
# Set breakpoint
bp_id = pykd.setBp("nt!NtCreateFile")
print(f"Breakpoint set at nt!NtCreateFile: {bp_id}")
# Run
pykd.go()
# Breakpoint triggered
print("Breakpoint hit")
regs = pykd.reg()
print(f"RAX: 0x{regs['rax']:016x}, RIP: 0x{regs['rip']:016x}")
# Clean up
pykd.removeBp(bp_id)
kernel_debug()
Output Example:
Attached to kernel
Breakpoint set at nt!NtCreateFile: 0
Breakpoint hit
RAX: 0x0000000000000000, RIP: 0xFFFFF80012345678
Advanced Debugging Notes
- Complex Anti-Debugging: Malware may combine multiple anti-debugging techniques; bypass from multiple angles.
- Kernel Debugging Risks: Incorrect operations may crash the system; test in a virtual machine.
- Tool Selection: WinDbg for kernel debugging, x64dbg for user-mode debugging, Frida for dynamic instrumentation.
- Legal Compliance: Bypassing anti-debugging may involve legal risks; operate within authorized scope.
Dynamic Injection and Modification
Dynamic injection and modification techniques modify program behavior at runtime, widely used in reverse engineering (bypassing protection), malware (loading malicious code), and security testing (feature enhancement). This section explains DLL injection, code injection and hooking, and runtime memory patching.
DLL Injection Techniques
DLL injection loads a custom dynamic link library into a target process’s address space, typically to modify behavior or monitor operations. Common methods include:
- CreateRemoteThread + LoadLibrary:
- Allocate memory in the target process, write the DLL path, create a remote thread to call
LoadLibrary. - Advantages: Simple and reliable.
- Disadvantages: Easily detected.
- Allocate memory in the target process, write the DLL path, create a remote thread to call
- SetWindowsHook:
- Use Windows hooks (e.g., keyboard, mouse hooks) to load the DLL.
- Advantages: Suitable for GUI programs.
- Disadvantages: Depends on specific events.
Example: CreateRemoteThread DLL Injection
The following code injects a DLL into a target process:
import win32api
import win32con
import win32process
import ctypes
def inject_dll(pid, dll_path):
# Open target process
h_process = win32api.OpenProcess(win32con.PROCESS_ALL_ACCESS, False, pid)
# Allocate memory
dll_path_bytes = dll_path.encode("ascii")
alloc_addr = ctypes.windll.kernel32.VirtualAllocEx(h_process, 0, len(dll_path_bytes),
win32con.MEM_COMMIT | win32con.MEM_RESERVE,
win32con.PAGE_READWRITE)
# Write DLL path
win32api.WriteProcessMemory(h_process, alloc_addr, dll_path_bytes)
# Get LoadLibraryA address
kernel32 = ctypes.WinDLL("kernel32.dll")
load_library = ctypes.cast(getattr(kernel32, "LoadLibraryA"), ctypes.c_void_p).value
# Create remote thread
h_thread = ctypes.windll.kernel32.CreateRemoteThread(h_process, None, 0, load_library, alloc_addr, 0, None)
# Wait for thread to finish
win32api.WaitForSingleObject(h_thread, -1)
# Clean up
ctypes.windll.kernel32.VirtualFreeEx(h_process, alloc_addr, 0, win32con.MEM_RELEASE)
win32api.CloseHandle(h_thread)
win32api.CloseHandle(h_process)
# Example: Inject into notepad.exe
inject_dll(1234, "C:\\mydll.dll")
DLL Code (C language):
#include <windows.h>
BOOL APIENTRY DllMain(HMODULE hModule, DWORD ul_reason_for_call, LPVOID lpReserved) {
if (ul_reason_for_call == DLL_PROCESS_ATTACH) {
MessageBoxA(NULL, "DLL Injected!", "Success", MB_OK);
}
return TRUE;
}
Compile:
cl /LD mydll.c
Example: SetWindowsHook DLL Injection
import win32api
import win32con
import ctypes
def set_hook_dll(dll_path, pid):
# Load DLL
h_module = ctypes.windll.kernel32.LoadLibraryA(dll_path.encode("ascii"))
# Get hook function address (assume DLL exports HookProc)
hook_proc = ctypes.cast(ctypes.windll.kernel32.GetProcAddress(h_module, b"HookProc"), ctypes.c_void_p).value
# Set keyboard hook
h_hook = ctypes.windll.user32.SetWindowsHookExA(win32con.WH_KEYBOARD_LL, hook_proc, h_module, 0)
# Wait for events
ctypes.windll.user32.GetMessageA(0, 0, 0, 0)
# Clean up
ctypes.windll.user32.UnhookWindowsHookEx(h_hook)
ctypes.windll.kernel32.FreeLibrary(h_module)
set_hook_dll("C:\\mydll.dll", 1234)
DLL Code:
#include <windows.h>
LRESULT CALLBACK HookProc(int nCode, WPARAM wParam, LPARAM lParam) {
if (nCode >= 0) {
MessageBoxA(NULL, "Hook Triggered!", "Success", MB_OK);
}
return CallNextHookEx(NULL, nCode, wParam, lParam);
}
BOOL APIENTRY DllMain(HMODULE hModule, DWORD ul_reason_for_call, LPVOID lpReserved) {
return TRUE;
}
Code Injection and Hooking
Code injection directly injects machine code into a target process; hooking modifies function calls to intercept behavior. Common techniques include:
- Inline Hook:
- Modify the target function’s beginning, insert
jmpto custom code. - Advantages: Precise control.
- Disadvantages: Complex, requires preserving original instructions.
- Modify the target function’s beginning, insert
- IAT Hook:
- Modify the Import Address Table (IAT), replace API addresses with custom functions.
- Advantages: Simple, suitable for API interception.
- Disadvantages: Limited to imported functions.
Example: Inline Hook
The following code inserts a hook at the beginning of MessageBoxA:
import win32api
import win32process
import ctypes
def inline_hook(pid, target_func, hook_code):
h_process = win32api.OpenProcess(win32con.PROCESS_ALL_ACCESS, False, pid)
# Get target function address
user32 = ctypes.WinDLL("user32.dll")
target_addr = ctypes.cast(getattr(user32, target_func), ctypes.c_void_p).value
# Allocate memory to store hook code
hook_addr = ctypes.windll.kernel32.VirtualAllocEx(h_process, 0, len(hook_code),
win32con.MEM_COMMIT | win32con.MEM_RESERVE,
win32con.PAGE_EXECUTE_READWRITE)
# Write hook code
win32api.WriteProcessMemory(h_process, hook_addr, hook_code)
# Save original instructions
original_bytes = win32api.ReadProcessMemory(h_process, target_addr, 5)
# Write jmp to hook
jmp_code = b"\xE9" + struct.pack("<I", hook_addr - target_addr - 5)
win32api.WriteProcessMemory(h_process, target_addr, jmp_code)
return original_bytes, target_addr, hook_addr
# Hook code: Display custom message
hook_code = (
b"\x6A\x00" # push 0
b"\x68" + struct.pack("<I", 0x1000) # push "Hooked!"
b"\x68" + struct.pack("<I", 0x1004) # push "Success"
b"\x6A\x00" # push 0
b"\xFF\x15\x00\x00\x00\x00" # call MessageBoxA (needs filling)
b"\xC3" # ret
)
original_bytes, target_addr, hook_addr = inline_hook(1234, "MessageBoxA", hook_code)
Example: IAT Hook
Modify the IAT to replace MessageBoxA with a custom function:
import pefile
import win32api
import win32process
def iat_hook(pid, dll_path):
pe = pefile.PE(dll_path)
h_process = win32api.OpenProcess(win32con.PROCESS_ALL_ACCESS, False, pid)
# Locate IAT
for entry in pe.DIRECTORY_ENTRY_IMPORT:
if entry.dll.lower() == b"user32.dll":
for imp in entry.imports:
if imp.name == b"MessageBoxA":
# Replace with custom function address
custom_addr = 0x10001000 # Assume custom function address
win32api.WriteProcessMemory(h_process, imp.address, struct.pack("<I", custom_addr))
win32api.CloseHandle(h_process)
iat_hook(1234, "sample.exe")
Runtime Memory Patching
Runtime memory patching modifies instructions or data in a process’s memory, bypassing protection or altering behavior.
Example: Memory Patch to Bypass Check
Assume the program has a check logic:
cmp eax, 0x12345678
jne error
Patch to unconditional jump:
def memory_patch(pid, address, new_bytes):
h_process = win32api.OpenProcess(win32con.PROCESS_ALL_ACCESS, False, pid)
win32api.WriteProcessMemory(h_process, address, new_bytes)
win32api.CloseHandle(h_process)
# Replace jne (0x75 0x02) with jmp (0xEB 0x02)
memory_patch(1234, 0x401008, b"\xEB\x02")
Example: Frida Memory Patch
// patch_memory.js
const addr = ptr("0x401008");
const new_bytes = [0xEB, 0x02]; // jmp 0x2
Memory.write(addr, new_bytes);
console.log("Patched memory at 0x401008");
Run:
frida -f sample.exe -l patch_memory.js
Dynamic Injection and Modification Notes
- Permission Issues: Injection and hooking require write permissions to the target process, may trigger antivirus.
- Stability: Inline hooks must preserve original instructions, or crashes may occur.
- Detection Risk: Malware may detect injection; hide traces (e.g., use APC injection).
- Tool Support: Frida is suitable for cross-platform injection, x64dbg for manual hooking.
Comprehensive Example: Debugging and Injection Analysis
The following code combines anti-debugging bypass, DLL injection, and memory patching to analyze a protected PE file:
import win32api
import win32con
import win32process
import pefile
import ctypes
def analyze_protected_binary(pid, dll_path):
# Open process
h_process = win32api.OpenProcess(win32con.PROCESS_ALL_ACCESS, False, pid)
# Bypass IsDebuggerPresent
peb_addr = ctypes.c_ulong()
ctypes.windll.kernel32.GetProcessInformation(h_process, 0x1A, ctypes.byref(peb_addr), ctypes.sizeof(peb_addr))
win32api.WriteProcessMemory(h_process, peb_addr + 2, b"\x00") # Clear BeingDebugged
# DLL injection
dll_path_bytes = dll_path.encode("ascii")
alloc_addr = ctypes.windll.kernel32.VirtualAllocEx(h_process, 0, len(dll_path_bytes),
win32con.MEM_COMMIT | win32con.MEM_RESERVE,
win32con.PAGE_READWRITE)
win32api.WriteProcessMemory(h_process, alloc_addr, dll_path_bytes)
load_library = ctypes.cast(getattr(ctypes.WinDLL("kernel32.dll"), "LoadLibraryA"), ctypes.c_void_p).value
h_thread = ctypes.windll.kernel32.CreateRemoteThread(h_process, None, 0, load_library, alloc_addr, 0, None)
win32api.WaitForSingleObject(h_thread, -1)
# Memory patch (bypass check)
win32api.WriteProcessMemory(h_process, 0x401008, b"\xEB\x02") # jne -> jmp
# Clean up
ctypes.windll.kernel32.VirtualFreeEx(h_process, alloc_addr, 0, win32con.MEM_RELEASE)
win32api.CloseHandle(h_thread)
win32api.CloseHandle(h_process)
analyze_protected_binary(1234, "C:\\mydll.dll")
Using x64dbg and Frida for Interactive Analysis
- Debugging:
- Open x64dbg, load
sample.exe. - Set a breakpoint at
IsDebuggerPresent, modify return value (eax = 0). - Patch check logic (change
jnetojmp).
- Open x64dbg, load
- Injection:
- Use Frida to inject DLL or hook key APIs (e.g.,
MessageBoxA). - Verify injection effect.
- Use Frida to inject DLL or hook key APIs (e.g.,
- Kernel Debugging:
- Use WinDbg to connect to a virtual machine, set breakpoint at
nt!NtCreateFile. - Analyze driver behavior.
- Use WinDbg to connect to a virtual machine, set breakpoint at
Dynamic Analysis Practice
- Anti-Debugging Bypass: Combine patching and Frida to disable
IsDebuggerPresentand timing checks. - DLL Injection: Implement
CreateRemoteThreadinjection to load a custom DLL. - Hook Analysis: Use inline hooks to intercept APIs and capture parameters.
- Memory Patching: Dynamically patch check logic to alter program behavior.



