Lesson 04-Reverse Engineering Environment

Virtual Machines and Sandboxes

Virtual machines and sandboxes provide isolated environments for reverse engineering, preventing malware from infecting the host. The following are the configuration and usage of three mainstream tools.

VMware Workstation

Overview:

  • Type: Virtual Machine
  • Platform: Windows, Linux
  • License: Commercial (VMware Workstation Pro, about $250), Free (VMware Player)
  • Features: Runs isolated operating systems, supports snapshots, cloning, and network isolation.

Configuration Steps:

  1. Installation:
    • Download VMware Workstation Pro or Player from vmware.com.
    • Install and activate (Player is free).
  2. Create Virtual Machine:
    • Select Windows 10/11 or Ubuntu 22.04 ISO file.
    • Configure hardware: 4GB RAM, 2-core CPU, 50GB disk.
    • Enable virtualization support (VT-x/AMD-V):
      • Enable Intel VT-x or AMD-V in BIOS settings.
  3. Network Configuration:
    • Use “Host-Only” or “Custom” network to isolate the virtual machine.
    • Disable shared folders and clipboard.
  4. Install Reverse Engineering Tools:
    • Install tools (such as Ghidra, x64dbg) in the virtual machine following Windows/Linux environment configuration.
  5. Snapshot Management:
    • Create an initial snapshot, restore after analyzing malware:
# VMware GUI: Snapshot -> Take Snapshot

Usage Example:

  • Run malware malware.exe:
    • Open x64dbg in the VMware virtual machine, load malware.exe.
    • Set breakpoints, monitor behavior (such as network requests).
    • Restore snapshot after analysis.

Advantages:

  • Powerful virtualization features, supports multiple platforms.
  • Snapshot and cloning functions facilitate experiments.
  • Enterprise-level support, high stability.

Limitations:

  • Pro version is expensive, Player has limited features.
  • High resource consumption.

QEMU

Overview:

  • Type: Virtual Machine/Emulator
  • Platform: Windows, Linux, macOS
  • License: Open source (free)
  • Features: Lightweight virtualization, supports x86, ARM, MIPS and other architectures, suitable for embedded system analysis.

Configuration Steps:

  1. Installation:
  • Ubuntu:
sudo apt install qemu-system-x86 qemu-system-arm -y
  • Windows: Download from qemu.org, extract and add to PATH.
  1. Create Virtual Machine:
  • Download Ubuntu ISO, create disk image:
qemu-img create -f qcow2 ubuntu.qcow2 20G
  • Start virtual machine to install system:
qemu-system-x86_64 -m 4G -cdrom ubuntu.iso -hda ubuntu.qcow2 -enable-kvm
  1. Run Virtual Machine:
qemu-system-x86_64 -m 4G -hda ubuntu.qcow2 -enable-kvm -net nic -net user
  1. Embedded System Analysis:
  • Emulate ARM firmware:
qemu-system-arm -M versatilepb -kernel firmware.bin -nographic

Usage Example:

  • Analyze ARM firmware:
  • Use QEMU to emulate and run firmware.
  • Combine with GDB for debugging:
qemu-system-arm -M versatilepb -kernel firmware.bin -s -S
gdb --eval-command="target remote localhost:1234"

Advantages:

  • Free and lightweight, supports multiple architectures.
  • Suitable for embedded and cross-platform analysis.
  • Good integration with GDB.

Limitations:

  • Complex configuration, requires manual setup.
  • Limited GUI support, relies on command line.
  • Lower performance than VMware (without KVM).

Cuckoo Sandbox

Overview:

  • Type: Automated Sandbox
  • Platform: Linux (Ubuntu recommended)
  • License: Open source (free)
  • Features: Automated malware analysis, monitors file, network, and memory activities, generates detailed reports.

Configuration Steps:

Install Dependencies:

  • Ubuntu:
sudo apt install python3 python3-pip python3-dev libffi-dev libssl-dev -y
sudo apt install virtualbox tcpdump -y

Install Cuckoo:

pip3 install cuckoo
cuckoo --version

Configure Virtual Machine:

  • Create a Windows 10 virtual machine in VirtualBox, install Python and reverse engineering tools.
  • Enable snapshots, configure network as “Host-Only”.
  • Configure Cuckoo:
cuckoo init
cuckoo community  # Download signatures
  • Edit configuration files ~/.cuckoo/conf/cuckoo.conf and virtualbox.conf:
  • Set VirtualBox virtual machine name.
  • Enable network capture.

Start Cuckoo:

cuckoo
  • Start Web interface:
cuckoo web --host 0.0.0.0 --port 8000

Usage Example:

  • Analyze malware:
cuckoo submit malware.exe
  • Visit http://localhost:8000 to view report (file operations, network traffic, etc.).

Advantages:

  • Automated analysis, generates detailed reports.
  • Supports multiple virtualization backends (VirtualBox, VMware, QEMU).
  • Rich community signature library.

Limitations:

  • Complex configuration, many dependencies.
  • Performance depends on host resources.
  • May be detected by advanced malware.

Multi-Platform Debugging Toolchain

The multi-platform debugging toolchain supports analysis of binary files across different architectures (such as x86, x64, ARM) and operating systems (Windows, Linux, Android). The following are configuration steps and recommended tools.

Toolchain Components

  1. Debugger:
    • GDB: Cross-platform, supports multiple architectures (x86, ARM, etc.).
    • LLDB: LLVM debugger, suitable for macOS and Linux.
    • WinDbg/x64dbg: Windows-specific.
  2. Emulator:
    • Unicorn: Lightweight CPU emulation, supports Python bindings.
    • QEMU: Full-system emulation.
  3. Disassembly Engine:
    • Capstone: Multi-architecture disassembly.
    • Keystone: Assembly engine.
  4. Dynamic Instrumentation:
    • Frida: Cross-platform dynamic injection.
    • DynamoRIO: Dynamic binary instrumentation.

Configuration Steps

Install GDB with Enhanced Plugins:

  • Ubuntu:
sudo apt install gdb-multiarch -y
git clone https://github.com/pwndbg/pwndbg
cd pwndbg
./setup.sh
  • Windows: Install via MinGW or WSL.

Install LLDB:

  • Ubuntu:
sudo apt install lldb -y
lldb --version
  • macOS: LLDB comes with Xcode. Install Unicorn:
pip3 install unicorn
  • Example (emulate x86 code):
from unicorn import *
from unicorn.x86_const import *

CODE = b"\xb8\x34\x12\x00\x00"  # mov eax, 0x1234
mu = Uc(UC_ARCH_X86, UC_MODE_32)
mu.mem_map(0x1000, 2 * 1024 * 1024)
mu.mem_write(0x1000, CODE)
mu.emu_start(0x1000, 0x1000 + len(CODE))
print(f"EAX: 0x{mu.reg_read(UC_X86_REG_EAX):x}")

Install Capstone and Keystone:

sudo apt install libcapstone-dev -y
pip3 install capstone keystone-engine
  • Example (disassembly):
from capstone import *
CODE = b"\xb8\x34\x12\x00\x00"
md = Cs(CS_ARCH_X86, CS_MODE_32)
for instr in md.disasm(CODE, 0x1000):
	print(f"0x{instr.address:x}: {instr.mnemonic} {instr.op_str}")

Configure Frida:

Install server (for Android/ARM debugging):

  • Download Frida server from frida.re.
  • Push to Android device:
adb push frida-server /data/local/tmp/
adb shell "chmod 755 /data/local/tmp/frida-server"
adb shell "/data/local/tmp/frida-server &"
  • Use Python client:
import frida
device = frida.get_usb_device()
pid = device.spawn(["com.example.app"])
session = device.attach(pid)
print("Attached to process")

Cross-Compilation Toolchain:

  • Install ARM toolchain:
sudo apt install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf -y
  • Compile ARM program:
arm-linux-gnueabihf-gcc -o sample_arm sample.c

Example: Debug ARM Binary

  1. Compile ARM Program:
#include <stdio.h>
int main() {
    printf("Hello, ARM!\n");
    return 0;
}
arm-linux-gnueabihf-gcc -g -o sample_arm sample.c
  1. Run with QEMU:
qemu-arm -g 1234 sample_arm
  1. Debug with GDB:
gdb-multiarch sample_arm
(gdb) target remote localhost:1234
(gdb) break main
(gdb) continue
  1. Inject with Frida:
import frida
def on_message(message, data):
    print(message)
script = frida.get_usb_device().attach("sample_arm")
script.load()

IDA Pro: Interface, Disassembly, Scripting

Interface Navigation

Overview: IDA Pro is a static analysis tool developed by Hex-Rays, supporting multiple architectures (x86, x64, ARM, etc.) and file formats (PE, ELF, Mach-O). Its interface is complex but powerful, suitable for professional reverse engineering.

Main Interface Components:

  1. Main Window (View-A): Displays disassembly code, supports text and graph views (Ctrl+Enter to switch).
  2. Functions Window: Lists all identified functions, double-click to jump.
  3. Strings Window (Shift+F12): Displays extracted strings, right-click to view cross-references.
  4. Imports/Exports Window: Displays dynamic link libraries (DLLs) and functions.
  5. Output Window: Displays analysis logs and script output.
  6. Navigation Bar: Color bar indicates code, data, and undefined areas.

Basic Operations:

  • Open file: File -> Open, select binary file (e.g., sample.exe).
  • Auto analysis: IDA runs auto analysis by default, identifying functions, strings, etc.
  • Manually define code: Press C to convert undefined area to code, U to undo.
  • Rename: Right-click variable/function, select Rename or press N.
  • Add comment: Press ; to add line comment, save analysis results.

Example:

  1. Open sample.exe, wait for auto analysis to complete.
  2. Press F5 (requires Hex-Rays plugin) to view pseudocode.
  3. Double-click main function in main window, view disassembly:
push    ebp
mov     ebp, esp
call    sub_401000

Disassembly Analysis

Features: IDA Pro’s disassembly engine converts machine code to assembly code, supports interactive analysis such as cross-references, function call graphs, and data flow tracking.

Key Operations:

  1. Cross References (Xrefs):
    • Right-click function or variable, select Jump to xref (shortcut X) to view call sites.
    • Example: Analyze printf calls, see which functions call it.
  2. Function Call Graph:
    • View -> Graphs -> Function calls displays call relationships.
  3. String Analysis:
    • Open strings window (Shift+F12), right-click Jump to to locate code references.
  4. Manual Fix:
  • If auto analysis is wrong (e.g., unrecognized function), press C to define code, D to define data.

Example: Analyze main function:

  1. Locate main (double-click in functions window).
  2. View disassembly:
call    printf
mov     eax, 0
retn
  1. Press X to view printf cross-references, find call sites.
  2. Press F5 to view pseudocode (if Hex-Rays is available):
int main() {
	printf("Hello, World!\n");
	return 0;
}

IDAPython Scripting

Overview: IDAPython is IDA Pro’s Python plugin, enabling automation of analysis tasks such as extracting functions, renaming, or generating reports.

Environment Configuration:

  • Ensure IDA Pro has Python support installed (included by default).
  • Verify: File -> Script command (Alt+F7), input print("Hello, IDA!").

Common APIs:

  • idaapi: Core IDA functions, such as getting function addresses.
  • idautils: Utility tools, such as iterating functions and strings.
  • idc: Interact with IDA interface, such as setting breakpoints and renaming.

Example Script: Extract all function names and addresses:

import idaapi
import idautils

for func_ea in idautils.Functions():
    func_name = idaapi.get_func_name(func_ea)
    print(f"Function: {func_name} at 0x{func_ea:x}")

Save as functions.py, run:

  1. File -> Script file (Ctrl+F7), select functions.py.
  2. Output example:
Function: main at 0x401000
Function: sub_401100 at 0x401100

Advanced Example: Automatically rename functions starting with sub_:

import idaapi
import idautils

prefix = "custom_"
for func_ea in idautils.Functions():
    name = idaapi.get_func_name(func_ea)
    if name.startswith("sub_"):
        new_name = prefix + name[4:]
        idaapi.set_name(func_ea, new_name)
        print(f"Renamed {name} to {new_name}")

Best Practices:

  • Save scripts to IDA’s scripts directory for reuse.
  • Use idaapi.require() to load modules, avoid conflicts.
  • Add error handling:
try:
    func_name = idaapi.get_func_name(func_ea)
except Exception as e:
    print(f"Error at 0x{func_ea:x}: {e}")

Ghidra: Project Management, Code Analysis

Project Management

Overview: Ghidra is an open-source reverse engineering framework developed by the NSA, supporting multiple architectures and file formats, providing disassembly, decompilation, and scripting capabilities. Its project management features facilitate organizing multiple binary files and collaboration.

Create Project:

  1. Launch Ghidra: ghidraRun (Linux) or ghidraRun.bat (Windows).
  2. Create new project:
    • File -> New Project, select Non-Shared Project.
    • Specify project directory (e.g., ~/ghidra_projects).
  3. Import binary file:
    • File -> Import File, select sample.exe.
    • Choose format (PE, ELF, etc.), Ghidra auto-detects.
  4. Open file:
    • Double-click sample.exe in the project, launch CodeBrowser.

Project Management Operations:

  • Version Control: File -> Version Control tracks analysis changes.
  • Batch Import: File -> Batch Import adds multiple files.
  • Export Analysis: File -> Export saves as C code, HTML, or XML.

Example:

  1. Create project ReverseEng, import sample.exe.
  2. Save project: File -> Save All.

Code Analysis

Features: Ghidra provides disassembly, decompilation, and data flow analysis, with a built-in decompiler generating C-like pseudocode.

Main Windows:

  1. Listing View: Displays disassembly code, right-click for operations (e.g., rename, define function).
  2. Decompiler View: Displays pseudocode, synchronized with Listing.
  3. Symbol Tree: Lists functions, strings, and imports.
  4. Function Graph: Displays function control flow graph.

Key Operations:

  1. Auto Analysis:
    • After import, select Analyze, check all analysis options (e.g., function identification, string extraction).
  2. Decompilation:
    • Double-click function in Listing View, Decompiler View shows pseudocode.
  3. Cross References:
    • Right-click variable/function, select References -> Show References to.
  4. Manual Fix:
    • Press C to define code, D to define data.
    • Right-click Rename to modify name.
  5. Search:
    • Search -> Memory to find strings or byte sequences.

Example: Analyze main function:

  1. Open sample.exe, run auto analysis.
  2. Find main in Symbol Tree, double-click to jump.
  3. View disassembly:
PUSH    EBP
MOV     EBP,ESP
CALL    printf
  1. View pseudocode:
undefined4 main(void) {
	printf("Hello, World!");
	return 0;
}
  1. Right-click printf, select Show References to, view call sites.

Ghidra Scripting

Overview: Ghidra supports Python and Java scripts, with Python scripts (GhidraScript) being more common for automating analysis tasks.

Environment Configuration:

  • Install Python support:
pip3 install ghidra-bridge
  • Verify: Script Manager -> Run Script, input print("Hello, Ghidra!").

Common APIs:

  • currentProgram: Access current program.
  • getFunctionManager(): Get function list.
  • findBytes(): Search byte sequences.

Example Script: Extract all functions:

from ghidra.program.model.listing import FunctionManager

fm = currentProgram.getFunctionManager()
for func in fm.getFunctions(True):
    print(f"Function: {func.getName()} at 0x{func.getEntryPoint()}")

Run:

  1. Open Script Manager (Window -> Script Manager).
  2. Create new script (New Script), paste code, save as list_functions.py.
  3. Run script, output:
Function: main at 0x00401000
Function: sub_401100 at 0x00401100

Advanced Example: Search strings and mark them:

from ghidra.program.model.listing import Listing

listing = currentProgram.getListing()
for data in listing.getDefinedData(True):
    if data.getDataType().getName() == "string":
        value = data.getValue()
        addr = data.getAddress()
        print(f"String: {value} at 0x{addr}")
        currentProgram.getBookmarkManager().setBookmark(addr, "Note", "String", f"Found: {value}")

Best Practices:

  • Save scripts to Ghidra’s ghidra_scripts directory.
  • Use try-except for error handling:
try:
    fm = currentProgram.getFunctionManager()
except Exception as e:
    print(f"Error: {e}")

x64dbg: Breakpoints, Register Tracking

Breakpoint Setting

Overview: x64dbg is an open-source Windows debugger supporting 32-bit and 64-bit programs, with a user-friendly interface suitable for dynamic analysis and vulnerability research.

Interface Navigation:

  1. CPU Window: Displays disassembly code, registers, and stack.
  2. Registers Window: Displays current register values.
  3. Memory Map: Displays memory layout.
  4. Breakpoints Window: Manages breakpoints.
  5. Log Window: Displays debug logs.

Set Breakpoints:

  1. Software Breakpoint:
    • Open sample.exe: File -> Open.
    • Locate main in CPU window (double-click to jump).
    • Press F2 to set breakpoint (or right-click Toggle Breakpoint).
  2. Hardware Breakpoint:
    • Right-click register (e.g., EAX), select Set Hardware Breakpoint on Access/Write.
  3. Conditional Breakpoint:
    • Right-click breakpoint, select Edit Breakpoint, input condition (e.g., EAX == 0x1234).
  4. Run:
    • Press F9 to run to breakpoint.
    • Press F8 to step over, F7 to step into.

Example:

  1. Load sample.exe, locate main:
push    ebp
mov     ebp, esp
call    printf
  1. Set breakpoint at call printf with F2.
  2. Press F9 to run, program pauses.
  3. View stack parameters (right-click stack window, check Hello, World!).

Register Tracking

Features: x64dbg allows real-time monitoring and modification of register values, suitable for analyzing program state and data flow.

Operations:

  1. View Registers:
    • Registers window displays EAX, EBX, RIP, etc.
    • Right-click register, select Follow in Dump to view memory content.
  2. Modify Registers:
    • Double-click register value, input new value (e.g., EAX = 0x100).
  3. Track Data:
    • Right-click register, select Watch to add to Watch window.
    • Use Trace function to record register changes (Trace -> Start Trace).

Example:

  1. Set breakpoint at main, run to breakpoint.
  2. Check EAX (assume before calling printf).
  3. Modify EAX:
    • Double-click EAX, input 0x1234.
  4. Step execution (F8), observe program behavior.

Advanced Features:

  • Script Support:
  • Use x64dbg script (File -> Script) for automated debugging.
  • Example:
bp 0x00401000
run
msg "Breakpoint hit!"
  • Plugins:
  • Install ScyllaHide (anti-anti-debugging plugin):
# Copy ScyllaHide.dll to x64dbg/plugins directory

Best Practices:

  • Save debug session: File -> Save Database.
  • Use conditional breakpoints to reduce debugging time.
  • Regularly check Log window to capture exceptions.

Python Script Integration

IDAPython

Features: IDAPython integrates Python into IDA Pro, suitable for batch analysis, data extraction, and custom functions.

Common Modules:

  • idaapi: Access IDA core functions.
  • idautils: Iterate functions, strings, etc.
  • idc: Control IDA interface.

Example: Extract strings and save to file:

import idautils
import idaapi

output_file = "strings.txt"
with open(output_file, "w") as f:
    for s in idautils.Strings():
        f.write(f"0x{s.ea:x}: {s.str}\n")
        print(f"String: {s.str} at 0x{s.ea:x}")

Run:

  1. File -> Script file, select script.
  2. Output to strings.txt:
0x401000: Hello, World!
0x401010: Error

Advanced Example: Automatically mark function calls:

import idautils
import idaapi

for func_ea in idautils.Functions():
    for xref in idautils.XrefsTo(func_ea):
        idaapi.add_code_xref(xref.frm, func_ea, idaapi.fl_CN)
        print(f"Marked call from 0x{xref.frm:x} to 0x{func_ea:x}")

Ghidra Scripting

Features: Ghidra’s Python scripts (GhidraScript) are used for automated analysis, such as batch renaming, data extraction, and report generation.

Common Modules:

  • ghidra.program.model: Access program structure.
  • currentProgram: Current analysis program.
  • getFunctionManager(): Function management.

Example: Extract imported functions:

from ghidra.program.model.symbol import SymbolType

symbols = currentProgram.getSymbolTable().getSymbols()
for symbol in symbols:
    if symbol.getSymbolType() == SymbolType.FUNCTION and symbol.isExternal():
        print(f"Imported Function: {symbol.getName()} at 0x{symbol.getAddress()}")

Run:

  1. Open Script Manager, create imports.py.
  2. Run script, output:
Imported Function: printf at 0x00402000
Imported Function: scanf at 0x00402010

Advanced Example: Automatically comment string references:

from ghidra.program.model.listing import Listing

listing = currentProgram.getListing()
for data in listing.getDefinedData(True):
    if data.getDataType().getName() == "string":
        addr = data.getAddress()
        value = data.getValue()
        listing.setComment(addr, listing.EOL_COMMENT, f"String: {value}")
        print(f"Commented: {value} at 0x{addr}")

Best Practices:

  • Use Ghidra Bridge to run external Python:
pip3 install ghidra-bridge
python3 -m ghidra_bridge sample.py
  • Save scripts to ghidra_scripts directory.
  • Add logging:
import logging
logging.basicConfig(level=logging.INFO)
logging.info("Script started")

Comprehensive Example: Analyze PE File

Objective: Use IDA Pro, Ghidra, and x64dbg to analyze sample.exe, extract main function logic, set breakpoints, and automate string extraction.

Steps

IDA Pro Analysis:

  • Open sample.exe, locate main.
  • View pseudocode (F5):
int main() {
	printf("Hello, World!");
	return 0;
}
  • Run IDAPython script to extract strings:
import idautils
with open("ida_strings.txt", "w") as f:
	for s in idautils.Strings():
		f.write(f"0x{s.ea:x}: {s.str}\n")

Ghidra Analysis:

  • Import sample.exe, run auto analysis.
  • View main pseudocode, confirm consistency with IDA.
  • Run Ghidra script to mark imported functions:
from ghidra.program.model.symbol import SymbolType
symbols = currentProgram.getSymbolTable().getSymbols()
for symbol in symbols:
	if symbol.getSymbolType() == SymbolType.FUNCTION and symbol.isExternal():
		addr = symbol.getAddress()
		currentProgram.getListing().setComment(addr, listing.EOL_COMMENT, "Imported Function")

x64dbg Debugging:

  • Load sample.exe, locate main.
  • Set breakpoint at call printf (F2).
  • Run (F9), check EAX and stack parameters.
  • Modify EAX:
Double-click EAX, set to 0x1234

Python Integration:

  • Use IDAPython to save function list:
import idautils
with open("functions.txt", "w") as f:
	for func_ea in idautils.Functions():
		f.write(f"{idaapi.get_func_name(func_ea)}: 0x{func_ea:x}\n")
  • Use Ghidra script to generate report:
with open("ghidra_report.txt", "w") as f:
	fm = currentProgram.getFunctionManager()
	for func in fm.getFunctions(True):
		f.write(f"Function: {func.getName()} at 0x{func.getEntryPoint()}\n")

Output (example):

  • IDA Pro: ida_strings.txt contains "Hello, World!".
  • Ghidra: ghidra_report.txt lists functions.
  • x64dbg: Breakpoint triggers, EAX modified to 0x1234.

Analysis:

  • IDA Pro provides high-quality disassembly and pseudocode, IDAPython automates string extraction.
  • Ghidra confirms analysis results, scripts add comments.
  • x64dbg verifies dynamic behavior, modifies registers.
  • Python enhances automation, generates reports.

Share your love