This tutorial will detail the application of reverse engineering in real-world security analysis through three typical practical cases: cracking a simple registration machine, analyzing commercial software protection mechanisms, and detecting game cheats. Each case includes a complete analysis process, technical details, and code implementations to help readers master the core skills of reverse engineering.
Case One: Cracking a Simple Registration Machine
Target Analysis
We will analyze a simple registration machine program that requires the user to input a serial number for validation. Our goal is to locate the validation logic and generate a valid registration code.
Program Behavior Observation:
- User inputs serial number
- Program validates the serial number
- Displays validation result (valid/invalid)
Analysis Steps
1. Preliminary Static Analysis
Load the program using IDA Pro or Ghidra:
# Use IDAPython to quickly locate key functions
import idautils
import idc
def find_validation_function():
# Search for common validation-related strings
for s in idautils.Strings():
if "valid" in s.s.lower() or "invalid" in s.s.lower():
print(f"Found string: {s.s} at {hex(s.ea)}")
# Find functions referencing this string
for ref in idautils.XrefsTo(s.ea):
func_ea = ref.frm
func_name = idc.get_func_name(func_ea)
print(f" Referenced by function: {func_name} at {hex(func_ea)}")
return func_ea
return None
validation_func = find_validation_function()
if validation_func:
print(f"Found potential validation function at {hex(validation_func)}")
else:
print("No validation function found")
2. Dynamic Debugging Analysis
Run the program using OllyDbg or x64dbg and set breakpoints:
- Set breakpoints at
GetWindowTextAor similar API calls (to capture user input) - Set breakpoints at validation result output
- Run the program and input test serial numbers
Key Breakpoint Settings:
; Set breakpoints in OllyDbg
bp GetWindowTextA ; Capture user input
bp MessageBoxA ; Display validation result
3. Reverse the Validation Logic
Observe program behavior through dynamic debugging:
- Input known valid and invalid serial numbers
- Observe how the program processes these inputs
- Locate the key code segment of the validation logic
Common Validation Patterns:
- Simple string comparison
- Mathematical operation validation (e.g., CRC check)
- Timestamp validation
- Hardware ID binding
4. Generate Valid Registration Code
Write a registration code generator based on reverse engineering results:
# Assume the validation logic is a simple string comparison
def generate_registration_code():
# Based on reverse engineering, the valid registration code is "VALID-KEY-123"
return "VALID-KEY-123"
# Test the generated registration code
print(f"Generated registration code: {generate_registration_code()}")
Handling More Complex Validation Logic:
If the validation logic is more complex (e.g., mathematical operations), the algorithm must be fully reversed:
# Assume the validation logic compares the CRC32 of the input serial with a fixed value
import zlib
def validate_serial(serial):
# Assume the fixed check value is 0x12345678
fixed_crc = 0x12345678
# Calculate CRC32 of the input serial
calculated_crc = zlib.crc32(serial.encode())
return calculated_crc == fixed_crc
# Generate a valid serial number (requires knowing the fixed check value)
def generate_serial():
# The fixed check value must be reversed
fixed_crc = 0x12345678 # Assumed value
# In practice, finding an input that generates this CRC requires more complex reverse engineering
return "GENERATED-SERIAL"
print(f"Generated serial: {generate_serial()}")
Complete Cracking Process
- Static Analysis: Locate the validation function
- Dynamic Debugging: Observe the validation logic
- Reverse Algorithm: Fully understand the validation mechanism
- Generate Registration Code: Write a generator
Case Two: Analyzing Commercial Software Protection
Target Analysis
Analyze a software protected with commercial protection tools (such as Themida or VMProtect), understand its protection mechanisms, and attempt to bypass them.
Protection Feature Observation:
- Performs anti-debugging checks on startup
- Critical code is virtualized or encrypted
- Memory access is protected
Analysis Steps
1. Anti-Debugging Detection Analysis
Run the program using OllyDbg or x64dbg:
- Set breakpoints at common anti-debugging API calls:
IsDebuggerPresentCheckRemoteDebuggerPresentNtQueryInformationProcessOutputDebugString
; Set anti-debugging breakpoints in OllyDbg
bp IsDebuggerPresent
bp CheckRemoteDebuggerPresent
bp NtQueryInformationProcess
bp OutputDebugStringA
- Run the program and observe anti-debugging behavior
2. Virtualized/Encrypted Code Analysis
Analyze the protected area using IDA Pro or Ghidra:
- Look for high-entropy regions (likely encrypted or virtualized code)
- Analyze characteristic code of the protection tool
High Entropy Detection Script:
import math
def calculate_entropy(data):
if not data:
return 0.0
entropy = 0.0
for x in range(256):
p_x = float(data.count(chr(x))) / len(data)
if p_x > 0:
entropy += - p_x * math.log(p_x, 2)
return entropy
with open("protected_program.exe", "rb") as f:
data = f.read()
entropy = calculate_entropy(data)
print(f"File entropy: {entropy:.2f}")
3. Bypassing Protection Mechanisms
Attempt to bypass protection based on analysis results:
- Anti-Debugging Bypass:
- Modify the return value of anti-debugging checks
- Remove anti-debugging code
- Virtualization/Encryption Bypass:
- Use a debugger to trace virtual machine execution
- Dump decrypted code
Anti-Debugging Bypass Example:
; In OllyDbg, modify the return value of IsDebuggerPresent
; Locate the instruction calling IsDebuggerPresent
; Modify the EAX register to 0 (non-debugging state)
Virtual Machine Code Dumping:
# Use x64dbg Python script to dump decrypted code from memory
import dbgscript
def dump_decrypted_code():
# Set breakpoint at a key point in virtual machine execution
dbgscript.breakpoint_set("vm_execution_point", lambda: dump_memory())
def dump_memory():
# Get current memory content
# Adjust based on actual protection tool
memory = dbgscript.memory.read(0x00401000, 0x1000) # Example address and size
with open("decrypted_code.bin", "wb") as f:
f.write(memory)
print("Decrypted code dumped to decrypted_code.bin")
dump_decrypted_code()
Complete Analysis Process
- Identify Protection Tool: By characteristic code and behavior
- Analyze Protection Mechanisms: Anti-debugging, virtualization, encryption, etc.
- Bypass Protection: Modify anti-debugging, dump decrypted code
- Verify Bypass: Run the modified program
Case Three: Reverse Engineering Game Cheat Detection
Target Analysis
Develop a tool to detect game cheats by analyzing common cheat features and behavior patterns.
Common Cheat Features:
- Memory modification (e.g., modifying player attributes in game memory)
- API hooking (e.g., hooking game rendering functions)
- Anti-debugging behavior
- Abnormal network communication
- Unconventional system calls
Analysis Steps
1. Cheat Behavior Monitoring
Use API monitoring tools (such as API Monitor) or custom hooks:
import ctypes
from ctypes import wintypes
# Define API hook function
def hook_function(original_func):
def hooked(*args, **kwargs):
print(f"Hooked function {original_func.__name__} called")
# Call the original function
return original_func(*args, **kwargs)
return hooked
# Example: Hook WriteProcessMemory (commonly used for memory modification)
kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
WriteProcessMemory = kernel32.WriteProcessMemory
WriteProcessMemory.argtypes = (
wintypes.HANDLE, wintypes.LPCVOID, wintypes.LPVOID,
wintypes.SIZE_T, ctypes.POINTER(wintypes.SIZE_T))
WriteProcessMemory.restype = wintypes.BOOL
# Hook WriteProcessMemory
original_WriteProcessMemory = WriteProcessMemory
WriteProcessMemory = hook_function(WriteProcessMemory)
2. Memory Scanning Detection
Scan suspicious patterns in the game process memory:
import win32process
import win32api
import win32con
def scan_process_memory(process_id):
# Open the process
h_process = win32api.OpenProcess(win32con.PROCESS_ALL_ACCESS, False, process_id)
# Scan memory regions
memory_regions = []
try:
# Get process memory information
# Requires implementation of memory region enumeration
# Simplified example; actual implementation is more complex
pass
finally:
win32api.CloseHandle(h_process)
return memory_regions
# Example: Scan game process
game_process_id = 1234 # Replace with actual game process ID
suspicious_regions = scan_process_memory(game_process_id)
print(f"Suspicious memory regions: {suspicious_regions}")
3. Behavioral Analysis Detection
Monitor abnormal behavior of the game process:
import psutil
def monitor_game_process(process_name):
for proc in psutil.process_iter(['pid', 'name', 'memory_info']):
if proc.info['name'] == process_name:
# Check memory usage patterns
mem_usage = proc.info['memory_info'].rss
print(f"Process {process_name} (PID: {proc.info['pid']}) memory usage: {mem_usage / 1024 / 1024:.2f} MB")
# Check network connections
connections = proc.connections()
for conn in connections:
print(f" Connection: {conn.laddr} -> {conn.raddr}")
# Check thread count
threads = proc.num_threads()
print(f" Threads: {threads}")
# Example: Monitor game process
monitor_game_process("game.exe")
4. Cheat Feature Matching
Use YARA rules or custom feature matching:
import yara
# Define YARA rule (example)
rules = """
rule GameHack_MemoryModification {
meta:
description = "Detects memory modification patterns"
strings:
$write_mem = { 6A ?? 68 ?? ?? ?? ?? 64 A1 ?? ?? ?? ?? 50 64 89 25 ?? ?? ?? ?? 83 EC 10 }
condition:
$write_mem
}
"""
# Compile the rule
yara_rules = yara.compile(source=rules)
# Scan file or memory
matches = yara_rules.match(filepath="game.exe") # Or scan memory
print(f"YARA matches: {matches}")
Complete Cheat Detection System
- Behavior Monitoring: API hooks, memory scanning, network monitoring
- Feature Matching: YARA rules, custom features
- Behavioral Analysis: Detect abnormal behavior (e.g., high-frequency memory modifications)
- Report Generation: Generate detection reports and evidence
Comprehensive Practical Techniques and Precautions
General Reverse Engineering Techniques
- Static Analysis:
- Use IDA Pro/Ghidra for function and string analysis
- Search for key API calls and string hints
- Dynamic Debugging:
- Use OllyDbg/x64dbg to set breakpoints and observe program behavior
- Trace key function calls and data flow
- Feature Extraction:
- Extract program characteristic code and behavior patterns
- Analyze protection mechanisms and validation logic
- Algorithm Reversal:
- Fully understand validation and encryption algorithms
- Write corresponding generators or decryption tools
Security and Legal Precautions
- Legal Authorization:
- Only analyze software you own or are authorized to analyze
- Comply with local laws and regulations
- Ethical Guidelines:
- Do not use reverse engineering for malicious purposes
- Respect software copyright and intellectual property
- Security Practices:
- Conduct analysis in an isolated environment (e.g., virtual machine)
- Use antivirus software to protect the analysis environment
Learning Path Recommendations
- Basic Stage:
- Learn assembly language and reverse engineering fundamentals
- Master the use of IDA Pro/Ghidra and debuggers
- Intermediate Stage:
- Deep dive into common protection mechanisms and bypass techniques
- Study algorithm reversal in reverse engineering
- Advanced Stage:
- Develop automated reverse engineering tools
- Research advanced protection technologies (e.g., virtual machine protection)
- Learn malware analysis techniques
Through the study and practice of these three practical cases, you will master the core skills of reverse engineering, enabling you to analyze various protection mechanisms and develop corresponding detection and cracking tools. Remember, reverse engineering is a double-edged sword—always use these skills within legal and ethical boundaries.



