Batch file analysis, custom decryption tools, and automated unpacking frameworks are key technologies for improving efficiency in reverse engineering. This tutorial will delve into how to develop these tools, including complete code implementations, technical details, and practical cases, helping readers master the core technologies of automated reverse engineering.
Batch File Analysis Script Development
Batch Analysis Framework Design
The core of a batch file analysis script is the ability to efficiently process large numbers of files, extract key information, and generate reports. The following is the design approach:
- File Traversal and Filtering:
- Recursively traverse directory structures
- Filter target files based on file extensions or characteristics
- Parallel Processing:
- Use multithreading/multiprocessing to accelerate analysis
- Reasonably allocate system resources
- Result Collection and Reporting:
- Uniformly collect analysis results
- Generate structured reports (HTML/CSV/JSON)
Python Batch Analysis Script Example
import os
import concurrent.futures
import json
from datetime import datetime
class FileAnalyzer:
def __init__(self):
self.results = []
def analyze_file(self, file_path):
"""Core logic for analyzing a single file"""
try:
# Implement specific analysis logic here
# Example: Get basic file information
file_size = os.path.getsize(file_path)
file_hash = self.calculate_md5(file_path)
# Return analysis result
return {
"file_path": file_path,
"file_size": file_size,
"file_hash": file_hash,
"analysis_time": datetime.now().isoformat()
}
except Exception as e:
return {
"file_path": file_path,
"error": str(e),
"analysis_time": datetime.now().isoformat()
}
def calculate_md5(self, file_path):
"""Calculate file MD5 hash value"""
import hashlib
hash_md5 = hashlib.md5()
with open(file_path, "rb") as f:
for chunk in iter(lambda: f.read(4096), b""):
hash_md5.update(chunk)
return hash_md5.hexdigest()
def batch_analyze(self, directory):
"""Batch analyze files in a directory"""
file_paths = []
# Recursively traverse directory
for root, _, files in os.walk(directory):
for file in files:
file_path = os.path.join(root, file)
file_paths.append(file_path)
# Use thread pool for parallel processing
with concurrent.futures.ThreadPoolExecutor() as executor:
futures = [executor.submit(self.analyze_file, path) for path in file_paths]
for future in concurrent.futures.as_completed(futures):
self.results.append(future.result())
return self.results
def generate_report(self, output_path):
"""Generate analysis report"""
with open(output_path, 'w') as f:
json.dump(self.results, f, indent=2)
print(f"Report generated: {output_path}")
# Usage example
if __name__ == "__main__":
analyzer = FileAnalyzer()
results = analyzer.batch_analyze("/path/to/target/directory")
analyzer.generate_report("analysis_report.json")
Extending Analysis Functions
In practical applications, the following analysis functions can be extended:
- PE File Analysis:
- Use the
pefilelibrary to analyze PE file header information - Extract import tables and export tables
- Use the
- Malware Feature Detection:
- Detect common malware features (such as specific strings, API call patterns)
- Use YARA rules for matching
- Behavioral Analysis:
- Run files in a sandbox and collect behavioral data
- Analyze network communication, file operations, etc.
Advanced Batch Analysis Framework
For more complex batch analysis needs, a complete analysis framework can be built:
class AdvancedBatchAnalyzer:
def __init__(self):
self.plugins = [] # Plugin system
self.results_db = {} # Results database
def register_plugin(self, plugin):
"""Register analysis plugin"""
self.plugins.append(plugin)
def analyze_file(self, file_path):
"""Analyze a single file using all plugins"""
file_results = {"file_path": file_path}
for plugin in self.plugins:
try:
result = plugin.analyze(file_path)
file_results.update(result)
except Exception as e:
file_results[f"plugin_error_{plugin.name}"] = str(e)
return file_results
def batch_analyze(self, directory):
"""Batch analyze directory"""
file_paths = self._collect_files(directory)
# Use multiprocessing to accelerate (more suitable for CPU-intensive tasks than threads)
from multiprocessing import Pool
with Pool() as pool:
results = pool.map(self.analyze_file, file_paths)
# Store results
for result in results:
self.results_db[result["file_path"]] = result
return results
def _collect_files(self, directory):
"""Collect target files"""
# Implement file filtering logic
pass
def generate_report(self, output_path):
"""Generate detailed report"""
# Implement report generation logic
pass
# Plugin example
class PEAnalyzerPlugin:
def __init__(self):
self.name = "PE Analyzer"
def analyze(self, file_path):
"""Analyze PE file"""
import pefile
try:
pe = pefile.PE(file_path)
return {
"pe_headers": {
"machine": pe.FILE_HEADER.Machine,
"number_of_sections": pe.FILE_HEADER.NumberOfSections
},
"imports": [entry.dll for entry in pe.DIRECTORY_ENTRY_IMPORT]
}
except Exception as e:
return {"pe_error": str(e)}
finally:
if 'pe' in locals():
pe.close()
# Usage example
if __name__ == "__main__":
analyzer = AdvancedBatchAnalyzer()
analyzer.register_plugin(PEAnalyzerPlugin())
# Can register more plugins...
results = analyzer.batch_analyze("/path/to/target/directory")
analyzer.generate_report("advanced_report.json")
Custom Decryption Tool Development
Decryption Tool Design Principles
- Modular Design:
- Separate decryption algorithms, key management, and file processing
- Support multiple encryption algorithms and key sources
- Extensibility:
- Easy to add new decryption algorithms
- Support custom key extraction logic
- Security:
- Securely handle keys and sensitive data
- Prevent memory leaks and information disclosure
Common Decryption Scenarios and Implementations
1. Simple Encrypted File Decryption
class SimpleDecryptor:
def __init__(self, key):
self.key = key
def decrypt_file(self, input_path, output_path):
"""Decrypt a simply encrypted file"""
with open(input_path, 'rb') as f_in:
encrypted_data = f_in.read()
# Simple XOR decryption example
decrypted_data = bytes([b ^ self.key for b in encrypted_data])
with open(output_path, 'wb') as f_out:
f_out.write(decrypted_data)
# Usage example
decryptor = SimpleDecryptor(key=0x55) # Assume 0x55 is used as the key
decryptor.decrypt_file("encrypted.bin", "decrypted.bin")
2. AES Encrypted File Decryption
from Crypto.Cipher import AES
from Crypto.Util.Padding import unpad
class AESDecryptor:
def __init__(self, key, iv):
self.key = key
self.iv = iv
def decrypt_file(self, input_path, output_path):
"""Decrypt an AES-encrypted file"""
with open(input_path, 'rb') as f_in:
encrypted_data = f_in.read()
# Create AES decryptor
cipher = AES.new(self.key, AES.MODE_CBC, self.iv)
# Decrypt and remove padding
decrypted_data = unpad(cipher.decrypt(encrypted_data), AES.block_size)
with open(output_path, 'wb') as f_out:
f_out.write(decrypted_data)
# Usage example
key = b'Sixteen byte key' # 16-byte key
iv = b'Sixteen byte iv ' # 16-byte IV
decryptor = AESDecryptor(key, iv)
decryptor.decrypt_file("aes_encrypted.bin", "aes_decrypted.bin")
3. Custom Encryption Algorithm Decryption
For custom encryption algorithms, the corresponding decryption logic needs to be implemented:
class CustomDecryptor:
def __init__(self, key):
self.key = key
def decrypt_file(self, input_path, output_path):
"""Decrypt a file with custom encryption"""
with open(input_path, 'rb') as f_in:
encrypted_data = f_in.read()
# Implement custom decryption algorithm
decrypted_data = bytearray()
key_index = 0
for byte in encrypted_data:
# Simple example: XOR with key byte
decrypted_byte = byte ^ self.key[key_index % len(self.key)]
decrypted_data.append(decrypted_byte)
key_index += 1
with open(output_path, 'wb') as f_out:
f_out.write(decrypted_data)
# Usage example
key = b'my_custom_key'
decryptor = CustomDecryptor(key)
decryptor.decrypt_file("custom_encrypted.bin", "custom_decrypted.bin")
Key Extraction and Dynamic Decryption
For encrypted files, the key may be hidden in the program or need to be dynamically extracted:
class KeyExtractor:
def __init__(self, program_path):
self.program_path = program_path
def extract_key(self):
"""Extract key from the program"""
# Implement key extraction logic here
# Example: Read hard-coded key from a specific location in the PE file
import pefile
pe = pefile.PE(self.program_path)
# Assume the key is stored at a specific location in the PE file
# In practice, this needs to be analyzed based on the specific program
key_offset = 0x1000 # Example offset
key_length = 16 # 16-byte key
with open(self.program_path, 'rb') as f:
f.seek(key_offset)
key_data = f.read(key_length)
return key_data
# Usage example
extractor = KeyExtractor("encrypted_program.exe")
key = extractor.extract_key()
print(f"Extracted key: {key}")
# Then use the extracted key to decrypt the file
decryptor = AESDecryptor(key, iv=b'Sixteen byte iv ')
decryptor.decrypt_file("encrypted_data.bin", "decrypted_data.bin")
Automated Unpacking Framework
Unpacking Framework Design
An automated unpacking framework needs to address the following key issues:
- Packing Detection:
- Identify whether a file is packed
- Identify the type of packer used
- Unpacking Execution:
- Control program execution flow
- Capture decrypted/decompressed code
- Result Verification:
- Verify the correctness of the unpacking result
- Handle possible error conditions
Python Automated Unpacking Framework Example
import os
import subprocess
from pefile import PE
class AutomatedUnpacker:
def __init__(self):
self.known_packers = {
"UPX": self.unpack_upx,
"VMProtect": self.unpack_vmp,
# Can add support for more packers
}
def detect_packer(self, file_path):
"""Detect if the file is packed and the packer type"""
try:
pe = PE(file_path)
# Check for common packing characteristics
# This is just a simple example; in practice, more complex detection logic is needed
if hasattr(pe, 'DIRECTORY_ENTRY_IMPORT'):
for entry in pe.DIRECTORY_ENTRY_IMPORT:
if entry.dll.lower() == 'upx.dll':
return "UPX"
elif entry.dll.lower() == 'vmp.dll':
return "VMProtect"
# Check entropy value
entropy = self.calculate_entropy(file_path)
if entropy > 7.0: # High entropy may indicate compression/encryption
return "Unknown (High Entropy)"
return None
except Exception as e:
print(f"Error detecting packer: {e}")
return None
def calculate_entropy(self, file_path):
"""Calculate file entropy value"""
import math
with open(file_path, 'rb') as f:
data = f.read()
if not data:
return 0.0
entropy = 0.0
for x in range(256):
p_x = float(data.count(chr(x))) / len(data)
if p_x > 0:
entropy += - p_x * math.log(p_x, 2)
return entropy
def unpack_file(self, file_path, output_path):
"""Automatically unpack the file"""
packer = self.detect_packer(file_path)
if not packer:
print("No known packer detected")
# Can try to directly copy the file
with open(file_path, 'rb') as f_in:
with open(output_path, 'wb') as f_out:
f_out.write(f_in.read())
return True
print(f"Detected packer: {packer}")
if packer in self.known_packers:
return self.known_packers[packer](file_path, output_path)
else:
print(f"No handler for packer: {packer}")
return False
def unpack_upx(self, file_path, output_path):
"""Unpack using UPX"""
try:
# Use UPX command-line tool to unpack
subprocess.run(["upx", "-d", file_path, "-o", output_path], check=True)
return True
except subprocess.CalledProcessError as e:
print(f"UPX unpack failed: {e}")
return False
def unpack_vmp(self, file_path, output_path):
"""Handle VMProtect packed files"""
# VMProtect requires more complex handling
# This is just an example; in practice, full unpacking logic needs to be implemented
print("VMProtect unpacking not fully implemented")
# Can try using a debugger or dedicated tool
# For example, use x64dbg to set breakpoints at decryption code
# Here simply return False indicating not implemented
return False
# Usage example
unpacker = AutomatedUnpacker()
if unpacker.unpack_file("packed_program.exe", "unpacked_program.exe"):
print("Unpacking succeeded")
else:
print("Unpacking failed")
Advanced Unpacking Framework
For more complex unpacking needs, a complete automated unpacking framework can be built:
class AdvancedUnpackerFramework:
def __init__(self):
self.unpackers = {} # Packer handlers
self.debuggers = {} # Debugger interfaces
self.memory_analyzers = {} # Memory analysis tools
def register_unpacker(self, packer_name, unpacker_func):
"""Register packer handler"""
self.unpackers[packer_name] = unpacker_func
def register_debugger(self, debugger_name, debugger_func):
"""Register debugger interface"""
self.debuggers[debugger_name] = debugger_func
def register_memory_analyzer(self, analyzer_name, analyzer_func):
"""Register memory analysis tool"""
self.memory_analyzers[analyzer_name] = analyzer_func
def detect_and_unpack(self, file_path, output_path):
"""Detect packing and automatically unpack"""
# Detect packer
packer = self.detect_packer(file_path)
if not packer:
# No packer detected, directly copy the file
with open(file_path, 'rb') as f_in:
with open(output_path, 'wb') as f_out:
f_out.write(f_in.read())
return True
# Get the corresponding unpacker
unpacker = self.unpackers.get(packer)
if unpacker:
return unpacker(file_path, output_path)
else:
# No corresponding unpacker, try using debugger
debugger = self.debuggers.get("x64dbg") # Example using x64dbg
if debugger:
return debugger.unpack(file_path, output_path)
else:
print(f"No handler for packer: {packer}")
return False
def detect_packer(self, file_path):
"""Detect packer"""
# Implement more complex detection logic
pass
# Usage example
framework = AdvancedUnpackerFramework()
# Register UPX unpacker
def unpack_upx(file_path, output_path):
import subprocess
subprocess.run(["upx", "-d", file_path, "-o", output_path], check=True)
return True
framework.register_unpacker("UPX", unpack_upx)
# Register x64dbg debugger
def x64dbg_unpack(file_path, output_path):
# Implement x64dbg automated unpacking logic
# This is just an example
print("Using x64dbg for unpacking")
return True
framework.register_debugger("x64dbg", x64dbg_unpack)
# Use framework to unpack
if framework.detect_and_unpack("packed_program.exe", "unpacked_program.exe"):
print("Unpacking succeeded")
else:
print("Unpacking failed")
Comprehensive Practical Case
Case: Batch Analysis of Malware Samples
Objective: Develop an automated tool to batch analyze malware samples, including:
- Batch file analysis
- Custom decryption (if needed)
- Automated unpacking
- Generate comprehensive report
Implementation Steps:
- Batch File Analysis:
- Use the previously developed
AdvancedBatchAnalyzerframework - Add plugins for PE file analysis, YARA rule matching, etc.
- Use the previously developed
- Custom Decryption:
- Implement decryption logic for specific malware
- Integrate into the analysis workflow
- Automated Unpacking:
- Use
AdvancedUnpackerFrameworkto handle packed samples - For unhandled packers, integrate dedicated tools
- Use
- Report Generation:
- Generate an HTML report containing all analysis results
- Include file basic information, decryption results, unpacking results, etc.
Complete Code Framework:
class MalwareAnalysisSuite:
def __init__(self):
self.batch_analyzer = AdvancedBatchAnalyzer()
self.unpacker = AdvancedUnpackerFramework()
# Can add more components
def analyze_samples(self, input_dir, output_report):
"""Batch analyze malware samples"""
# 1. Batch analysis
results = self.batch_analyzer.batch_analyze(input_dir)
# 2. Handle samples that need decryption or unpacking
for result in results:
file_path = result["file_path"]
# Check if decryption is needed
if self.needs_decryption(result):
decrypted_path = self.decrypt_file(file_path)
result["decrypted_path"] = decrypted_path
# Check if unpacking is needed
if self.needs_unpacking(result):
unpacked_path = self.unpack_file(file_path)
result["unpacked_path"] = unpacked_path
# 3. Generate report
self.generate_report(results, output_report)
def needs_decryption(self, result):
"""Determine if decryption is needed"""
# Implement judgment logic
return False
def decrypt_file(self, file_path):
"""Decrypt file"""
# Implement decryption logic
return file_path
def needs_unpacking(self, result):
"""Determine if unpacking is needed"""
# Implement judgment logic
return False
def unpack_file(self, file_path):
"""Unpack file"""
# Implement unpacking logic
return file_path
def generate_report(self, results, output_path):
"""Generate report"""
# Implement report generation logic
pass
# Usage example
if __name__ == "__main__":
suite = MalwareAnalysisSuite()
suite.analyze_samples("/path/to/malware/samples", "malware_analysis_report.html")
Summary and Learning Path
Batch file analysis, custom decryption tools, and automated unpacking frameworks are core technologies for reverse engineering automation. This tutorial has detailed:
- Batch File Analysis:
- Design efficient analysis frameworks
- Implement parallel processing and result collection
- Extend analysis functions (PE analysis, malware detection, etc.)
- Custom Decryption Tools:
- Implement decryption for common encryption algorithms
- Handle key extraction and dynamic decryption
- Build extensible decryption frameworks
- Automated Unpacking Frameworks:
- Design packing detection and handling processes
- Implement automated unpacking for common packers like UPX
- Build complete automated unpacking frameworks
Learning Path Recommendations:
- Basic Stage:
- Learn Python programming and file operations
- Master PE file analysis fundamentals
- Understand principles of common encryption algorithms
- Intermediate Stage:
- Deep dive into parallel processing and multiprocessing programming
- Study more encryption algorithms and unpacking techniques
- Learn YARA rule writing and malware detection
- Advanced Stage:
- Research advanced unpacking techniques (such as virtual machine protection)
- Develop complete automated analysis frameworks
- Study the latest malware analysis and reverse engineering technologies
Through continuous learning and practice, you will be able to master advanced automated reverse engineering techniques and contribute to security research and malware analysis.



