Reverse Engineering Fundamentals
Foundations
Understanding CPU architecture, registers, memory layout, and calling conventions is essential before diving into disassembly and binary analysis.
x86-64 Registers
asm
; General Purpose Registers (64-bit)
RAX - Accumulator (return values)
RBX - Base register
RCX - Counter (loop counter, 1st arg Windows)
RDX - Data (2nd arg Windows)
RSI - Source Index (2nd arg Linux)
RDI - Destination Index (1st arg Linux)
RBP - Base Pointer (stack frame)
RSP - Stack Pointer
; Additional x64 registers
R8-R15 - Extended registers
; Calling Convention (x64 Linux - System V ABI)
; Arguments: RDI, RSI, RDX, RCX, R8, R9, then stack
; Return value: RAX
; Calling Convention (x64 Windows)
; Arguments: RCX, RDX, R8, R9, then stack
; Return value: RAX
; Important flags (RFLAGS)
ZF - Zero Flag (result is zero)
SF - Sign Flag (result is negative)
CF - Carry Flag (unsigned overflow)
OF - Overflow Flag (signed overflow)Memory Layout
High Memory
┌─────────────────┐
│ Stack │ ← Grows downward (local vars, return addresses)
│ ↓ │
├─────────────────┤
│ │
│ Free Memory │
│ │
├─────────────────┤
│ ↑ │
│ Heap │ ← Grows upward (dynamic allocation)
├─────────────────┤
│ .bss │ ← Uninitialized global data
├─────────────────┤
│ .data │ ← Initialized global data
├─────────────────┤
│ .text │ ← Code (executable)
└─────────────────┘
Low Memory