What are Transparent Huge Pages (THP) and how do they reduce TLB miss overhead?

In x86_64 architecture, standard virtual memory uses 4KB base pages managed through a 4-level or 5-level page table hierarchy (PGD -> P4D -> PUD -> PMD -> PTE). The CPU Translation Lookaside Buffer (TLB) caches virtual-to-physical address translations. With 4KB pages, a 64-entry L1 D-TLB can only cover 256KB of active memory, causing frequent TLB misses on memory-intensive workloads (databases, JVM, scientific computing). Transparent Huge Pages (THP) collapse 512 contiguous 4KB pages into a single 2MB PMD-level compound page. A single 2MB TLB entry covers 512 times more virtual memory, reducing TLB miss cycles from 15-25% of total CPU runtime down to under 1%.

How does the khugepaged kernel daemon dynamically collapse base pages into huge pages?

The khugepaged kernel thread runs periodically in the background to identify memory regions qualified for huge page promotion without application intervention. It iterates through process VMAs at intervals governed by /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs, scanning up to pages_to_scan per pass. When it detects 512 contiguous 4KB pages mapped within a single 2MB-aligned boundary, it allocates an order-9 (2MB) compound page, acquires the split page table lock (split_ptlock), copies the 512 base pages into the huge page, updates the PMD entry to point directly to the compound page, frees the old 4KB PTEs, and issues a TLB shootdown (flush_tlb_mm).

Why does transparent_hugepage=always cause severe latency spikes in databases like Redis and PostgreSQL?

Under "always" mode, the Linux kernel attempts synchronous huge page allocation on every minor page fault. If physical memory is fragmented and no contiguous 2MB physical block is available, the kernel triggers synchronous Direct Compaction and Memory Reclamation. The faulting application thread stalls for tens or hundreds of milliseconds while the kernel shifts surrounding pages and flushes caches. Additionally, copy-on-write (COW) forks (such as Redis background BGSAVE) must duplicate entire 2MB pages instead of 4KB pages on small writes, ballooning memory usage and freezing client transactions. Enterprise production environments universally set THP to "madvise" so only dedicated memory regions opt-in.

What triggers split_huge_page() and how does the kernel un-collapse a compound page?

When an application performs operations requiring sub-2MB granularity—such as unmapping a portion of memory (munmap), changing page protections (mprotect), punching holes with fallocate, or swapping out pages under memory pressure—the kernel cannot handle the request at PMD granularity. It invokes split_huge_page(), which allocates a new PTE page table, redistributes the compound page metadata across 512 individual struct page entries, and demotes the PMD entry back to a regular page table pointer with 512 distinct 4KB PTE mappings.

Linux Transparent Huge Pages (THP) & Khugepaged Studio

Verified Specification | Standardized Formulas | Instant Precision

Secure & Private (Zero Data Retention) Free Access • No Sign-Up

Linux Transparent Huge Pages (THP) & Khugepaged Studio

Model Linux virtual memory paging architectures. Simulate 4KB base pages vs 2MB PMD compound huge pages, khugepaged scan-and-collapse passes, TLB miss penalty reductions, and direct compaction latency spikes.

Linux MM 2MB PMD Compound khugepaged TLB Coverage 512x

THP Kernel Mode

/sys/kernel/mm/transparent_hugepage/enabled

Memory Allocation Size: 32 MB

Virtual address space allocated by application

Memory Access Pattern

Workload memory access topology

Memory Fragmentation: 25%

External physical page fragmentation level

2MB PMD Blocks & Base Page Table Topology (512 Pages per PMD)

Collapsed 2MB THP 4KB Base Page Fragmentation Hole Zero Page (Unfaulted)

Allocated RAM

32 MB

16 PMD Regions

2MB Huge Pages

75.0% THP Coverage

4KB Base Pages

2,048

Uncollapsed PTEs

TLB Hit Rate

98.4%

TLB Miss Overhead: 1.6%

P99 Alloc Latency

2.4 µs

Clean Alloc (Async)

Memory Bloat

4.2%

Internal Fragmentation

Kernel Sysfs Configuration & Application madvise() Code

Paging Hierarchy & TLB Architecture Specifications

1. Page Table Hierarchy Depth Standard 4KB paging requires walking 4 levels: PGD (Page Global Dir) -> P4D -> PUD (Page Upper Dir) -> PMD (Page Mid Dir) -> PTE (Page Table Entry). A 2MB THP maps directly at the PMD layer with the _PAGE_PSE (Page Size Extension) bit set, bypassing the 5th level PTE entirely.

2. Split Page Table Lock (split_ptlock) When khugepaged collapses or splits pages, it locks only the specific PMD entry rather than acquiring the global mm->mmap_lock write lock. This enables multi-threaded applications to service concurrent read/write faults without global lock contention.

3. Direct Compaction Overhead Under memory pressure with transparent_hugepage=always, if an order-9 allocation fails, kernel compaction moves order-0 pages to consolidate free blocks. In latency-critical workloads (trading, databases), this introduces catastrophic 50-200ms latency spikes.

Linux Transparent Huge Pages (THP) & Khugepaged Studio

2MB PMD Blocks & Base Page Table Topology (512 Pages per PMD)

Kernel Sysfs Configuration & Application madvise() Code

Paging Hierarchy & TLB Architecture Specifications

🔍 Companion Workbenches & Verification Tools

Linux EEVDF Kernel Scheduler & Virtual Deadline Studio

WebAssembly (Wasm), WASI & Component Model Architecture Studio

Linux AF_XDP (XSK), UMEM Ring Buffers & Zero-Copy Packet Processing Studio

Unix Timestamp & Epoch Converter

Why Smart Users Are Switching to This Free Toolkit

The Browser Feature Most People Never Use

Stop Paying for Software You Can Get Free

File Conversion Tricks That Save Hours Every Week

This Free Tool Replaced 5 Paid Apps for Our Team

What Your Browser Can Do Without Any Extensions