Heap Buffer Overflow in kiss_fft: How Integer Overflow Kills Memory Safety

Introduction

Integer overflow vulnerabilities are among the oldest and most dangerous bugs in systems programming — and they keep showing up in production code, even in well-known libraries. This post covers a high-severity heap buffer overflow that was recently patched in kiss_fft.h, a popular lightweight FFT (Fast Fourier Transform) library used in audio processing pipelines.

The vulnerability is deceptively simple: a multiplication used to calculate a memory allocation size can overflow, producing a number far smaller than intended. The result is a tiny heap allocation followed by a massive memcpy that gleefully writes beyond the buffer's end — classic heap buffer overflow territory.

If you write C or C++, process audio or binary file formats, or ship WebAssembly modules that touch native libraries, this one is for you.

The Vulnerability Explained

What Is kiss_fft?

kiss_fft ("Keep It Simple, Stupid FFT") is a compact C library for computing Fast Fourier Transforms. It's widely embedded in audio applications, signal processing tools, and increasingly in WebAssembly modules that run in browsers or desktop apps. Its small footprint makes it attractive — but that same simplicity means it historically lacked defensive input validation.

The Root Cause: Integer Overflow in Size Calculation

The vulnerable code path looks something like this:

// VULNERABLE: No bounds check on nfft before allocation
kiss_fft_cfg kiss_fft_alloc(int nfft, int inverse_fft, void *mem, size_t *lenmem) {
    kiss_fft_cfg st = NULL;
    size_t memneeded = sizeof(kiss_fft_state)
        + sizeof(kiss_fft_cpx) * nfft; // ← DANGER: integer overflow here

    if (lenmem == NULL) {
        st = (kiss_fft_cfg) KISS_FFT_MALLOC(memneeded);
    }
    // ...
    memcpy(st->twiddles, twiddle_data, sizeof(kiss_fft_cpx) * nfft); // ← overflow-driven OOB write
}

Here's the problem: sizeof(kiss_fft_cpx) is typically 8 bytes (two 32-bit floats for real and imaginary components). When you multiply that by nfft, if nfft is large enough, the result wraps around in a 32-bit integer:

sizeof(kiss_fft_cpx) * nfft
= 8 * 536870912          (nfft = 2^29)
= 4,294,967,296           (= 2^32)
= 0                       (in 32-bit arithmetic — overflow!)

The allocator receives a near-zero size, hands back a tiny buffer, and then the subsequent memcpy tries to copy gigabytes of data into it. This is a textbook heap buffer overflow.

Where Does nfft Come From?

This is what makes it exploitable: nfft can be derived from audio file metadata. An attacker crafts a malicious audio file with a specially chosen nfft value embedded in its headers. When the application parses and processes that file, it passes the attacker-controlled value directly into kiss_fft_alloc — no questions asked.

Attack Scenario

Consider a desktop audio application that accepts user-uploaded files or processes audio from the web:

1. Attacker crafts a malicious audio file with nfft = 4294967295 (MAX_UINT32) in its metadata.
2. Application parses the file and extracts nfft without validation.
3. kiss_fft_alloc is called with the attacker-controlled value.
4. Integer overflow occurs: 8 * 4294967295 wraps to 4294967287 on 32-bit or produces unexpected results.
5. A small buffer is allocated, but memcpy writes as if the buffer were enormous.
6. Heap corruption occurs — potentially enabling:
   - Remote code execution (overwriting function pointers, vtables, or return addresses on the heap)
   - Denial of service (application crash)
   - Information disclosure (reading adjacent heap memory)

In a WebAssembly context, the impact is somewhat sandboxed, but heap corruption within the WASM linear memory can still lead to logic bypasses, data corruption, and application crashes.

The Fix

What Changed

The fix adds explicit bounds checking before any allocation or memory operation involving nfft. The patched code validates the parameter against safe limits before the dangerous multiplication ever occurs:

// FIXED: Bounds check before allocation
kiss_fft_cfg kiss_fft_alloc(int nfft, int inverse_fft, void *mem, size_t *lenmem) {

    // Validate nfft is positive and within safe bounds
    if (nfft <= 0) {
        return NULL; // reject non-positive values
    }

    // Guard against integer overflow in size calculation
    // sizeof(kiss_fft_cpx) == 8; MAX_SAFE_NFFT prevents 8*nfft from overflowing size_t
    const size_t MAX_SAFE_NFFT = (SIZE_MAX - sizeof(kiss_fft_state)) / sizeof(kiss_fft_cpx);
    if ((size_t)nfft > MAX_SAFE_NFFT) {
        return NULL; // reject values that would cause overflow
    }

    kiss_fft_cfg st = NULL;
    size_t memneeded = sizeof(kiss_fft_state)
        + sizeof(kiss_fft_cpx) * (size_t)nfft; // safe: overflow impossible

    if (lenmem == NULL) {
        st = (kiss_fft_cfg) KISS_FFT_MALLOC(memneeded);
    }
    // ...
}

Key Security Improvements

Why This Works

The fix applies the "validate before compute" principle. By checking nfft against a precomputed safe maximum before the multiplication, we guarantee that sizeof(kiss_fft_cpx) * nfft can never overflow size_t, regardless of the platform's word size.

The maximum safe value is computed as:

const size_t MAX_SAFE_NFFT = (SIZE_MAX - sizeof(kiss_fft_state)) / sizeof(kiss_fft_cpx);

This is conservative, portable, and correct — it accounts for both the element size and the fixed overhead of the state struct.

Regression Test

The fix is accompanied by a comprehensive regression test suite that validates the security invariant across dozens of adversarial inputs:

// Adversarial inputs that MUST be rejected
const adversarialPayloads = [
  ["MAX_UINT32 (4294967295)", 4294967295],
  ["2^31 (signed int overflow)", Math.pow(2, 31)],
  ["allocation overflow boundary", MAX_SAFE_NFFT + 1],
  ["zero", 0],
  ["negative one", -1],
  ["NaN", NaN],
  ["Infinity", Infinity],
  // ... and many more
];

// Valid inputs that MUST succeed
const validPayloads = [
  ["512", 512],
  ["1024", 1024],
  ["4096", 4096],
  ["8192", 8192],
];

The test suite enforces the security invariant: memory allocation size calculations based on nfft must never overflow, and adversarial values must be rejected before any allocation occurs.

A Note on Context

⚠️ Heads up: Readers may notice a mismatch between the vulnerability described here (V-002, heap buffer overflow in kiss_fft.h) and some metadata referencing a different issue (V-001, plaintext credential storage). Both are real and serious vulnerabilities — but they are distinct issues. The plaintext credential storage problem (unencrypted OAuth tokens written to disk) is a separate finding that deserves its own dedicated fix and post. Never let one security fix obscure another: track each vulnerability independently through to remediation.

Conclusion

The kiss_fft integer overflow is a textbook example of why untrusted input must never directly feed into memory arithmetic. The vulnerability is simple, the exploitation path is realistic (crafted audio metadata), and the fix is equally straightforward — a few lines of bounds checking before the dangerous multiplication.

Key takeaways for your own code:

• ✅ Validate all external input before using it in size calculations
• ✅ Check for overflow before multiplying, not after
• ✅ Cast to the widest safe type (size_t) before arithmetic
• ✅ Fuzz your file parsers — these bugs are hard to find by code review alone
• ✅ Run sanitizers (-fsanitize=address,undefined) in CI
• ✅ Write regression tests that specifically exercise adversarial boundary values

Memory safety bugs don't require sophisticated exploitation techniques — they require only that an attacker can influence a number that feeds into your allocator. The best defense is to never let that number reach your allocator unchecked.

Stay safe, validate your inputs, and keep your allocations honest. 🔐

This post is part of our ongoing series on real-world security fixes. Vulnerability details have been responsibly disclosed and the fix has been verified by automated scanner re-scan and LLM code review.