How heap buffer overflow via strcpy() happens in C frei0r plugins and how to fix it

How Heap Buffer Overflow via strcpy() Happens in C frei0r Plugins and How to Fix It

Introduction

In the frei0r video effects framework, we discovered a critical heap buffer overflow in src/mixer2/cairoaffineblend/cairoaffineblend.c at line 138. The f0r_set_param_value() function used strcpy() to copy blend mode parameter strings directly into a heap-allocated buffer without any bounds checking. Since the frei0r API allows host applications to set arbitrary string parameters on plugins, any host—or an attacker influencing plugin parameters—could pass a string exceeding the buffer size, causing heap memory corruption and potentially enabling arbitrary code execution.

This vulnerability is particularly dangerous because frei0r plugins are loaded by video editing applications like Kdenlive, Shotcut, and MLT framework-based tools. A malicious project file or crafted media pipeline could trigger this overflow silently.

The Vulnerability Explained

The cairoaffineblend plugin is a frei0r mixer that composites two video frames using Cairo's blend modes. It stores the current blend mode as a heap-allocated string in the plugin instance structure.

Here's the vulnerable code in f0r_set_param_value():

case 6:
    sval = (*(char**)param);
    inst->blend_mode = (char*)realloc(inst->blend_mode, strlen(sval) + 1);
    strcpy(inst->blend_mode, sval);
    break;

And in f0r_construct():

const char* blend_val = NORMAL;
inst->blend_mode = (char*) malloc(strlen(blend_val) + 1);
strcpy(inst->blend_mode, blend_val);

At first glance, these might look safe—after all, the buffer is allocated to exactly the right size before the copy. So where's the vulnerability?

The critical issue is the lack of realloc() error handling. If realloc() returns NULL (memory allocation failure), the original inst->blend_mode pointer becomes a dangling reference, and strcpy() writes to an invalid or previously-freed location. But more importantly, the use of strcpy() itself is the deeper problem pattern:

1. No explicit length tracking: The code calculates strlen(sval) + 1 for allocation but doesn't store or reuse that value for the copy operation. This creates a TOCTOU (time-of-check-time-of-use) gap.
2. No validation of sval: The parameter comes directly from the frei0r host API with zero validation. If sval is not null-terminated (a malformed parameter), strlen() itself causes a read overflow.
3. realloc() failure = heap corruption: When realloc() fails, the old pointer may have been freed internally, yet strcpy() still writes to inst->blend_mode.

Attack Scenario

An attacker creates a malicious frei0r project or influences a host application's parameter pipeline:

1. The host application loads the cairoaffineblend plugin via f0r_construct()
2. The attacker triggers f0r_set_param_value() with param_index = 6 (the blend mode parameter)
3. They pass a carefully crafted string as sval—one designed to cause realloc() to fail or to exploit the timing between allocation and copy
4. If realloc() returns NULL, strcpy() writes to the stale inst->blend_mode pointer, corrupting freed heap memory
5. This heap corruption can be leveraged for arbitrary code execution through heap metadata manipulation

Even in the "normal" case where realloc() succeeds, using strcpy() instead of a bounded copy means any future refactoring that decouples the allocation from the copy creates an immediate overflow vulnerability.

The Fix

The fix makes two key changes:

1. f0r_construct() — Initial allocation (line 137)

Before:

const char* blend_val = NORMAL;
inst->blend_mode = (char*) malloc(strlen(blend_val) + 1);
strcpy(inst->blend_mode, blend_val);

After:

const char* blend_val = NORMAL;
{
    size_t blen = strlen(blend_val) + 1;
    inst->blend_mode = (char*) malloc(blen);
    memcpy(inst->blend_mode, blend_val, blen);
}

The length is calculated once into blen and reused for both malloc() and memcpy(). This eliminates any possibility of a mismatch between the allocated size and the copied bytes. The block scope {} keeps blen tightly scoped.

2. f0r_set_param_value() — Runtime parameter update (line 178)

Before:

case 6:
    sval = (*(char**)param);
    inst->blend_mode = (char*)realloc(inst->blend_mode, strlen(sval) + 1);
    strcpy(inst->blend_mode, sval);
    break;

After:

case 6:
    sval = (*(char**)param);
    {
        size_t slen = strlen(sval) + 1;
        char *new_mode = (char*)realloc(inst->blend_mode, slen);
        if (new_mode) {
            inst->blend_mode = new_mode;
            memcpy(inst->blend_mode, sval, slen);
        }
    }
    break;

This fix addresses multiple issues:

• realloc() result stored in a temporary: new_mode is checked before assignment, preventing the dangling pointer problem. If allocation fails, inst->blend_mode retains its old (valid) value.
• memcpy() with explicit length: The copy is bounded to exactly slen bytes—the same value used for allocation. No reliance on null-terminator scanning during the copy.
• Graceful failure: If memory allocation fails, the plugin silently keeps its previous blend mode rather than crashing or corrupting memory.

3. Regression test added

A new test file tests/test_invariant_cairoaffineblend.c was added with payloads of varying sizes (64 bytes, 128 bytes, and larger) to verify the security invariant: buffer writes never exceed the declared length.

Prevention & Best Practices

Never Use strcpy() in Security-Sensitive Code

The strcpy() function has no concept of destination buffer size. Even when you "know" the buffer is large enough, defensive coding demands explicit bounds:

// BAD: Relies on implicit assumption about buffer size
strcpy(dest, src);

// GOOD: Explicit length, single source of truth
size_t len = strlen(src) + 1;
dest = malloc(len);
memcpy(dest, src, len);

Always Check realloc() Return Values

// BAD: Loses original pointer on failure
ptr = realloc(ptr, new_size);

// GOOD: Preserves original pointer on failure
char *tmp = realloc(ptr, new_size);
if (tmp) {
    ptr = tmp;
}

Validate External Input at API Boundaries

For frei0r plugins specifically, add maximum length checks on string parameters:

#define MAX_BLEND_MODE_LEN 64

if (strlen(sval) >= MAX_BLEND_MODE_LEN) {
    return; // Reject oversized input
}

Use Compiler and Static Analysis Warnings

• Enable -Wstringop-overflow and -fortify-source=2 in GCC/Clang
• Run static analysis tools that flag strcpy() usage
• Consider using strlcpy() on platforms where it's available

Key Takeaways

• Never use strcpy() with externally-supplied parameters in frei0r plugins—the host API provides no guarantees about string lengths
• Always store realloc() results in a temporary variable—assigning directly to the original pointer causes memory leaks and dangling pointers on failure
• Calculate string lengths once and reuse the value for both allocation and copy to prevent allocation/copy size mismatches
• The frei0r f0r_set_param_value() function is an untrusted input boundary—treat all parameters from host applications as potentially malicious
• Block-scoping temporary variables (like blen and slen) prevents accidental reuse and makes the code's intent clearer

How Orbis AppSec Detected This

• Source: The param argument passed to f0r_set_param_value() via the frei0r host API — specifically (*(char**)param) when param_index == 6
• Sink: strcpy(inst->blend_mode, sval) in src/mixer2/cairoaffineblend/cairoaffineblend.c:178
• Missing control: No bounds checking on the input string length, no realloc() error handling, and use of unbounded strcpy() instead of length-limited copy
• CWE: CWE-120 (Buffer Copy without Checking Size of Input)
• Fix: Replaced strcpy() with memcpy() using pre-calculated lengths and added realloc() NULL-check to prevent heap corruption on allocation failure

Orbis AppSec automatically detected this vulnerability and opened a pull request with the fix. Try Orbis AppSec on your repositories to find and fix issues like this automatically.

Conclusion

This heap buffer overflow in cairoaffineblend.c is a textbook example of why strcpy() remains one of the most dangerous functions in C. Even when the allocation appears correct, the absence of error handling on realloc() and the use of an unbounded copy function created a critical vulnerability exploitable through the frei0r plugin API. The fix demonstrates three essential patterns for safe C string handling: calculate lengths once, check allocations before use, and copy with explicit bounds. If your codebase still uses strcpy(), treat every instance as a potential security vulnerability waiting to be exploited.

References

• CWE-120: Buffer Copy without Checking Size of Input
• OWASP Buffer Overflow Prevention
• Linux man page: memcpy(3)
• Linux man page: realloc(3)
• Semgrep rules for buffer overflow
• fix: add buffer-length check in cairoaffineblend.c