How path traversal happens in Python and how to fix it

Title: How Path Traversal Happens in Python and How to Fix It

SEO Title: posttrain_runner.py Path Traversal: Safe Fix

SEO Description: Learn how unvalidated input in check_lineage() created a path traversal vulnerability. See the os.path.realpath() fix that prevents arbitrary file reads in ML pipeline scripts.

Summary

A high-severity path traversal vulnerability in posttrain_runner.py allowed arbitrary file reads through the base_ckpt parameter. The fix implements os.path.realpath() validation to ensure all file paths remain within the working directory, preventing attackers from accessing sensitive system files.

Answer Summary

This is a path traversal vulnerability (CWE-22) in Python's posttrain_runner.py where user-controlled input (base_ckpt) was concatenated directly into a file path for open() without sanitization. The vulnerability allowed attackers to read arbitrary files using ../ sequences. The fix uses os.path.realpath() to resolve and validate that resolved paths start within os.getcwd(), rejecting any traversal attempts with sys.exit(2).

Introduction

In a machine learning pipeline repository, we discovered a HIGH severity path traversal vulnerability in skills/packs/pipeline-phase-8-post-train-runs/scripts/posttrain_runner.py at line 23. The check_lineage() function, responsible for validating model checkpoint lineage before training, accepted a base_ckpt parameter and directly concatenated it into a file path without any validation.

The vulnerable code looked innocent enough:

meta = base_ckpt + ".manifest.json"
if os.path.exists(meta):
    d = json.load(open(meta, encoding="utf-8"))

This pattern—concatenating user input into a filesystem path—is one of the most common sources of path traversal vulnerabilities. In ML pipelines where checkpoints may be specified via configuration files, CI/CD parameters, or even derived from external data sources, this creates a significant attack surface. An attacker who controls the base_ckpt value could traverse outside the intended directory and read arbitrary files on the system.

The Vulnerability Explained

The Problematic Code Pattern

The original check_lineage() function contained this vulnerable pattern:

def check_lineage(base_ckpt):
    meta = base_ckpt + ".manifest.json"  # Line 23: DANGEROUS
    if os.path.exists(meta):
        d = json.load(open(meta, encoding="utf-8"))  # Line 26: EXPLOITABLE

What's wrong here?

1. Direct concatenation: The base_ckpt parameter is concatenated directly with ".manifest.json" using string concatenation (+)
2. No path normalization: The path is not resolved to its canonical form before use
3. Unrestricted file access: Any path string is accepted, including those containing directory traversal sequences like ../../../etc/passwd

How Could This Be Exploited?

Consider an ML pipeline that loads checkpoint configurations from external sources. An attacker could provide a malicious base_ckpt value:

# Malicious input
base_ckpt = "../../../etc/passwd"

# Resulting path
meta = "../../../etc/passwd.manifest.json"

Wait—that doesn't directly hit /etc/passwd. But with more creative input:

# More dangerous input
base_ckpt = "../../../etc/passwd%00"  # Null byte injection on some systems
# Or simply:
base_ckpt = "/etc/passwd"  # Absolute path bypass

The real danger emerges when base_ckpt is derived from user-controlled configuration or when the pipeline processes checkpoints from untrusted sources. An attacker could:

1. Read sensitive configuration files: Access cloud credentials, API keys, or database connection strings stored in configuration files
2. Exfiltrate model artifacts: Read proprietary model weights or training data from other directories
3. Bypass lineage checks: By controlling which manifest file is read, an attacker could falsify lineage verification

Real-World Impact for This Application

This vulnerability exists in a post-training pipeline script that validates model checkpoint lineage. The check_lineage() function ensures that parent models passed quality gates before being used for fine-tuning. If compromised:

• Training integrity: Attackers could bypass lineage validation by providing a crafted manifest path
• Data exfiltration: Sensitive training data or model checkpoints from other pipeline stages could be accessed
• Supply chain attacks: Malicious checkpoints could be introduced into the training pipeline

The Fix

The remediation implements defense in depth with multiple security controls:

Before (Vulnerable)

def check_lineage(base_ckpt):
    meta = base_ckpt + ".manifest.json"
    if os.path.exists(meta):
        d = json.load(open(meta, encoding="utf-8"))

After (Fixed)

def check_lineage(base_ckpt):
    safe_root = os.path.realpath(os.getcwd())
    resolved = os.path.realpath(base_ckpt)
    if not resolved.startswith(safe_root + os.sep) and resolved != safe_root:
        print("REJECT: path traversal — base path must be within working directory")
        sys.exit(2)
    meta = resolved + ".manifest.json"
    if os.path.exists(meta):
        with open(meta, encoding="utf-8") as f:
            d = json.load(f)

Security Improvements Explained

The fix uses os.sep (the platform-specific path separator) rather than hardcoded / to ensure correct behavior across Windows and Unix systems. The check resolved != safe_root handles the edge case where base_ckpt points exactly to the working directory.

Prevention & Best Practices

For Python File Operations

1. Always validate paths before use
   ```python
   import os

def safe_open(user_path, safe_root):
resolved = os.path.realpath(user_path)
if not resolved.startswith(os.path.realpath(safe_root) + os.sep):
raise ValueError("Path traversal detected")
return open(resolved, 'r')
```

1. Use pathlib for modern path handling
   ```python
   from pathlib import Path

safe_root = Path.cwd().resolve()
target = (safe_root / user_input).resolve()
if not target.is_relative_to(safe_root):
raise ValueError("Traversal attempt")
```

1. Avoid string concatenation for paths
   - Use os.path.join() or pathlib.Path operators
   - Still validate after joining—os.path.join() doesn't prevent traversal!

Detection Tools

Security Standards

• CWE-22: Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal')
• OWASP Top 10 2021: A01:2021 – Broken Access Control (includes path traversal)
• CWE-73: External Control of File Name or Path

Key Takeaways

• Never concatenate user-controlled base_ckpt directly into file paths in ML pipeline scripts — always resolve and validate against a safe root directory
• os.path.realpath() is essential for path validation — it eliminates symlinks and normalizes ../ sequences that could bypass simple string checks
• The check_lineage() function now enforces a workspace boundary — all checkpoint paths must resolve within os.getcwd() or the pipeline exits with code 2
• Use explicit separator checking with os.sep — platform-aware validation prevents bypasses on Windows (\) vs. Unix (/) systems
• Fail securely with sys.exit(2) — reject traversal attempts immediately rather than attempting dangerous sanitization

How Orbis AppSec Detected This

Orbis AppSec automatically detected this vulnerability and opened a pull request with the fix. Try Orbis AppSec on your repositories to find and fix issues like this automatically.

Conclusion

Path traversal vulnerabilities remain prevalent in Python applications, particularly in data processing and ML pipeline code where file paths are frequently constructed from external input. The fix in posttrain_runner.py demonstrates that robust protection requires more than simple string checks—it demands canonical path resolution and explicit boundary validation.

By adopting os.path.realpath()-based validation as a standard pattern for all file operations, development teams can eliminate an entire class of vulnerabilities from their codebase. The key is treating all external path input as untrusted and enforcing strict workspace boundaries before any filesystem access.

References

• CWE-22: Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal'): https://cwe.mitre.org/data/definitions/22.html
• OWASP Path Traversal Prevention Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Path_Traversal_Prevention_Cheat_Sheet.html
• Python os.path.realpath() documentation: https://docs.python.org/3/library/os.path.html#os.path.realpath
• Semgrep path traversal rules: https://semgrep.dev/r?q=path-traversal
• GitHub PR: harden: add path validation in posttrain_runner.py...