Skip to content

🧯 fix: Add Pre-Parse File Size Guard to Document Parser#12275

Merged
danny-avila merged 1 commit into
devfrom
fix/document-parser-pre-parse-size-guard
Mar 17, 2026
Merged

🧯 fix: Add Pre-Parse File Size Guard to Document Parser#12275
danny-avila merged 1 commit into
devfrom
fix/document-parser-pre-parse-size-guard

Conversation

@danny-avila

@danny-avila danny-avila commented Mar 17, 2026

Copy link
Copy Markdown
Owner

Summary

Added a pre-parse file size guard to the document parser to close a memory exhaustion DoS vector between the 512 MB upload limit and the unbounded in-memory reads performed by the PDF, Word, and Excel parsers. Also refactored the MIME-type-to-parser dispatch into a dedicated helper for clarity.

  • Added DOCUMENT_PARSER_MAX_FILE_SIZE (15 MB) derived from the shared megabyte constant imported from librechat-data-provider, replacing a locally duplicated definition.
  • Enforced the size limit before any file read occurs in parseDocument, short-circuiting the new Uint8Array(readFile(...)) and equivalent allocations for oversized files.
  • Derived the limit value in the rejection error message directly from DOCUMENT_PARSER_MAX_FILE_SIZE so the message can never drift from the enforced constant.
  • Used Math.ceil for the reported file size in the error message so files just over the limit never produce a misleading "exceeds 15MB limit (15MB)" string.
  • Guarded the fs.promises.stat() fallback behind a file.path != null check to prevent a silent TypeError when file.path is undefined.
  • Extracted the MIME-type-to-parser mapping into getParserForMimeType() with a JSDoc comment consistent with the surrounding helpers.

Change Type

  • Bug fix (non-breaking change which fixes an issue)

Testing

Added unit tests to packages/api/src/files/documents/crud.spec.ts:

  • Asserts that a file with size: 16 * 1024 * 1024 is rejected with an error matching /exceeds the 15MB document parser limit \(16MB\)/.
  • Asserts that a file with size: 15 * 1024 * 1024 (exactly at the limit) resolves successfully, pinning the strictly-greater-than semantics of the guard.
  • Existing passing tests (which omit size) exercise the fs.promises.stat() fallback path on real fixture files, validating that branch in the success direction.

Test Configuration:

Run from the packages/api workspace:

cd packages/api && npx jest documents/crud

Checklist

  • My code adheres to this project's style guidelines
  • I have performed a self-review of my own code
  • My changes do not introduce new warnings
  • I have written tests demonstrating that my changes are effective or that my feature works
  • Local unit tests pass with my changes

Copilot AI review requested due to automatic review settings March 17, 2026 06:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a pre-parse file size guard (15 MB) to the document parser to prevent memory exhaustion DoS attacks. It also refactors the mime-type-to-parser dispatch into a dedicated helper function for improved readability.

Changes:

  • Adds a 15 MB file size check before reading documents into memory, throwing a descriptive error for oversized files.
  • Extracts the mime-type → parser-function mapping into getParserForMimeType() for clarity and maintainability.
  • Adds a test verifying that files exceeding the 15 MB limit are rejected.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
packages/api/src/files/documents/crud.ts Adds DOCUMENT_PARSER_MAX_FILE_SIZE constant, file size guard, and refactors parser selection into getParserForMimeType().
packages/api/src/files/documents/crud.spec.ts Adds a test case for the 15 MB size limit rejection.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Prevent memory exhaustion DoS by rejecting documents exceeding 15MB
before reading them into memory, closing the gap between the 512MB
upload limit and unbounded in-memory parsing.
@danny-avila danny-avila force-pushed the fix/document-parser-pre-parse-size-guard branch from c249659 to d16bd6b Compare March 17, 2026 06:27
@danny-avila danny-avila changed the title 🔒 fix: Add pre-parse file size guard to document parser 🧯 fix: Add Pre-Parse File Size Guard to Document Parser Mar 17, 2026
@danny-avila danny-avila merged commit 68435cd into dev Mar 17, 2026
7 checks passed
@danny-avila danny-avila deleted the fix/document-parser-pre-parse-size-guard branch March 17, 2026 06:36
jcbartle pushed a commit to jcbartle/LibreChat that referenced this pull request May 11, 2026
…12275)

Prevent memory exhaustion DoS by rejecting documents exceeding 15MB
before reading them into memory, closing the gap between the 512MB
upload limit and unbounded in-memory parsing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants