Skip to content

🔎 fix: Specify Explicit Primary Key for Meilisearch Document Operations#12542

Merged
danny-avila merged 3 commits into
devfrom
claude/wonderful-elgamal
Apr 3, 2026
Merged

🔎 fix: Specify Explicit Primary Key for Meilisearch Document Operations#12542
danny-avila merged 3 commits into
devfrom
claude/wonderful-elgamal

Conversation

@danny-avila

@danny-avila danny-avila commented Apr 3, 2026

Copy link
Copy Markdown
Owner

Summary

Fixed message search returning empty results on Meilisearch v1.0+ and resolved several pre-existing bugs in the mongoMeili Mongoose plugin discovered during review.

Meilisearch v1.0 tightened primary key inference: when a document contains multiple fields ending with id, it refuses to auto-infer and requires the primary key to be specified explicitly. The messages index contains both conversationId and messageId, causing every addDocuments / addDocumentsInBatches / updateDocuments call to silently fail with index_primary_key_multiple_candidates_found. Documents get marked as _meiliIndex: true in MongoDB but never actually reach Meilisearch, leaving message search empty while conversation search continues working.

  • Pass { primaryKey } to index.addDocumentsInBatches() in processSyncBatch (bulk sync path).
  • Pass { primaryKey } to index.addDocuments() in addObjectToMeili (per-save hook path).
  • Pass { primaryKey } to index.updateDocuments() in updateObjectToMeili (per-update hook path).
  • Fix deleteObjectFromMeili using MongoDB _id instead of the Meilisearch primary key (conversationId/messageId), which caused post-remove cleanup to silently no-op and leave orphaned documents in the index.
  • Pass options.primaryKey explicitly to the createMeiliMongooseModel factory function instead of deriving it from attributesToIndex[0] (schema field order), eliminating a fragile implicit contract.
  • Fix updateObjectToMeili skipping preprocessObjectForIndex(), which meant updates bypassed content array-to-text conversion and conversationId pipe character escaping.
  • Change collection.updateMany to collection.updateOne in addObjectToMeili since _id is unique.
  • Add primaryKey to validateOptions required keys to fail fast if omitted.
  • Strengthen test assertions to verify { primaryKey } argument is passed to addDocuments, addDocumentsInBatches, and updateDocuments. Add tests for the update path including preprocessObjectForIndex pipe escaping.

Closes #12538

Change Type

  • Bug fix (non-breaking change which fixes an issue)

Testing

All 43 tests pass in the mongoMeili.spec.ts suite.

To verify the primary fix on a live deployment:

  1. Deploy with Meilisearch v1.12+ (e.g., v1.35.1).
  2. Clear the Meilisearch messages index (or start with a fresh volume).
  3. Reset _meiliIndex flags in MongoDB: db.messages.updateMany({}, { $set: { _meiliIndex: false } }).
  4. Restart LibreChat and confirm the sync completes.
  5. Check Meilisearch tasks — addDocuments tasks should now succeed instead of failing with index_primary_key_multiple_candidates_found.
  6. Verify message search returns results in the UI.

Checklist

  • My code adheres to this project's style guidelines
  • I have performed a self-review of my own code
  • My changes do not introduce new warnings
  • I have written tests demonstrating that my changes are effective or that my feature works
  • Local unit tests pass with my changes

Copilot AI review requested due to automatic review settings April 3, 2026 13:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes MeiliSearch message indexing/search on MeiliSearch v1.0+ by explicitly passing the index primaryKey to all document mutation calls in the mongoMeili Mongoose plugin, avoiding primary-key inference failures when multiple *id fields exist (e.g., messageId + conversationId).

Changes:

  • Pass { primaryKey } to addDocumentsInBatches in the bulk sync path (processSyncBatch).
  • Pass { primaryKey } to addDocuments (post-save hook) and updateDocuments (post-update hook).
  • Replace this.collection.updateMany with Model.updateMany(..., { timestamps: false }) to avoid raw driver calls that bypass Mongoose middleware (tenant isolation).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 256 to +258
try {
// Add documents to MeiliSearch
await index.addDocumentsInBatches(formattedDocs);
await index.addDocumentsInBatches(formattedDocs, undefined, { primaryKey });

Copilot AI Apr 3, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test coverage: the new behavior relies on passing { primaryKey } into Meili document mutation calls. The existing mongoMeili.spec.ts assertions only check that addDocuments*/updateDocuments were called, so this regression could reappear without failing tests. Please extend the spec(s) to assert that addDocumentsInBatches, addDocuments, and updateDocuments are invoked with the expected primaryKey option (e.g., messageId for messages, conversationId for convos).

Copilot uses AI. Check for mistakes.
@danny-avila

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila danny-avila changed the base branch from main to dev April 3, 2026 18:09
…ents calls

Meilisearch v1.0+ refuses to auto-infer the primary key when a document
contains multiple fields ending with 'id'. The messages index has both
conversationId and messageId, causing addDocuments to silently fail with
index_primary_key_multiple_candidates_found, leaving message search empty.

Pass { primaryKey } to addDocumentsInBatches, addDocuments, and
updateDocuments — the variable was already in scope.

Also replace raw this.collection.updateMany with Mongoose Model.updateMany
to satisfy the no-restricted-syntax ESLint rule (tenant isolation guard).

Closes #12538
Address review findings from PR #12542:

- Fix deleteObjectFromMeili using MongoDB _id instead of the Meilisearch
  primary key (conversationId/messageId), causing post-remove cleanup to
  silently no-op and leave orphaned documents in the index.

- Pass options.primaryKey explicitly to createMeiliMongooseModel factory
  instead of deriving it from attributesToIndex[0] (schema field order),
  eliminating a fragile implicit contract.

- Fix updateObjectToMeili skipping preprocessObjectForIndex, which meant
  updates bypassed content array-to-text conversion and conversationId
  pipe character escaping.

- Change collection.updateMany to collection.updateOne in addObjectToMeili
  since _id is unique (semantic correctness).

- Add primaryKey to validateOptions required keys.

- Strengthen test assertions to verify { primaryKey } argument is passed
  to addDocuments, addDocumentsInBatches, and updateDocuments. Add tests
  for the update path including preprocessObjectForIndex pipe escaping.
Address follow-up review findings:

- Add test for deleteObjectFromMeili verifying it uses messageId (not
  MongoDB _id) when calling index.deleteDocument, guarding against
  regression of the silent orphaned-document bug.

- Add test for message model update path asserting { primaryKey:
  'messageId' } is passed to updateDocuments (previously only the
  conversation model update path was tested).

- Add @param config.primaryKey to createMeiliMongooseModel JSDoc.
@danny-avila danny-avila force-pushed the claude/wonderful-elgamal branch from 92e5c11 to 3e6f90e Compare April 3, 2026 21:48
@danny-avila danny-avila merged commit 33ee7de into dev Apr 3, 2026
8 checks passed
@danny-avila danny-avila deleted the claude/wonderful-elgamal branch April 3, 2026 22:01
yidianyiko pushed a commit to yidianyiko/LibreChat that referenced this pull request Apr 13, 2026
…ns (danny-avila#12542)

* fix: pass explicit primaryKey to Meilisearch addDocuments/updateDocuments calls

Meilisearch v1.0+ refuses to auto-infer the primary key when a document
contains multiple fields ending with 'id'. The messages index has both
conversationId and messageId, causing addDocuments to silently fail with
index_primary_key_multiple_candidates_found, leaving message search empty.

Pass { primaryKey } to addDocumentsInBatches, addDocuments, and
updateDocuments — the variable was already in scope.

Also replace raw this.collection.updateMany with Mongoose Model.updateMany
to satisfy the no-restricted-syntax ESLint rule (tenant isolation guard).

Closes danny-avila#12538

* fix: resolve additional Meilisearch plugin bugs found in review

Address review findings from PR danny-avila#12542:

- Fix deleteObjectFromMeili using MongoDB _id instead of the Meilisearch
  primary key (conversationId/messageId), causing post-remove cleanup to
  silently no-op and leave orphaned documents in the index.

- Pass options.primaryKey explicitly to createMeiliMongooseModel factory
  instead of deriving it from attributesToIndex[0] (schema field order),
  eliminating a fragile implicit contract.

- Fix updateObjectToMeili skipping preprocessObjectForIndex, which meant
  updates bypassed content array-to-text conversion and conversationId
  pipe character escaping.

- Change collection.updateMany to collection.updateOne in addObjectToMeili
  since _id is unique (semantic correctness).

- Add primaryKey to validateOptions required keys.

- Strengthen test assertions to verify { primaryKey } argument is passed
  to addDocuments, addDocumentsInBatches, and updateDocuments. Add tests
  for the update path including preprocessObjectForIndex pipe escaping.

* fix: add regression tests for delete and message update paths

Address follow-up review findings:

- Add test for deleteObjectFromMeili verifying it uses messageId (not
  MongoDB _id) when calling index.deleteDocument, guarding against
  regression of the silent orphaned-document bug.

- Add test for message model update path asserting { primaryKey:
  'messageId' } is passed to updateDocuments (previously only the
  conversation model update path was tested).

- Add @param config.primaryKey to createMeiliMongooseModel JSDoc.
jcbartle pushed a commit to jcbartle/LibreChat that referenced this pull request May 11, 2026
…ns (danny-avila#12542)

* fix: pass explicit primaryKey to Meilisearch addDocuments/updateDocuments calls

Meilisearch v1.0+ refuses to auto-infer the primary key when a document
contains multiple fields ending with 'id'. The messages index has both
conversationId and messageId, causing addDocuments to silently fail with
index_primary_key_multiple_candidates_found, leaving message search empty.

Pass { primaryKey } to addDocumentsInBatches, addDocuments, and
updateDocuments — the variable was already in scope.

Also replace raw this.collection.updateMany with Mongoose Model.updateMany
to satisfy the no-restricted-syntax ESLint rule (tenant isolation guard).

Closes danny-avila#12538

* fix: resolve additional Meilisearch plugin bugs found in review

Address review findings from PR danny-avila#12542:

- Fix deleteObjectFromMeili using MongoDB _id instead of the Meilisearch
  primary key (conversationId/messageId), causing post-remove cleanup to
  silently no-op and leave orphaned documents in the index.

- Pass options.primaryKey explicitly to createMeiliMongooseModel factory
  instead of deriving it from attributesToIndex[0] (schema field order),
  eliminating a fragile implicit contract.

- Fix updateObjectToMeili skipping preprocessObjectForIndex, which meant
  updates bypassed content array-to-text conversion and conversationId
  pipe character escaping.

- Change collection.updateMany to collection.updateOne in addObjectToMeili
  since _id is unique (semantic correctness).

- Add primaryKey to validateOptions required keys.

- Strengthen test assertions to verify { primaryKey } argument is passed
  to addDocuments, addDocumentsInBatches, and updateDocuments. Add tests
  for the update path including preprocessObjectForIndex pipe escaping.

* fix: add regression tests for delete and message update paths

Address follow-up review findings:

- Add test for deleteObjectFromMeili verifying it uses messageId (not
  MongoDB _id) when calling index.deleteDocument, guarding against
  regression of the silent orphaned-document bug.

- Add test for message model update path asserting { primaryKey:
  'messageId' } is passed to updateDocuments (previously only the
  conversation model update path was tested).

- Add @param config.primaryKey to createMeiliMongooseModel JSDoc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Message search broken on Meilisearch v1.12+ — primary key not specified on addDocuments calls

2 participants