Skip to content

Conversation

@ixchio
Copy link
Contributor

@ixchio ixchio commented Dec 6, 2025

Changes

Fixes #14893

  • Replaced O(n²) object spread pattern with direct mutation in generateLookupMap
  • For each content/data entry, the code was creating new objects via {...lookupMap[collection]?.entries, [key]: value}
  • With 10,000+ entries, this caused ~50 million string copies (n × n/2)
  • Now uses nullish coalescing (??=) and direct property assignment - O(n) complexity

Before:

lookupMap[collection] = {
  type: 'content',
  entries: {
    ...lookupMap[collection]?.entries, // ← Creates new object each time
    [slug]: rootRelativePath(root, filePath),
  },
};

After

lookupMap[collection] ??= { type: 'content', entries: {} };
lookupMap[collection].entries[slug] = rootRelativePath(root, filePath);

Testing
All 1326 existing tests pass (3 unrelated failures: clipboard + network flakes)
All content-related tests pass: Content Collections, references, render(), Intellisense

Docs
No docs changes needed - this is an internal performance optimization with no API changes.

@changeset-bot
Copy link

changeset-bot bot commented Dec 6, 2025

🦋 Changeset detected

Latest commit: b90634a

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions bot added the pkg: astro Related to the core `astro` package (scope) label Dec 6, 2025
@ematipico ematipico added the pr preview Apply this label to a PR to generate a preview release label Dec 6, 2025
@ematipico
Copy link
Member

ematipico commented Dec 6, 2025

I'm going to create a preview release so we can test if this change will provide benefits

@pkg-pr-new
Copy link

pkg-pr-new bot commented Dec 6, 2025

npm i https://pkg.pr.new/astro@14973
npm i https://pkg.pr.new/@astrojs/svelte@14973
npm i https://pkg.pr.new/@astrojs/markdown-remark@14973

commit: b90634a

@codspeed-hq
Copy link

codspeed-hq bot commented Dec 6, 2025

CodSpeed Performance Report

Merging #14973 will not alter performance

Comparing amankumarpandeyin:fix/lookup-map-oom (b90634a) with main (ef53716)1

Summary

✅ 6 untouched

Footnotes

  1. No successful run was found on main (4264a36) during the generation of this report, so ef53716 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@matthewp
Copy link
Contributor

matthewp commented Dec 6, 2025

Wonderful @amankumarpandeyin thank you! Do you have any anecdotal data, did this improve your build performance for example?

@ixchio
Copy link
Contributor Author

ixchio commented Dec 6, 2025

Wonderful @amankumarpandeyin thank you! Do you have any anecdotal data, did this improve your build performance for example?

@matthewp thanks! i was experiencing OOM errors with large content collections (10k+ entries)... after this change builds complete without memory issues... the fix eliminates the quadratic object copying that was causing 50M string copies for that scale

@matthewp matthewp merged commit ed7cc2f into withastro:main Dec 9, 2025
28 checks passed
@astrobot-houston astrobot-houston mentioned this pull request Dec 9, 2025
dadezzz pushed a commit to dadezzz/ice-notes that referenced this pull request Dec 14, 2025
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [astro](https://astro.build) ([source](https://github.com/withastro/astro/tree/HEAD/packages/astro)) | [`5.16.4` -> `5.16.5`](https://renovatebot.com/diffs/npm/astro/5.16.4/5.16.5) | ![age](https://developer.mend.io/api/mc/badges/age/npm/astro/5.16.5?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/npm/astro/5.16.4/5.16.5?slim=true) |

---

### Release Notes

<details>
<summary>withastro/astro (astro)</summary>

### [`v5.16.5`](https://github.com/withastro/astro/blob/HEAD/packages/astro/CHANGELOG.md#5165)

[Compare Source](https://github.com/withastro/astro/compare/[email protected]@5.16.5)

##### Patch Changes

- [#&#8203;14985](withastro/astro#14985) [`c016f10`](withastro/astro@c016f10) Thanks [@&#8203;florian-lefebvre](https://github.com/florian-lefebvre)! - Fixes a case where JSDoc annotations wouldn't show for fonts related APIs in the Astro config

- [#&#8203;14973](withastro/astro#14973) [`ed7cc2f`](withastro/astro@ed7cc2f) Thanks [@&#8203;amankumarpandeyin](https://github.com/amankumarpandeyin)! - Fixes performance regression and OOM errors when building medium-sized blogs with many content entries. Replaced O(n²) object spread pattern with direct mutation in `generateLookupMap`.

- [#&#8203;14958](withastro/astro#14958) [`70eb542`](withastro/astro@70eb542) Thanks [@&#8203;ascorbic](https://github.com/ascorbic)! - Gives a helpful error message if a user sets `output: "hybrid"` in their Astro config.

  The option was removed in Astro 5, but lots of content online still references it, and LLMs often suggest it. It's not always clear that the replacement is `output: "static"`, rather than `output: "server"`. This change adds a helpful error message to guide humans and robots.

- [#&#8203;14901](withastro/astro#14901) [`ef53716`](withastro/astro@ef53716) Thanks [@&#8203;Darknab](https://github.com/Darknab)! - Updates the `glob()` loader to log a warning when duplicated IDs are detected

- Updated dependencies \[[`d8305f8`](withastro/astro@d8305f8)]:
  - [@&#8203;astrojs/markdown-remark](https://github.com/astrojs/markdown-remark)@&#8203;6.3.10

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0Mi4zOS4yIiwidXBkYXRlZEluVmVyIjoiNDIuMzkuMiIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->

Co-authored-by: Renovate Bot <[email protected]>
Co-committed-by: Renovate Bot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pkg: astro Related to the core `astro` package (scope) pr preview Apply this label to a PR to generate a preview release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance regression & OOM errors when building medium sized blogs with finite resources

3 participants