Skip to content

Conversation

@som-sm
Copy link
Collaborator

@som-sm som-sm commented Oct 10, 2025

This PR introduces a lint rule that extracts example code blocks from JSDoc comments and validates them using @typescript/vfs. It then reports any errors at their corresponding locations.

The errors and their locations are identical to normal TS errors, so the experience feels just like editing a regular TS file.


image image

The lint rule surfaced around 400 errors, so this PR also fixes/improves multiple JSDoc code examples. Some common errors that it catches:

  • Missing import statements
    image

  • Floating examples
    image

  • References to hypothetical functions/variables
    image

  • Duplicate identifiers
    image

  • Syntax errors
    image

@claude

This comment was marked as outdated.

@som-sm som-sm force-pushed the feat/add-lint-rule-to-validate-jsdoc-codeblocks branch from c641bf8 to 89b14c8 Compare October 10, 2025 13:27
@claude

This comment was marked as outdated.

@som-sm
Copy link
Collaborator Author

som-sm commented Oct 10, 2025

@sindresorhus This PR is still a WIP, because there are still a lot of code blocks that have errors in them, but would appreciate if you could give an early feedback on this.

Couple of questions:

  • At several places we have code examples for internal types, like:

    /**
    Returns true if `LongString` is made up out of `Substring` repeated 0 or more times.
    @example
    ```
    ConsistsOnlyOf<'aaa', 'a'> //=> true
    ConsistsOnlyOf<'ababab', 'ab'> //=> true
    ConsistsOnlyOf<'aBa', 'a'> //=> false
    ConsistsOnlyOf<'', 'a'> //=> true
    ```
    */
    type ConsistsOnlyOf<LongString extends string, Substring extends string> =

    Now, in such cases, we can't have an import statement because internal types are not publicly available.

    So, we can either make this rule work only for exported types, or inline the examples like I've done in this change:
    image

    Or, maybe we can do something even better, wdyt?

  • At times we highlight errors in our codeblocks, but with this lint rule in place, we can't have errors in codeblocks, so I guess it's fine to add an @ts-expect-error directive in such cases. This would silence the error and also clearly state the intent that an error is indeed expected. Refer this change:

    github.com_sindresorhus_type-fest_pull_1265_files (1)
  • I think this rule makes sense and is useful, but unfortunately it takes around min and a half to run, which might be a problem. Although, the in editor DX is pretty fast, it's just when it runs on all files together.

    Screen.Recording.2025-10-10.at.6.10.33.PM.mov

    Performance Consideration: Line 68 creates a new virtual TypeScript environment for each code block. For files with many examples, this could be slow. Consider reusing the environment or batching:

    // Current approach creates new env per codeblock
    const env = createVirtualTypeScriptEnvironment(system, [FILENAME], ts, compilerOptions);

    Suggestion: Create one environment per file and update it with new content, rather than recreating for each block.

    This suggestion isn't possible because you can't update the virtual file once the virtual environment is created.

    I'm looking into ways for improving this, but let me know if you have some thoughts around this.

@claude

This comment was marked as outdated.

@claude

This comment was marked as outdated.

@claude

This comment was marked as outdated.

@claude

This comment was marked as outdated.

@som-sm
Copy link
Collaborator Author

som-sm commented Oct 10, 2025

  • but unfortunately it takes around min and a half to run, which might be a problem.

Should be fairly fast now, the env creation happens only once now!

Now, the only task left is to fix the remaining 214 errors 😪

@som-sm som-sm force-pushed the feat/add-lint-rule-to-validate-jsdoc-codeblocks branch from 186f9e2 to d974efa Compare October 10, 2025 18:23
@som-sm
Copy link
Collaborator Author

som-sm commented Oct 10, 2025

the env creation happens only once now!

No, it can't run just once.


  • but unfortunately it takes around min and a half to run, which might be a problem.

The env creation now happens per file instead of the initial per file per code block setup. So, it now takes around a min. Refer #d974efa.

@sindresorhus
Copy link
Owner

This is a very useful rule 🙌

@sindresorhus
Copy link
Owner

Now, in such cases, we can't have an import statement because internal types are not publicly available.

I think we should just ignore the internal types. Many will eventually be exposed and then they can be fixed then.

At times we highlight errors in our codeblocks, but with this lint rule in place, we can't have errors in codeblocks, so I guess it's fine to add an @ts-expect-error directive in such cases. This would silence the error and also clearly state the intent that an error is indeed expected. Refer this change:

👍

I think this rule makes sense and is useful, but unfortunately it takes around min and a half to run, which might be a problem. Although, the in editor DX is pretty fast, it's just when it runs on all files together.

Doesn't ESLint cache things? Maybe something is not working with the cache.

Alternatively, we could only run this rule in CI.

@som-sm
Copy link
Collaborator Author

som-sm commented Oct 14, 2025

I think we should just ignore the internal types. Many will eventually be exposed and then they can be fixed then.

My bad, I didn't mean types within the internal directory, I meant helper types within the same file as the exported type, like

/**
Returns true if `LongString` is made up out of `Substring` repeated 0 or more times.
@example
```
ConsistsOnlyOf<'aaa', 'a'> //=> true
ConsistsOnlyOf<'ababab', 'ab'> //=> true
ConsistsOnlyOf<'aBa', 'a'> //=> false
ConsistsOnlyOf<'', 'a'> //=> true
```
*/
type ConsistsOnlyOf<LongString extends string, Substring extends string> =

But, I guess it's fine to not lint those codeblocks, because those are anyways not publicly accessible. I'll update this.


Doesn't ESLint cache things? Maybe something is not working with the cache.

Umm… not too sure about this. Is there some built-in caching that ESLint does?


Alternatively, we could only run this rule in CI.

Umm...but it's fast enough in the editor (refer the video in #1265 (comment)). It only takes time when it has to lint all files together.

Repository owner deleted a comment from claude bot Nov 4, 2025
Repository owner deleted a comment from claude bot Nov 4, 2025
Repository owner deleted a comment from claude bot Nov 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants