fix: use emoji regex to correctly mask pair emojis

[mhyassin]

Marked version: 17.0.1

Markdown flavor: n/a

Description

• Fixes: Bold markdown isn't parsed if there are specific trailing emojis #3852

This PR worked for me to fix the above issue, I'm not sure if I need to update more tests or whatever, I'm not even sure why it works in the Marked Demo Website but not in the codesandbox mentioned in the issue, would love if I get some help pointing out how we can get this fixed asap

Contributor

• Test(s) exist to ensure functionality and minimize regression (if no tests added, list tests covering this PR); or,
• no tests required for this PR.
• If submitting new feature, it has been documented in the appropriate places.

Committer

In most cases, this should be a different person than the contributor.

• CI is green (no forced merge required).
• Squash and Merge PR following conventional commit guidelines.

[vercel]

@mhyassin is attempting to deploy a commit to the MarkedJS Team on Vercel.

A member of the Team first needs to authorize it.

[gemini-code-assist]

Summary of Changes

Hello @mhyassin, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where emojis, particularly those composed of multiple characters, were not being correctly masked during the lexing process. By integrating the emoji-regex library, the system can now accurately determine the full length of various emoji sequences and mask them appropriately, thereby improving the robustness and correctness of the lexer's handling of emoji characters.

Highlights

• Dependency Update: The emoji-regex package (version 10.6.0) has been added as a development dependency to the project to facilitate accurate emoji matching.
• Improved Emoji Masking: The lexer now utilizes the emoji-regex library to correctly identify and mask emojis, including complex paired emojis, ensuring they are replaced with a string of 'a' characters corresponding to their precise length. This resolves an issue where emojis might have been incorrectly masked.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request correctly fixes an issue where pair emojis were being misinterpreted as markdown formatting. By using emoji-regex to mask emojis before tokenization, the parser now handles them correctly. The implementation is sound and directly addresses the problem. I have one suggestion to improve performance by creating the regex instance only once.

[UziTech]

Marked does not have any direct dependencies and we would like to keep it that way.

This would be easy to create as an extension. Just use the emStrongMask hook.

[mhyassin]

Ok thanks, the hooks works as well for me thanks will close this now