Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
164 commits
Select commit Hold shift + click to select a range
80cb760
implement URLPattern skeleton
anonrig Nov 30, 2024
fc4b193
use correct value for clang-format
anonrig Nov 30, 2024
cd9df5e
fix build errors
anonrig Nov 30, 2024
09c3f42
create url_pattern-inl.h
anonrig Nov 30, 2024
6656757
add canonicalize methods
anonrig Dec 4, 2024
686af7d
add ada::parse_url_pattern function
anonrig Dec 4, 2024
3c08805
add more comments
anonrig Dec 4, 2024
66da95c
implement getters
anonrig Dec 4, 2024
8377009
add has_regexp_groups()
anonrig Dec 4, 2024
61f4b67
start implementing tokenizer & tokenize
anonrig Dec 4, 2024
fe3af13
add initial parser_url_pattern method
anonrig Dec 4, 2024
1262f8c
add todos and remove redundant qualifiers
anonrig Dec 4, 2024
20691e3
implement escape pattern
anonrig Dec 4, 2024
eed6d80
add CompileComponentOptions
anonrig Dec 7, 2024
f153e00
minor fixes for add-url-pattern (#800)
lemire Dec 8, 2024
7dbf175
rename commits
anonrig Dec 8, 2024
5f36b46
add more parse_url_pattern
anonrig Dec 9, 2024
4619c9d
rename url_pattern class
anonrig Dec 9, 2024
5cdb6db
complete parse_url_pattern implementation
anonrig Dec 9, 2024
5f28a37
add `_component` suffix to components
anonrig Dec 9, 2024
20d7529
remove unnecessary void
anonrig Dec 9, 2024
16aace3
implement generate regular expression methods
anonrig Dec 10, 2024
14d217c
continue working on parser
anonrig Dec 10, 2024
4a31b3f
fix build error
anonrig Dec 12, 2024
f10d3b2
implement constructor string parser
anonrig Dec 12, 2024
4c20080
implement all of tokenizer's functions
anonrig Dec 12, 2024
74f72fd
fix build errors
anonrig Dec 12, 2024
969c87a
fix warnings
anonrig Dec 12, 2024
6276ce8
complete tokenizer
anonrig Dec 12, 2024
5f02b24
implement escape_regexp_string
anonrig Dec 12, 2024
f55e3f5
implement generate_pattern_string
anonrig Dec 12, 2024
cc73e97
fix compiler warnings
anonrig Dec 12, 2024
ca60161
semi-implement match
anonrig Dec 13, 2024
1a47532
complete one more todo
anonrig Dec 13, 2024
a67fe01
simplify create_component_match_result
anonrig Dec 13, 2024
37dc747
simplify
anonrig Dec 13, 2024
d4d843d
use correct inputs for match/exec/test
anonrig Dec 13, 2024
5c212d7
rename wpt_tests to wpt_url_tests
anonrig Dec 13, 2024
d33f228
add wpt_urlpattern_tests skeleton
anonrig Dec 13, 2024
530deb4
add first test
anonrig Dec 14, 2024
f1e04ce
Build fixes (#801)
lemire Dec 14, 2024
a10ba16
fix 2 bugs
anonrig Dec 14, 2024
b67580d
fix linter issues
anonrig Dec 14, 2024
42d6c32
fix 2 more bugs
anonrig Dec 14, 2024
8d8acb2
more progress on missing features
anonrig Dec 16, 2024
ac0817e
move url_pattern_helpers to separate file
anonrig Dec 16, 2024
8523594
fix build errors
anonrig Dec 16, 2024
096e159
use url_pattern_encoding_callback
anonrig Dec 17, 2024
f711faf
fix url pattern constructor error
anonrig Dec 17, 2024
fc5b020
fix more issues
anonrig Dec 17, 2024
690a14a
add initial version of wpt test runner
anonrig Dec 17, 2024
4f1dc9b
simplify json logic (#802)
lemire Dec 17, 2024
21109d1
add fuzzer
anonrig Dec 17, 2024
8929462
removing the reset
lemire Dec 17, 2024
6759d37
update ada idna
anonrig Dec 18, 2024
e8897d9
use ada idna method for valid name code point
anonrig Dec 18, 2024
4e96bbe
fix add part implementation
anonrig Dec 18, 2024
43c806d
fix invalid access errors
anonrig Dec 18, 2024
029e17f
implement tests correctly
anonrig Dec 18, 2024
c4c373b
improve test runner
anonrig Dec 18, 2024
4b3f34d
add url_pattern_init to_string() method
anonrig Dec 19, 2024
8d4994c
update WPT tests
anonrig Dec 19, 2024
5e6f934
fix last remaining todo
anonrig Dec 19, 2024
71468e2
simplify test runner
anonrig Dec 19, 2024
6a4c9a5
minor fixes
lemire Dec 19, 2024
fd6d1d4
some reworking
lemire Dec 19, 2024
7dca1de
make sure to skip invalid tests
anonrig Dec 19, 2024
6d38085
remove std::ranges::iota due to clang
anonrig Dec 20, 2024
abb2af0
add more fuzzing coverage
anonrig Dec 20, 2024
a0df533
try to fix windows issues
anonrig Dec 20, 2024
aeb4699
remove unnecessary copy
anonrig Dec 20, 2024
1eeab05
start testing the validity of the correct responses
anonrig Dec 20, 2024
208c2ff
fix couple of bugs
anonrig Dec 20, 2024
664ed1c
fix invalid ascii checks
anonrig Dec 20, 2024
60c4015
make pattern generation more verbose
anonrig Dec 20, 2024
5e989f0
fix regex error
anonrig Dec 20, 2024
5539349
remove semicolon due to -Werror,-Wextra-semi
anonrig Dec 20, 2024
04252cd
guarding regex call (#805)
lemire Dec 20, 2024
3eac233
add more logging
anonrig Dec 23, 2024
3f7536c
change ada_idna to char32_t
anonrig Dec 23, 2024
602a565
remove try/catch
anonrig Dec 23, 2024
fc3e76e
make canonicalize_ methods more flexible
anonrig Dec 23, 2024
9407a49
fix change_state
anonrig Dec 23, 2024
6d8e960
fix invalid substr call
anonrig Dec 23, 2024
67fb323
fix generate_pattern_string impl
anonrig Dec 23, 2024
dbd003d
fix more small issues
anonrig Dec 23, 2024
8619179
improve url_pattern_init::process
anonrig Dec 23, 2024
a4f0c42
correctly computing the next code point (#808)
lemire Dec 23, 2024
099fb43
adding checks
lemire Dec 23, 2024
049dd11
use std string view to avoid copy
anonrig Dec 23, 2024
6b29fed
use next_index instead of index
anonrig Dec 23, 2024
61f45be
highlight the error message
anonrig Dec 23, 2024
d2bcf67
better decoding
lemire Dec 23, 2024
e997a28
I think that the test is in error (#810)
lemire Dec 23, 2024
6e96857
remove invalid WPT test data
anonrig Dec 24, 2024
188e171
remove invalid assertion
anonrig Dec 24, 2024
5682bf1
fix ipv6 address canonicalize
anonrig Dec 24, 2024
67f9708
fix canonicalize_ipv6_hostname
anonrig Dec 24, 2024
681bf67
simplify test runner
anonrig Dec 24, 2024
3304dd0
fix test runner
anonrig Dec 24, 2024
40f85e3
add a todo
anonrig Dec 24, 2024
fdb044e
remove invalid test case
anonrig Dec 24, 2024
8ee26f4
add tests for expected object
anonrig Dec 24, 2024
7f4acf2
fix hostname tests
anonrig Dec 25, 2024
505f526
complete match implementation
anonrig Dec 25, 2024
6f284c4
fix empty component tests
anonrig Dec 26, 2024
d928625
revert some wpt changes
anonrig Dec 26, 2024
64c6968
add some optional result logging (#812)
lemire Dec 26, 2024
8090940
lint
lemire Dec 26, 2024
f204a8c
fixing logging
lemire Dec 26, 2024
d7b92eb
removing diagram printout
Dec 27, 2024
fc884cb
fix asan build errors
anonrig Dec 28, 2024
77f44d3
simpler version of the yagiz/add-url-pattern branch (#815)
lemire Dec 28, 2024
ab71fa0
simplify implementation
anonrig Dec 29, 2024
ca66004
improve url_pattern_part emplace_back calls
anonrig Dec 29, 2024
b2d9e70
fix url_pattern_component constructor
anonrig Dec 31, 2024
baeafc6
remove the usage of ada.h inside src
anonrig Dec 31, 2024
487582d
move all helper methods to url_pattern.cpp
anonrig Dec 31, 2024
ffee76c
fix urlpatterntestdata.json
anonrig Dec 31, 2024
0100006
fix build errors
anonrig Dec 31, 2024
757683b
add missing check
anonrig Dec 31, 2024
8dc937e
more tests (#817)
lemire Dec 31, 2024
53ba80f
fix assertion error
anonrig Dec 31, 2024
edbf6c0
don't move function calls
anonrig Dec 31, 2024
5f74dd3
fix token reference asan error
anonrig Dec 31, 2024
a5580c7
another test (#818)
lemire Dec 31, 2024
db7acf9
simplify parser and tests
anonrig Jan 1, 2025
c60c2dc
remove unnecessary duplicate_name method
anonrig Jan 1, 2025
385f554
convert Token to class
anonrig Jan 1, 2025
dab41f6
minor cleanups
anonrig Jan 1, 2025
cf69585
remove invalid std::move
anonrig Jan 1, 2025
bd9655d
simplify parser
anonrig Jan 1, 2025
393f515
remove invalid pathname WPT
anonrig Jan 1, 2025
64f66c6
leave some todos for WPT
anonrig Jan 1, 2025
1f563d4
complete inputs parsing
anonrig Jan 1, 2025
52c33b5
removed duplicated code
anonrig Jan 3, 2025
6ae710b
merge error enums
anonrig Jan 3, 2025
1b59155
fix a boolean operation
anonrig Jan 3, 2025
dd20066
update urlpatterntestdata.json
anonrig Jan 3, 2025
528027c
remove unnecessary assertions
anonrig Jan 3, 2025
65fe0b6
removing GLIBCXX debug
Jan 3, 2025
613d60d
updating macos ci
Jan 3, 2025
943f0aa
indent
Jan 3, 2025
9bb11ad
keeping only static
Jan 3, 2025
5b1de58
improve wpt runner
anonrig Jan 3, 2025
1ec8ea0
fix match
anonrig Jan 3, 2025
c858831
add assertions for object return
anonrig Jan 4, 2025
36a7b72
check __cpp_lib_format
lemire Jan 4, 2025
ff2bf00
adding version header (#824)
lemire Jan 4, 2025
0feb9a6
fix match related bugs
anonrig Jan 5, 2025
57accd5
fix port canonicalize
anonrig Jan 5, 2025
67f9988
fix port setting caused by url parser bug
anonrig Jan 5, 2025
9deaa41
add temporary check for special schemes
anonrig Jan 5, 2025
d47ca13
revert opaque host change
anonrig Jan 6, 2025
14e6c53
fix match when input needs to be parsed
anonrig Jan 6, 2025
6f2838f
fix match hash and search prefix
anonrig Jan 6, 2025
89c8bea
fix internal assertion
anonrig Jan 6, 2025
e3f4fe2
improve wpt test runner
anonrig Jan 6, 2025
8b8d5e6
improve regexp matching
anonrig Jan 7, 2025
a47d8c5
fix wpt testrunner
anonrig Jan 7, 2025
b620b09
fix test implementation
anonrig Jan 7, 2025
36a9097
add half-working match_result
anonrig Jan 7, 2025
87def0a
improve regex matching
anonrig Jan 8, 2025
61728b2
remove invalid WPT test
anonrig Jan 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix url pattern constructor error
  • Loading branch information
anonrig committed Jan 8, 2025
commit f711faf1d4f9f62209cc0478245fdb8f42d8184a
7 changes: 7 additions & 0 deletions include/ada/unicode.h
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,13 @@ ada_really_inline constexpr bool is_alnum_plus(char c) noexcept;
*/
ada_really_inline constexpr bool is_ascii_hex_digit(char c) noexcept;

/**
* @private
* @details If a char is between U+0000 and U+007F inclusive, then it's an ASCII
* character.
*/
ada_really_inline constexpr bool is_ascii(uint16_t c) noexcept;

/**
* @private
* Checks if the input is a C0 control or space character.
Expand Down
4 changes: 2 additions & 2 deletions include/ada/url_pattern_helpers-inl.h
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,7 @@ inline bool constructor_string_parser::is_port_prefix() {
inline void Tokenizer::get_next_code_point() {
// Set tokenizer’s code point to the Unicode code point in tokenizer’s input
// at the position indicated by tokenizer’s next index.
code_point = &input[next_index];
code_point = input[next_index];
// Increment tokenizer’s next index by 1.
next_index++;
}
Expand Down Expand Up @@ -321,7 +321,7 @@ Tokenizer::process_tokenizing_error(size_t next_position,
}

// @see https://urlpattern.spec.whatwg.org/#is-a-valid-name-code-point
inline bool is_valid_name_code_point(char cp, bool first) {
inline bool is_valid_name_code_point(uint16_t cp, bool first) {
// If first is true return the result of checking if code point is contained
// in the IdentifierStart set of code points. Otherwise return the result of
// checking if code point is contained in the IdentifierPart set of code
Expand Down
4 changes: 2 additions & 2 deletions include/ada/url_pattern_helpers.h
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ class Tokenizer {
// has an associated next index, a number, initially 0.
size_t next_index = 0;
// has an associated code point, a Unicode code point, initially null.
std::string_view code_point{};
uint16_t code_point{};
};

// @see https://urlpattern.spec.whatwg.org/#constructor-string-parser
Expand Down Expand Up @@ -333,7 +333,7 @@ std::string generate_segment_wildcard_regexp(
url_pattern_compile_component_options options);

// @see https://urlpattern.spec.whatwg.org/#is-a-valid-name-code-point
bool is_valid_name_code_point(char code_point, bool first);
bool is_valid_name_code_point(uint16_t code_point, bool first);

} // namespace ada::url_pattern_helpers

Expand Down
39 changes: 21 additions & 18 deletions src/parser.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -911,26 +911,26 @@ tl::expected<url_pattern, url_pattern_errors> parse_url_pattern_impl(
// Set init to the result of running parse a constructor string given input.
auto parse_result = url_pattern_helpers::constructor_string_parser::parse(
std::get<std::string_view>(input));
if (!parse_result.has_value()) {
if (!parse_result) {
return tl::unexpected(parse_result.error());
}
init = *parse_result;

// If baseURL is null and init["protocol"] does not exist, then throw a
// TypeError.
if (base_url == nullptr && !init.protocol.has_value()) {
if (!base_url && !init.protocol) {
return tl::unexpected(url_pattern_errors::type_error);
}

// If baseURL is not null, set init["baseURL"] to baseURL.
if (base_url != nullptr) {
if (base_url) {
init.base_url = std::string(*base_url);
}
} else {
// Assert: input is a URLPatternInit.
ADA_ASSERT_TRUE(std::holds_alternative<url_pattern_init>(input));
// If baseURL is not null, then throw a TypeError.
if (base_url != nullptr) {
if (base_url) {
return tl::unexpected(url_pattern_errors::type_error);
}
// Optimization: Avoid copy by moving the input value.
Expand All @@ -950,6 +950,7 @@ tl::expected<url_pattern, url_pattern_errors> parse_url_pattern_impl(
// For each componentName of « "protocol", "username", "password", "hostname",
// "port", "pathname", "search", "hash" If processedInit[componentName] does
// not exist, then set processedInit[componentName] to "*".
ADA_ASSERT_TRUE(processed_init.has_value());
if (!processed_init->protocol) processed_init->protocol = "*";
if (!processed_init->username) processed_init->username = "*";
if (!processed_init->username) processed_init->username = "*";
Expand All @@ -963,26 +964,16 @@ tl::expected<url_pattern, url_pattern_errors> parse_url_pattern_impl(
// If processedInit["protocol"] is a special scheme and processedInit["port"]
// is a string which represents its corresponding default port in radix-10
// using ASCII digits then set processedInit["port"] to the empty string.
if (scheme::is_special(*processed_init->protocol)) {
// TODO: Implement this.
// TODO: Optimization opportunity.
if (scheme::is_special(*processed_init->protocol) &&
std::to_string(scheme::get_special_port(*processed_init->protocol)) ==
processed_init->port) {
processed_init->port = "";
}

// Let urlPattern be a new URL pattern.
auto url_pattern_ = url_pattern{};

// Set urlPattern’s username component to the result of compiling a component
// given processedInit["username"], canonicalize a username, and default
// options.
auto username_component = url_pattern_component::compile(
processed_init->username.value(),
url_pattern_helpers::canonicalize_username,
url_pattern_compile_component_options::DEFAULT);
if (!username_component) {
return tl::unexpected(username_component.error());
}
url_pattern_.username_component = std::move(*username_component);

// Set urlPattern’s protocol component to the result of compiling a component
// given processedInit["protocol"], canonicalize a protocol, and default
// options.
Expand All @@ -995,6 +986,18 @@ tl::expected<url_pattern, url_pattern_errors> parse_url_pattern_impl(
}
url_pattern_.protocol_component = std::move(*protocol_component);

// Set urlPattern’s username component to the result of compiling a component
// given processedInit["username"], canonicalize a username, and default
// options.
auto username_component = url_pattern_component::compile(
processed_init->username.value(),
url_pattern_helpers::canonicalize_username,
url_pattern_compile_component_options::DEFAULT);
if (!username_component) {
return tl::unexpected(username_component.error());
}
url_pattern_.username_component = std::move(*username_component);

// Set urlPattern’s password component to the result of compiling a component
// given processedInit["password"], canonicalize a password, and default
// options.
Expand Down
5 changes: 5 additions & 0 deletions src/unicode.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,11 @@ ada_really_inline constexpr bool is_ascii_hex_digit(const char c) noexcept {
(c >= 'a' && c <= 'f');
}

ada_really_inline constexpr bool is_ascii(const uint16_t c) noexcept {
// If code point is between U+0000 and U+007F inclusive, then return true.
return c <= 0x7F;
}

ada_really_inline constexpr bool is_c0_control_or_space(const char c) noexcept {
return (unsigned char)c <= ' ';
}
Expand Down
34 changes: 17 additions & 17 deletions src/url_pattern_helpers.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -504,23 +504,23 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(
tokenizer.seek_and_get_next_code_point(tokenizer.index);

// If tokenizer’s code point is U+002A (*):
if (tokenizer.code_point == "*") {
if (tokenizer.code_point == '*') {
// Run add a token with default position and length given tokenizer and
// "asterisk".
tokenizer.add_token_with_defaults(token_type::ASTERISK);
continue;
}

// If tokenizer’s code point is U+002B (+) or U+003F (?):
if (tokenizer.code_point == "+" || tokenizer.code_point == "?") {
if (tokenizer.code_point == '+' || tokenizer.code_point == '?') {
// Run add a token with default position and length given tokenizer and
// "other-modifier".
tokenizer.add_token_with_defaults(token_type::OTHER_MODIFIER);
continue;
}

// If tokenizer’s code point is U+005C (\):
if (tokenizer.code_point == "\\") {
if (tokenizer.code_point == '\\') {
// If tokenizer’s index is equal to tokenizer’s input's code point length
// − 1:
if (tokenizer.index == tokenizer.input.size() - 1) {
Expand All @@ -546,23 +546,23 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(
}

// If tokenizer’s code point is U+007B ({):
if (tokenizer.code_point == "{") {
if (tokenizer.code_point == '{') {
// Run add a token with default position and length given tokenizer and
// "open".
tokenizer.add_token_with_defaults(token_type::OPEN);
continue;
}

// If tokenizer’s code point is U+007D (}):
if (tokenizer.code_point == "}") {
if (tokenizer.code_point == '}') {
// Run add a token with default position and length given tokenizer and
// "close".
tokenizer.add_token_with_defaults(token_type::CLOSE);
continue;
}

// If tokenizer’s code point is U+003A (:):
if (tokenizer.code_point == ":") {
if (tokenizer.code_point == ':') {
// Let name position be tokenizer’s next index.
auto name_position = tokenizer.next_index;
// Let name start be name position.
Expand All @@ -577,8 +577,8 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(
bool first_code_point = name_position == name_start;
// Let valid code point be the result of running is a valid name code
// point given tokenizer’s code point and first code point.
auto valid_code_point = is_valid_name_code_point(
tokenizer.code_point.at(0), first_code_point);
auto valid_code_point =
is_valid_name_code_point(tokenizer.code_point, first_code_point);
// If valid code point is false break.
if (!valid_code_point) break;
// Set name position to tokenizer’s next index.
Expand All @@ -603,7 +603,7 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(
}

// If tokenizer’s code point is U+0028 (():
if (tokenizer.code_point == "(") {
if (tokenizer.code_point == '(') {
// Let depth be 1.
size_t depth = 1;
// Let regexp position be tokenizer’s next index.
Expand All @@ -622,8 +622,8 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(

// TODO: Optimization opportunity: The next 2 if statements can be
// merged. If the result of running is ASCII given tokenizer’s code
// point is false:
if (!idna::is_ascii(tokenizer.code_point)) {
// point is false:i
if (!unicode::is_ascii(tokenizer.code_point)) {
// Run process a tokenizing error given tokenizer, regexp start, and
// tokenizer’s index.
if (auto process_error = tokenizer.process_tokenizing_error(
Expand All @@ -638,7 +638,7 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(

// If regexp position equals regexp start and tokenizer’s code point is
// U+003F (?):
if (regexp_position == regexp_start && tokenizer.code_point == "?") {
if (regexp_position == regexp_start && tokenizer.code_point == '?') {
// Run process a tokenizing error given tokenizer, regexp start, and
// tokenizer’s index.
if (auto process_error = tokenizer.process_tokenizing_error(
Expand All @@ -652,7 +652,7 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(
}

// If tokenizer’s code point is U+005C (\):
if (tokenizer.code_point == "\\") {
if (tokenizer.code_point == '\\') {
// If regexp position equals tokenizer’s input's code point length − 1
if (regexp_position == tokenizer.input.size() - 1) {
// Run process a tokenizing error given tokenizer, regexp start, and
Expand All @@ -670,7 +670,7 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(
tokenizer.get_next_code_point();
// If the result of running is ASCII given tokenizer’s code point is
// false:
if (!idna::is_ascii(tokenizer.code_point)) {
if (!unicode::is_ascii(tokenizer.code_point)) {
// Run process a tokenizing error given tokenizer, regexp start, and
// tokenizer’s index.
if (auto process_error = tokenizer.process_tokenizing_error(
Expand All @@ -688,15 +688,15 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(
}

// If tokenizer’s code point is U+0029 ()):
if (tokenizer.code_point == ")") {
if (tokenizer.code_point == ')') {
// Decrement depth by 1.
depth--;
if (depth == 0) {
// Set regexp position to tokenizer’s next index.
regexp_position = tokenizer.next_index;
break;
}
} else if (tokenizer.code_point == "(") {
} else if (tokenizer.code_point == '(') {
// Otherwise if tokenizer’s code point is U+0028 (():
// Increment depth by 1.
depth++;
Expand All @@ -719,7 +719,7 @@ tl::expected<std::vector<Token>, url_pattern_errors> tokenize(
// Run get the next code point given tokenizer.
tokenizer.get_next_code_point();
// If tokenizer’s code point is not U+003F (?):
if (tokenizer.code_point != "?") {
if (tokenizer.code_point != '?') {
// Run process a tokenizing error given tokenizer, regexp start, and
// tokenizer’s index.
if (auto process_error = tokenizer.process_tokenizing_error(
Expand Down