Invalid output utf8 chars

[romeOz]

Hi, Cebe. May all the same you will add the flag u, as shown PHP Regex: How to match \r and \n without using [\r\n].

Namely:

$text = preg_replace('~\R~u', "\n", $text);

[cebe]

why?

[samdark]

Unicode variant matches more line break types i.e. U+2028 and U+2029.

[cebe]

I know but is there any usecase for these? I have no idea what they mean so I'd like to discuss this before adding a feature that I do not understand.

[cebe]

note that Markdown is not generally restricted to UTF-8 encoding. I currently have UTF-8 hardcoded for htmlspecialchars but I am thinking about making this configurable. So when there is no need for UTF-8 linebreaks I'd not add this feature and allow using different encodings.

[samdark]

I don't have any concrete examples for it.

[romeOz]

Problem in displaying Cyrillic character x (lowercase): U+0445

$markdown->parseParagraph('абвгдеёжзийклмнопрстуфхцчшщъыьэюя');
// output: абвгдеёжзийклмнопрстуф�цчшщъыьэюя

Otherwise, change the \cebe\markdown\Parser::prepareMarkers() on protected, so I could to abstract.

[samdark]

If it's reproduceable I'd fix it.

[chigix]

Here is a case appeared for me:
In Chinese，there is a charater, 配, that would be in conflict with the pattern without unicode statement.

I've create the pull request for this problem.

[arabcoders]

Same here with Arabic, the code interfere with Arabic Latter "م (M)"

$text = preg_replace('~\R~', "\n", $text);

but after changing the code to

$text = preg_replace('~\R~u', "\n", $text);

it does fix the issue,

[cebe]

Fixed, thanks all for the detailed information!