Skip to content

[SPARK-23649][SQL] Skipping chars disallowed in UTF-8 #20796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 13 commits into from
Closed

[SPARK-23649][SQL] Skipping chars disallowed in UTF-8 #20796

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
6d6b3ca
Test: skip bytes disallowed in UTF-8
MaxGekk Feb 27, 2018
0f474a0
Making correct map of first byte to char size in UTF-8 and skip bytes…
MaxGekk Feb 27, 2018
2ee6616
Check inferred schema and returned bad string
MaxGekk Feb 28, 2018
d6c5f02
The test csv was simplified
MaxGekk Feb 28, 2018
264e5ac
Merge branch 'master' of github.com:apache/spark into skip-wrong-utf8…
MaxGekk Mar 13, 2018
50b17b1
Test for handling of the first byte of UTF-8 chars
MaxGekk Mar 15, 2018
c65f827
Using explicit charset in converting of a string to a byte array
MaxGekk Mar 16, 2018
45f67fe
Merge branch 'master' of github.com:apache/spark into skip-wrong-utf8…
MaxGekk Mar 16, 2018
27e5a5b
Basic tests for 2-4 bytes chars and a test for skipping of wrong firs…
MaxGekk Mar 16, 2018
7891767
Merge branch 'master' of github.com:apache/spark into skip-wrong-utf8…
MaxGekk Mar 16, 2018
e4f6e1a
Merge branch 'master' of github.com:apache/spark into skip-wrong-utf8…
MaxGekk Mar 17, 2018
8a501e3
Removing tests for CSV and adding additional cases for UTF8StringSuite
MaxGekk Mar 17, 2018
5557a80
Comment about disallowed first bytes according to UTF-8 standard
MaxGekk Mar 19, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'master' of github.com:apache/spark into skip-wrong-utf8… 
…-chars
  • Loading branch information
@MaxGekk
MaxGekk committed Mar 17, 2018
commit e4f6e1a0d78e8732b33472bcf0b12ff9ae2b0fc9

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.