Skip to content

Enable StringView by default by passing CI #11862

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
XiangpengHao wants to merge 11 commits into from
Closed

Enable StringView by default by passing CI #11862

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
3b8d1a1
avoid copy
XiangpengHao Aug 6, 2024
05bf29c
fmt
XiangpengHao Aug 6, 2024
26d2826
update sqlogictest
XiangpengHao Aug 7, 2024
af8204a
support string view in stats
XiangpengHao Aug 7, 2024
c83fee9
update tests
XiangpengHao Aug 7, 2024
57feefd
allow Utf8View in subtrait
XiangpengHao Aug 9, 2024
dc075c3
fmt
XiangpengHao Aug 9, 2024
225b3aa
update tests
XiangpengHao Aug 9, 2024
c65fff6
make examples work
XiangpengHao Aug 9, 2024
8311bd6
prost version
XiangpengHao Aug 9, 2024
ec82289
update
XiangpengHao Aug 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
avoid copy
  • Loading branch information
@XiangpengHao
XiangpengHao committed Aug 9, 2024
commit 3b8d1a19dd66c3062aa0dd12cfba425e33195f02
16 changes: 6 additions & 10 deletions datafusion/core/src/datasource/physical_plan/arrow_file.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -297,10 +297,7 @@ impl FileOpener for ArrowOpener {
for (dict_block, dict_result) in
footer.dictionaries().iter().flatten().zip(dict_results)
{
decoder.read_dictionary(
dict_block,
&Buffer::from_bytes(dict_result.into()),
)?;
decoder.read_dictionary(dict_block, &Buffer::from_bytes(dict_result.into()))?;
}

// filter recordbatches according to range
Expand Down Expand Up @@ -335,12 +332,11 @@ impl FileOpener for ArrowOpener {
.into_iter()
.zip(recordbatch_results)
.filter_map(move |(block, data)| {
decoder
.read_record_batch(
&block,
&Buffer::from_bytes(data.into()),
)
.transpose()
match decoder.read_record_batch(&block, &Buffer::from_bytes(data.into())) {
Ok(Some(record_batch)) => Some(Ok(record_batch)),
Ok(None) => None,
Err(err) => Some(Err(err)),
}
}),
)
.boxed())
Expand Down