Skip to content

Bug: interaction between groupby/orderby/filter/slice(-1) #393

Description

@hbwhbw

I thought that slice(-1) should reliably produce the last element of each group after groupby/orderby/filter is applied.
Further, whenever slice(0,1) returns non-empty partitions, slice(-1) should also return non-empty partitions.

However, here is a tiny example demonstrating this is not true:

const d = aq.from([{a: 1, b: 2, c: 3}, {a: 1, b: 3, c: 4}])
// prints 0 (slice(-1))  <== this is the problem
console.log(d.groupby("a").orderby("b").filter(`d.a === 1 && d.c === 3`).slice(-1).partitions().length);
// prints 1 (no slice)
console.log(d.groupby("a").orderby("b").filter(`d.a === 1 && d.c === 3`).partitions().length);
// prints 1 (slice(0,1))
console.log(d.groupby("a").orderby("b").filter(`d.a === 1 && d.c === 3`).slice(0,1).partitions().length);

Some unintuitive things "fix" the problem:

// remove orderby => prints 1
console.log(d.groupby("a").filter(`d.a === 1 && d.c === 3`).slice(-1).partitions().length);
// remove filter condition on c => prints 1
console.log(d.groupby("a").orderby("b").filter(`d.a === 1`).slice(-1).partitions().length);
// call reify() before slicing => prints 1
console.log(d.groupby("a").orderby("b").filter(`d.a === 1 && d.c === 3`).reify().slice(-1).partitions().length);

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions