Skip to content

Conversation

@marmbrus
Copy link
Contributor

In this PR I delete a method that breaks type inference for aggregators (only in the REPL)

The error when this method is present is:

<console>:38: error: missing parameter type for expanded function ((x$2) => x$2._2)
              ds.groupBy(_._1).agg(sum(_._2), sum(_._3)).collect()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dragos any idea why it fails to infer the type only in the REPL?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't reproduce the difference. It won't infer it in a standalone program either.

As I mentioned in our conversation, it's a chicken and egg problem: type inference is guided by the expected type, but if the method is overloaded, the expected type is not known. And the argument type is what guides overload resolution. It works in simple cases, when the overloads have different aritites, but with varargs that's no longer the case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It always works fine here though. Why is this different?

asfgit pushed a commit that referenced this pull request Nov 20, 2015
In this PR I delete a method that breaks type inference for aggregators (only in the REPL)

The error when this method is present is:
```
<console>:38: error: missing parameter type for expanded function ((x$2) => x$2._2)
              ds.groupBy(_._1).agg(sum(_._2), sum(_._3)).collect()
```

Author: Michael Armbrust <[email protected]>

Closes #9870 from marmbrus/dataset-repl-agg.

(cherry picked from commit 968acf3)
Signed-off-by: Michael Armbrust <[email protected]>
@asfgit asfgit closed this in 968acf3 Nov 20, 2015
@SparkQA
Copy link

SparkQA commented Nov 20, 2015

Test build #46441 has finished for PR 9870 at commit d7367da.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@marmbrus
Copy link
Contributor Author

Ugh, sorry :(

I'll fix this.

@marmbrus marmbrus deleted the dataset-repl-agg branch March 8, 2016 00:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants