when colum is use alias ,the order by result is wrong #16890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

muyannian wants to merge 5,455 commits into branch-1.6 from master

Closed

when colum is use alias ,the order by result is wrong #16890

[MINOR][BUILD] Fix javadoc8 break

## What changes were proposed in this pull request?

These error below seems caused by unidoc that does not understand double commented block.

```
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:69: error: class, interface, or enum expected
[error]  * MapGroupsWithStateFunction&lt;String, Integer, Integer, String&gt; mappingFunction =
[error]                                  ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:69: error: class, interface, or enum expected
[error]  * MapGroupsWithStateFunction&lt;String, Integer, Integer, String&gt; mappingFunction =
[error]                                                                       ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:70: error: class, interface, or enum expected
[error]  *    new MapGroupsWithStateFunction&lt;String, Integer, Integer, String&gt;() {
[error]                                         ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:70: error: class, interface, or enum expected
[error]  *    new MapGroupsWithStateFunction&lt;String, Integer, Integer, String&gt;() {
[error]                                                                             ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:72: error: illegal character: '#'
[error]  *      &#64;Override
[error]          ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:72: error: class, interface, or enum expected
[error]  *      &#64;Override
[error]              ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:73: error: class, interface, or enum expected
[error]  *      public String call(String key, Iterator&lt;Integer&gt; value, KeyedState&lt;Integer&gt; state) {
[error]                ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:73: error: class, interface, or enum expected
[error]  *      public String call(String key, Iterator&lt;Integer&gt; value, KeyedState&lt;Integer&gt; state) {
[error]                                                    ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:73: error: class, interface, or enum expected
[error]  *      public String call(String key, Iterator&lt;Integer&gt; value, KeyedState&lt;Integer&gt; state) {
[error]                                                                ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:73: error: class, interface, or enum expected
[error]  *      public String call(String key, Iterator&lt;Integer&gt; value, KeyedState&lt;Integer&gt; state) {
[error]                                                                                     ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:73: error: class, interface, or enum expected
[error]  *      public String call(String key, Iterator&lt;Integer&gt; value, KeyedState&lt;Integer&gt; state) {
[error]                                                                                                 ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:76: error: class, interface, or enum expected
[error]  *          boolean shouldRemove = ...; // Decide whether to remove the state
[error]  ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:77: error: class, interface, or enum expected
[error]  *          if (shouldRemove) {
[error]  ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:79: error: class, interface, or enum expected
[error]  *          } else {
[error]  ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:81: error: class, interface, or enum expected
[error]  *            state.update(newState); // Set the new state
[error]  ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:82: error: class, interface, or enum expected
[error]  *          }
[error]  ^
[error] .../forked/spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:85: error: class, interface, or enum expected
[error]  *          state.update(initialState);
[error]  ^
[error] .../forked/spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:86: error: class, interface, or enum expected
[error]  *        }
[error]  ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:90: error: class, interface, or enum expected
[error]  * </code></pre>
[error]  ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:92: error: class, interface, or enum expected
[error]  * tparam S User-defined type of the state to be stored for each key. Must be encodable into
[error]            ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:93: error: class, interface, or enum expected
[error]  *           Spark SQL types (see {link Encoder} for more details).
[error]                                          ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:94: error: class, interface, or enum expected
[error]  * since 2.1.1
[error]           ^
```

And another link seems unrecognisable.

```
.../spark/sql/core/target/java/org/apache/spark/sql/KeyedState.java:16: error: reference not found
[error]  * That is, in every batch of the {link streaming.StreamingQuery StreamingQuery},
[error]
```

Note that this PR does not fix the two breaks as below:

```
[error] .../spark/sql/core/target/java/org/apache/spark/sql/DataFrameStatFunctions.java:43: error: unexpected content
[error]    * see {link DataFrameStatsFunctions.approxQuantile(col:Str* approxQuantile} for
[error]      ^
[error] .../spark/sql/core/target/java/org/apache/spark/sql/DataFrameStatFunctions.java:52: error: bad use of '>'
[error]    * param relativeError The relative target precision to achieve (>= 0).
[error]                                                                     ^
[error]
```

because these seem probably fixed soon in #16776 and I intended to avoid potential conflicts.

## How was this patch tested?

Manually via `jekyll build`

Author: hyukjinkwon <[email protected]>

Closes #16926 from HyukjinKwon/javadoc-break.

Loading branch information

HyukjinKwon authored and srowen committed Feb 16, 2017

commit 3b4376876fabf7df4bd245dcf755222f4fe5f190

sql/core/src/main/scala/org/apache/spark/sql/KeyedState.scala

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -17,8 +17,6 @@
  
    package org.apache.spark.sql

    import java.lang.IllegalArgumentException

    import org.apache.spark.annotation.{Experimental, InterfaceStability}

    import org.apache.spark.sql.catalyst.plans.logical.LogicalKeyedState

    @@ -36,7 +34,7 @@ import org.apache.spark.sql.catalyst.plans.logical.LogicalKeyedState
  
     * `Dataset.groupByKey()`) while maintaining user-defined per-group state between invocations.

     * For a static batch Dataset, the function will be invoked once per group. For a streaming

     * Dataset, the function will be invoked for each group repeatedly in every trigger.

     * That is, in every batch of the [[streaming.StreamingQuery StreamingQuery]],

     * That is, in every batch of the `streaming.StreamingQuery`,

     * the function will be invoked once for each group that has data in the batch.

     *

     * The function is invoked with following parameters.

    @@ -65,7 +63,7 @@ import org.apache.spark.sql.catalyst.plans.logical.LogicalKeyedState
  
     *

     * Scala example of using KeyedState in `mapGroupsWithState`:

     * {{{

     * /* A mapping function that maintains an integer state for string keys and returns a string. */

     * // A mapping function that maintains an integer state for string keys and returns a string.

     * def mappingFunction(key: String, value: Iterator[Int], state: KeyedState[Int]): String = {

     *   // Check if state exists

     *   if (state.exists) {

    @@ -88,7 +86,7 @@ import org.apache.spark.sql.catalyst.plans.logical.LogicalKeyedState
  
     *

     * Java example of using `KeyedState`:

     * {{{

     * /* A mapping function that maintains an integer state for string keys and returns a string. */

     * // A mapping function that maintains an integer state for string keys and returns a string.

     * MapGroupsWithStateFunction<String, Integer, Integer, String> mappingFunction =

     *    new MapGroupsWithStateFunction<String, Integer, Integer, String>() {

     *

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

when colum is use alias ,the order by result is wrong #16890

Uh oh!

Diff view

Diff view

Uh oh!

There are no files selected for viewing