Skip to content

Conversation

@rxin
Copy link
Contributor

@rxin rxin commented Nov 19, 2015

We currently rely on the optimizer's constant folding to replace current_timestamp and current_date. However, this can still result in different values for different instances of current_timestamp/current_date if the optimizer is not running fast enough.

A better solution is to replace these functions in the analyzer in one shot.

@rxin
Copy link
Contributor Author

rxin commented Nov 19, 2015

cc @cloud-fan

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: space before {

@rxin
Copy link
Contributor Author

rxin commented Nov 19, 2015

k updated.

@SparkQA
Copy link

SparkQA commented Nov 19, 2015

Test build #46311 has finished for PR 9833 at commit 33013e6.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * public final class UnsafeSorterSpillReader extends UnsafeSorterIterator implements Closeable\n * class Pipeline(override val uid: String) extends Estimator[PipelineModel] with MLWritable\n * class CountVectorizerModelWriter(instance: CountVectorizerModel) extends MLWriter\n * final class IDF(override val uid: String) extends Estimator[IDFModel] with IDFBase\n * class MinMaxScalerModelWriter(instance: MinMaxScalerModel) extends MLWriter\n * class SQLTransformer @Since(\"1.6.0\") (override val uid: String) extends Transformer\n * class StandardScalerModelWriter(instance: StandardScalerModel) extends MLWriter\n * class StringIndexModelWriter(instance: StringIndexerModel) extends MLWriter\n * class ALS(override val uid: String) extends Estimator[ALSModel] with ALSParams\n * abstract class MLWriter extends BaseReadWrite with Logging\n * trait MLWritable\n * abstract class MLReader[T] extends BaseReadWrite\n * trait MLReadable[T]\n * trait ScalaReflection\n * case class Schema(dataType: DataType, nullable: Boolean)\n * s\"Unable to generate an encoder for inner class$\n *public abstract class SpecificParquetRecordReaderBase extends RecordReader<Void, T> \n * abstract static class IntIterator \n * protected static final class ValuesReaderIntIterator extends IntIterator \n * protected static final class RLEIntIterator extends IntIterator \n * protected static final class NullIntIterator extends IntIterator \n *public class UnsafeRowParquetRecordReader extends SpecificParquetRecordReaderBase `\n

@SparkQA
Copy link

SparkQA commented Nov 19, 2015

Test build #46319 has finished for PR 9833 at commit 1ddbe84.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor

LGTM

@rxin
Copy link
Contributor Author

rxin commented Nov 19, 2015

Merging this. Thanks.

@asfgit asfgit closed this in f449992 Nov 19, 2015
asfgit pushed a commit that referenced this pull request Nov 19, 2015
…imestamp with literals

We currently rely on the optimizer's constant folding to replace current_timestamp and current_date. However, this can still result in different values for different instances of current_timestamp/current_date if the optimizer is not running fast enough.

A better solution is to replace these functions in the analyzer in one shot.

Author: Reynold Xin <[email protected]>

Closes #9833 from rxin/SPARK-11849.

(cherry picked from commit f449992)
Signed-off-by: Reynold Xin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants