Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
[SPARK-9401][SQL] codeGen concatWs
  • Loading branch information
tarekbecker committed Jul 30, 2015
commit 46a6c20da83939d70020c880a964d6bd10fcd00c
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,32 @@ case class ConcatWs(children: Seq[Expression])
boolean ${ev.isNull} = ${ev.primitive} == null;
"""
} else {
// Contains a mix of strings and array<string>s. Fall back to interpreted mode for now.
super.genCode(ctx, ev)
val list = ctx.freshName("list")
val array = ctx.freshName("array")
val sep = children.head.gen(ctx)
val argsCode = children.tail.map(x => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this is just

val val argsCode = children.tail.map { x =>
  ...
}.mkString("\n")

val gen = x.gen(ctx)
if (x.dataType == StringType) {
s"""
${gen.code}
$list.add(${gen.primitive});
"""
} else {
s"""
${gen.code}
$list.addAll(scala.collection.JavaConversions.asJavaCollection(${gen.primitive}));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think this breaks because now we refactored the codebase to use ArrayData as the internal representation for Seq.

"""
}
}).foldLeft("")((a, b) => a + "\n" + b)
s"""
${sep.code}
java.util.ArrayList<UTF8String> $list = new java.util.ArrayList<UTF8String>();
$argsCode
UTF8String[] $array = new UTF8String[$list.size()];
$list.toArray($array);
UTF8String ${ev.primitive} = UTF8String.concatWs(${sep.primitive}, $array);
boolean ${ev.isNull} = ${ev.primitive} == null;
"""
}
}
}
Expand Down