Generalized comprehension syntax

[lindahua]

What about the following?

(f(x) for x in a)   
# this may be some indexable view of a special type that can be 
# passed as arguments, such that we can do

sum(f(x) for x in a) 

In this way, statistics can be easily implemented in an efficient way, for example

var(a) = sum(abs2(x - mean(x)) for x in a) / (length(a) - 1)

This is a one-liner, but it does not create any temporary arrays.

If the Cartesian iteration discussed in #1917 can be implemented, this would become even more powerful:

# weighted mean
wmean(a, ws) = dot(a, ws) / sum(ws)

# weighted variance
wvar(a, ws) = sum(abs2(x - mean(x)) * w for (x, w) in (a, ws)...)

[kmsquire]

A known as a 'generator' in python parlance. I swear this was discussed previously, but I can't find an issue, so perhaps it was only on the mailing list (see here: https://groups.google.com/forum/#!topic/julia-dev/iGVJ-61UmVk).

[lindahua]

I also remembered I saw the discussion somewhere. But I did not actually find an issue here, so I open one.

This, IMHO, is worth our attention.

[JeffBezanson]

Yes this would be a really good feature.

[johnmyleswhite]

+1

[StefanKarpinski]

Yes, I've long wanted this. One nice feature is that although it in principle uses tasks which may require multiple stacks and therefore might be slow, there are probably many cases where one can determine that it isn't necessary to actually have two stacks so it can be optimized away to very efficient code.

[JeffBezanson]

It's a Map iterator, that wraps another iterator and applies some function to each of its elements.

[StefanKarpinski]

Ah, ok. That will work as well.

[quinnj]

+1. Looking forward to having this.

[mbauman]

See my experiments on the mailing list with an implementation on my generators branch. (You have to manually include("base/generator.jl"), which I had forgotten myself when I rebased it today).

(edit: thanks @rfourquet)

[rfourquet]

without an "s" : include("base/generator.jl")

[rfourquet]

On top of @mbauman branch, I tried to prevent expensive allocations, at least up to dim 3.