Proposal idea - generators

[Neal Gafter]

The problem with this approach is that, without cooperation from the
client, the other thread can stick around for arbitrarily long, which
is a severe resource leak.  I wrote about this at
<http://gafter.blogspot.com/2007/07/internal-versus-external-iterators.html>.
 That's because the generating thread has no way of knowing when the
client did a "break" from the loop.

You can solve this if your language has support for an AutoCloseable
interface, and the Iterator is closed at the end by the expansion of
the for-each loop when the Iterator is a subtype of AutoCloseable.
Unfortunately, the current ARM proposal does not do that.

On Wed, Mar 25, 2009 at 6:33 AM, Tim Peierls <tim at peierls.net> wrote:
> My sense (bolstered by a quick Google code search) is that people have been
> coming up with this kind of functionality on their own as they encounter a
> need for it. Is it really something that needs special language support?
> Using another thread to do the generation is quite reasonable. You can build
> a nice coroutine-style facility with a pair of SynchronousQueues (or, more
> generally and flexibly, with TransferQueues, expected for Java 7; or maybe
> with Phasers). Using that as a building block you could then rewrite your
> example below as something like this:
>
> final List<User> allUsersList = ...;
> Iterator<User> userIterator = generatorToIterator(new Generator<User>() {
>    public void generate(Generation<User> generation) {
>        for (User user : allUsersList) {
>            if (user.isActive()) generation.yield(user);
>        }
>    }
> });
> // Now userIterator lazily produces active users.
>
>
> Getting this kind of thing right is hard enough that someone should propose
> putting a production version in the standard library, but I don't see the
> need for a language change.
>
> For your particular example, you could use Google Collections'
> Iterators.filter<http://google-collections.googlecode.com/svn/trunk/javadoc/com/google/common/collect/Iterators.html#filter(java.util.Iterator,
> com.google.common.base.Predicate)>:
>
> Iterator<User> userIterator = filter(allUsersList.iterator(), new
> Predicate<User>() {
>    public boolean apply(User user) {
> return user.isActive();
>    }
> });
>
>
> That's not true generation, but I bet lazy filtering handles a lot of the
> common cases that people think they want generation for.
>
> --tim
>
>
>
> On Tue, Mar 24, 2009 at 3:07 PM, Mark Derricutt <mark at talios.com> wrote:
>
>> Hi all,
>>
>> This isn't an actual proposal, but more a proposal idea that hopefully
>> someone could run with.  I was thinking about all the closures issues
>> and looking at a lot of my code where I'm using closure like patterns
>> and started thinking what would really make things easier for me would
>> be some form of yield/generators api to give me a cleaner way of
>> making Iterable's that could be used in the JDK5 for loop.
>>
>> The usage I'd like would be something like:
>>
>> for ( User user : allUsersList) {
>>  if (user.isActive()) yield user;
>> }
>>
>> Effectively I could see this as being like comparing an XML pull
>> parser to SAX.  The problem I have is I can't think of a suitable way
>> of expressing, or implementing this in java, unless I split the code
>> into another thread that sleeps between the yield calling back to the
>> iterable next() call which is nasty as I could see a lot of thread
>> leakage there.
>>
>> Has anyone looked at, or already written some form of generator proposal?
>>
>>
>> ...and then Buffy staked Edward.  The End.
>>
>>
>
>