Single Thread Continuation

[Robert Engels]

The generator is closed in response to yield() - this could have been an exception as well. It is also “closed” if it returns for any reason including an exception. It is all built into the framework - the dev does not need to code anything special. 

The code is less than 150 lines for the complete generic implementation. The code to write and use a generator can be a single line. How much simpler do you need?

> On Jul 5, 2023, at 6:57 AM, Alex Otenko <oleksandr.otenko at gmail.com> wrote:
> 
> 
> Well, the code you shared only illustrates that it is not as simple as claimed.
> 
> So, you don't use a queue. That's fine. Now, how will the generator stop? You have the close() in finalize(), and a dance around hasNext() to totally order the delivery of elements vs end of stream.
> 
> Basically, it's two channels.
> 
> 
>> On Tue, 4 Jul 2023, 15:07 robert engels, <rengels at ix.netcom.com> wrote:
>> It is also straightforward to use a buffered queue to allow the producer to read ahead rather than the hand-off queue as implemented - which is often necessary when the producer is reading from a database, network calls, etc.
>> 
>> There is no need to for specific generator support in Java when you have cheap virtual threads.
>> 
>>> On Jul 4, 2023, at 8:58 AM, robert engels <rengels at ix.netcom.com> wrote:
>>> 
>>> I created a project to demonstrate. It uses finalizers for simplicity but it easily adaptable to weak reference queues. It only needs a single producer virtual thread per generator.
>>> 
>>> see https://github.com/robaho/generators/tree/main
>>> 
>>>> On Jul 4, 2023, at 7:24 AM, Robert Engels <rengels at ix.netcom.com> wrote:
>>>> 
>>>> You need 3 threads - the producer the consumer and the watcher (the watcher and producer might be able to be combined with async notifications from the weak reference queue) But it is all encapsulated and virtual threads makes this very cheap - as cheap as actual continuations would be. 
>>>> 
>>>>>> On Jul 4, 2023, at 7:19 AM, Alex Otenko <oleksandr.otenko at gmail.com> wrote:
>>>>>> 
>>>>> 
>>>>> So it's not really just the queue.
>>>>> 
>>>>> Plus you need two of them. Otherwise you can't control the timing of when the next value is produced.
>>>>> 
>>>>>> On Tue, 4 Jul 2023, 13:13 Robert Engels, <rengels at ix.netcom.com> wrote:
>>>>>> To elaborate - the producer knows when no more values are coming and marks the iterator as such. So the next calls to hasNext()/next() fail appropriately. That is trivial. 
>>>>>> 
>>>>>> The only “complex” part of the design is that the returned iterator is tracked with a weak reference by the producer. So if it goes out of scope the producer can be woken from the blocked/waiting state and clean up. 
>>>>>> 
>>>>>>>> On Jul 4, 2023, at 7:01 AM, Robert Engels <rengels at ix.netcom.com> wrote:
>>>>>>>> 
>>>>>>> 
>>>>>>> That is built into the Iterator interface. 
>>>>>>> 
>>>>>>>>> On Jul 4, 2023, at 4:05 AM, Alex Otenko <oleksandr.otenko at gmail.com> wrote:
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> How does the generator tell the consumer that no more values are forthcoming? 
>>>>>>>> 
>>>>>>>>> On Mon, 3 Jul 2023, 12:38 Robert Engels, <rengels at ix.netcom.com> wrote:
>>>>>>>>> Believe me. Queues are all you need there is no memory leak and no need to “close”. The producer side uses a weak reference to the queue. When there are no more strong references the producer side can terminate. 
>>>>>>>>> 
>>>>>>>>> You can’t use a standard blocking queue for this - but the queue implementation is fairly trivial - with a wake-up thread that listens on the weak reference queue. 
>>>>>>>>> 
>>>>>>>>>>> On Jul 3, 2023, at 6:19 AM, Attila Kelemen <attila.kelemen85 at gmail.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 2. We need to synchronize access to mutable state to avoid memory
>>>>>>>>>>>    hazards. This is a separate issue from synchronizing access to
>>>>>>>>>>>    mutable state to avoid correctness issues. With virtual threads on
>>>>>>>>>>>    a single platform thread, this goes away too (because it's always
>>>>>>>>>>>    the same thread observing memory operations; no barriers needed).
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> That still seems incorrect to me (in principle, in practice it most likely will end up to be fine, but I just wouldn't rely on it), because the barrier is needed to prevent instruction reordering by the compiler, and you are not safe from that by using the same platform thread.
>>> 
>> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20230705/980a38bf/attachment.htm>