Loom on Scala

[Paul Bjorkstrand]

On Sat, Nov 27, 2021 at 10:08 PM Eric Kolotyluk <eric at kolotyluk.net> wrote:

> @Paul Bjorkstrand <paul.bjorkstrand at gmail.com>
>
> The vast majority of Java developers I have worked with that understand
>> streams, also understand that they are lazy, and require a terminal
>> operation.
>>
>
> In a polyglot world, where people are using multiple languages, expecting
> people to always be an expert on all idioms is asking a lot. I am
> advocating designs and idioms that are safer and more intuitive in the big
> picture. Also, I can see the utility in not having to *always *having to
> use lazy evaluation. WRT Koltin and Scala, Java is the oddball here.
>

I understand that different languages have different features. One could
argue that eagerly evaluated "stream-like" operations on collections is
just as dangerous. Let's put this conversation to the side, though. It is
not very pertinent to this list. (Happy to chat about it on the side, and
hear your POV. I have limited Kotlin and zero Scala experience.)


> Exceptions thrown by .close() are already handled by the catch block(s)
>> attached to the try-with-resources.
>>
>
> Ah, right, I remember now encountering that scenario. Somehow I thought
> the close() was done in the finally, but it's done before, so catchable.
> Thanks for the correction.
>

Happy to help! Also, I am thankful I never have to write this kind of code
ever again, in Java:

try {
  CloseableThing ct = ...
  try {
    ... use ct ...
  } finally {
    ct.close();
  }
} catch (SomeException e) {
  ... handle ...
}


> I could see this being useful to have close() call join, in addition to it
>> calling shutdown()
>>
>
> I don't think close() needs to call shutdown(), it just needs to detect if
> all tasks have been completed, and call join() if they have not. Even
> better, join() should be idempotent (and may be already). If people want to
> shutdown(), it should be explicit.
>

It already does call shutdown (well, implShutdown(), but both shutdown()
and close() call that method). I think the point of calling shutdown() (or
its internal equivalent) in close() is to ensure the executor is entirely
done being used once the scope of the try-with-resources is exited. I hope
a loom dev can either confirm this or correct me if I am wrong here.

As for idempotency of join(), it is not. The join() call depends on if
there are any currently executing any tasks created by .fork(). From what I
have read, it is perfectly valid to call fork() a bunch of times, call
join() to sync them, then call fork(), then call join(), then fork(), then
join()... ad infinitum, until either shutdown() or close() are called. A
quick test has validated this understanding.


> I could see an additional parameter flag for StructuredExecutor.open()
>> which changes the shutdown implementation from the default "interrupt" to
>> "wait" aka join.
>>
>
> Sorry, I don't really see the utility in that. My preferred solution is to
> use something like
>

Apologies if this caused any confusion, but I meant "close implementation"
above instead of "shutdown implementation".  The flag would merely say
"call join() before you call shutdown() in close()". That way, for the
simplest cases, you can omit the join() entirely. I suspect those types of
use cases are not as useful, even though I recently made a case for
something similar.

> structuredExecutor.joinUntil(Instant.now().plusSeconds(10));
>
> catch the TimeoutException, and then call shutdown(). Unfortunately, that
> won't work because the variable structuredExecutor is out of scope in the
> catch. That's another Java pet peeve of mine.
>

The problem is more than the fact that the variable is out of scope (more
on that below). The biggest problem you will encounter is that the executor
is *already closed* by the time the catch block is executed. At that point,
there is nothing left you really can do with the executor. It is just
garbage waiting to be collected.

That was part of the point of try-with-resources in the first place - you
don't need a reference to the closable objects inside catch anymore because
they are already closed and effectively useless.

But, if you really want to access a try-with-resources object outside the
try...

class Main {
  public static void main(String args[]) {
    AC ac = new AC();

    // as long as the variable inside try-with-resources is effectively
final, it can be used
    try (ac) {

    } catch (Exception e) {
      ac.debug();
    }
  }

  static class AC implements AutoCloseable {
    public void close() throws Exception {
      throw new Exception();
    }

    public void debug() {
      System.out.println("Called from catch");
    }
  }
}

Using an object meant for try-with-resources outside the try-block is a
pretty big red flag, IMO.


> Maybe something like
>
> structuredExector.joinUntil(Instant.now().plusSeconds(10)).then( () -> {
> structuredExector.shutdown();  structuredExector.join(); } )
>
> But now I am really just fantasizing.
>

I don't see the point of calling join() after a shutdown(), because
shutdown interrupts the other threads. Also, join() just returns if the
executor is already shutdown (and throws if it is closed).

Besides, since shutdown() is called in the close(), which
try-with-resources calls right after the try-block has exited, there is no
need to call it explicitly, unless you want to shut down early.


> Cheers, Eric
>
> On Sat, Nov 27, 2021 at 5:20 PM Paul Bjorkstrand <
> paul.bjorkstrand at gmail.com> wrote:
>
>>
>>>    1. Java Collections need non-lazy monadic operators in addition to
>>>    Stream.
>>>
>>
>> The vast majority of Java developers I have worked with that understand
>> streams, also understand that they are lazy, and require a terminal
>> operation. There may be some places where you could simplify some code with
>> non-lazy operations, but every non-lazy operation is just a lazy operation
>> that has an immediate terminal operation.
>>
>>
>>>    - Even in Java, join() should not be necessary because it is possible
>>> to
>>>    wait for all Futures to be complete by other means.
>>>       - Indeed, close() should be able to detect if all tasks have
>>>       completed, even if join() has not been called.
>>>       - It is reasonable to expect StructuredExecutor#close to implicitly
>>>    join() if necessary,
>>>       - and throw an exception when appropriate, even if this requires
>>>       wrapping try-with-resources with another try to catch the
>>> closing exception.
>>>       - yes, this would be really ugly, but we have trade-offs on where
>>>       ugly and beautiful present themselves
>>>
>>
>> Exceptions thrown by .close() are already handled by the catch block(s)
>> attached to the try-with-resources. There is no need for nesting. Example:
>>
>> class Main {
>>   public static void main(String args[]) {
>>     try (var twc = new ThrowsWhenClosed()) {
>>       // Do nothing
>>     } catch (Exception e) {
>>       System.out.println(e.getMessage());
>>     }
>>   }
>> }
>>
>> class ThrowsWhenClosed implements AutoCloseable {
>>   public void close() throws Exception {
>>     throw new Exception("Always throws");
>>   }
>> }
>>
>> This outputs "Always throws" to stdout when run.
>>
>>
>>>    - Yes, calling join() before close() is an extremely good practice for
>>>    many reasons, but it's not always semantically necessary.
>>>    - Is calling join() before close() a seatbelt where Loom is trying to
>>>    protect us from ourselves?
>>>    - Is it too much work to implement this otherwise?
>>>    - Is there some hard-core technical reason why this is necessary?
>>
>>
>> I could see this being useful to have close() call join, in addition to
>> it calling shutdown(), but shutdown() currently cancels futures and
>> interrupts threads (
>> https://github.com/openjdk/loom/blob/70b2a4f3723379ba7e36115651b2b18bd7acee5d/src/java.base/share/classes/java/util/concurrent/StructuredExecutor.java#L557
>> )
>>
>> I could see an additional parameter flag for StructuredExecutor.open()
>> which changes the shutdown implementation from the default "interrupt" to
>> "wait" aka join.
>>
>>
>> -Paul
>>
>