Loom on Scala
Eric Kolotyluk
eric at kolotyluk.net
Sat Nov 27 23:12:12 UTC 2021
@Alex Otenko <oleksandr.otenko at gmail.com>
I saw this late "I meant stream.map(... executor.fork(...)) of course" --
and yes, I agree that is one source of trouble...
In this example, I was not implying Stream.map(_.get), because in this
context futureResults is implicitly,
val futureResults : IndexedSeq[Future[Int]] = . . .
which is a Scala collection that is not lazy. But you are right in that "it
is lazy evaluation in general, and that closures *can* [maybe, maybe not]
make things outlive their syntactic scope" which can be a problem in any
language, even Scala. Stream.map(_.get) might be a problem in some
contexts, but might be fine in other contexts.
However, I am trying to focus on the point that in Java, to use operations
like map(), Stream is the only path to take, so we are forced to deal with
lazy evaluation.
This experience of using Scala leads me to two conclusions:
1. Java Collections need non-lazy monadic operators in addition to
Stream.
2. StructuredExecutor#join should not be required before
StructuredExecutor#close.
val results =
Using(StructuredExecutor.open("HelloScala")) { structuredExecutor =>
val futureResults = (0 to 15).map { item =>
println(s"item = $item, Thread ID = ${Thread.currentThread}")
structuredExecutor.fork { () =>
println(s"\ttask = $item, Thread ID = ${Thread.currentThread}")
item
}
}
futureResults.map(_.get)
}
println(results)
- This is a perfectly valid expression of logic because
furtureResults.map(_.get) will wait/block until all the Futures have been
completed, which implicitly is the same as a join().
- Project Loom is an extension of the JVM and libraries, not the Java
Language. Other language users, such as Scala, should not be penalized by
having to use join() when it is not semantically necessary.
- Even in Java, join() should not be necessary because it is possible to
wait for all Futures to be complete by other means.
- Indeed, close() should be able to detect if all tasks have
completed, even if join() has not been called.
- It is reasonable to expect StructuredExecutor#close to implicitly
join() if necessary,
- and throw an exception when appropriate, even if this requires
wrapping try-with-resources with another try to catch the
closing exception.
- yes, this would be really ugly, but we have trade-offs on where
ugly and beautiful present themselves
- In some cases, it's really important/useful to call join, such as
dealing with any completion failures, but there are other mechanisms to
catch those failures.
My sense is that requiring join() before close() is a kluge, and I invite
someone to convince me otherwise. I am not trying to insult anyone, just
convey an emotion, and I am willing to be convinced otherwise.
- Yes, calling join() before close() is an extremely good practice for
many reasons, but it's not always semantically necessary.
- Is calling join() before close() a seatbelt where Loom is trying to
protect us from ourselves?
- Is it too much work to implement this otherwise?
- Is there some hard-core technical reason why this is necessary?
Cheers, Eric
On Sat, Nov 27, 2021 at 2:14 PM Alex Otenko <oleksandr.otenko at gmail.com>
wrote:
> Scala Stream.map(_.get) will cause the same grief as you encountered in
> Java. So it's not a problem specific to Java, it is lazy evaluation in
> general, and that closures can make things outlive their syntactic scope.
>
> Alex
>
> On Sat, 27 Nov 2021, 21:03 Eric Kolotyluk, <eric at kolotyluk.net> wrote:
>
>> Alex was right, Using.Manager was consuming the exception. The better
>> solution is
>>
>> object HelloScala {
>> def main(args: Array[String]) {
>> Context.printHeader(HelloScala.getClass)
>>
>> val results =
>> Using(StructuredExecutor.open("HelloScala")) { structuredExecutor =>
>> val futureResults = (0 to 15).map { item =>
>> println(s"item = $item, Thread ID = ${Thread.currentThread}")
>> structuredExecutor.fork { () =>
>> println(s"\ttask = $item, Thread ID = ${Thread.currentThread}")
>> item
>> }
>> }
>> structuredExecutor.join
>> futureResults.map(_.get)
>> }
>>
>> println(results)
>> }
>> }
>>
>> Which outputs either
>>
>> Success(Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15))
>> or
>> Failure(java.lang.IllegalStateException: Owner did not invoke join or
>> joinUntil)
>>
>> Depending on if join() is called.
>>
>> Alex, map(_.get) works fine because get() will wait for the value to
>> become available, while resultNow() won't.
>>
>> I am somewhat disappointed that Using consumes the exception rather than
>> letting it be caught in a try block, but it does return the exception as a
>> Scala Try result of either Success or Failure, which I can live with.
>> Still, this is not intuitive.
>>
>> It is interesting to note that IMHO the Scala code is aesthetically more
>> beautiful than the equivalent Java code, but Java has a stronger commitment
>> to safety and backward compatibility which creates challenges to good
>> aesthetics.
>>
>> Is there a JEP on extending Java collections to support monadic operators
>> such as map()? After this experience, I now feel that Java Stream creates
>> dangerous situations that are not intuitive to troubleshoot, and it would
>> be nice if Java Collections had the same functionality as Kotlin and Scala
>> Collections.
>>
>> Cheers, Eric
>>
>>
>> On Fri, Nov 26, 2021 at 11:41 PM Alex Otenko <oleksandr.otenko at gmail.com>
>> wrote:
>>
>>> Most likely Scala's try-with-resource catches and handles the exception
>>> during close()
>>>
>>> Same with map(_.get) vs map(Future::resultNow) - the difference is only
>>> in who's waiting and what kind of exception is thrown, so if Scala handles
>>> checked exceptions for you, you aren't inconvenienced.
>>>
>>> Alex
>>>
>>> On Sat, 27 Nov 2021, 07:19 Eric Kolotyluk, <eric at kolotyluk.net> wrote:
>>>
>>>> package net.kolotyluk.loom
>>>>
>>>> import java.util.concurrent.StructuredExecutor
>>>> import scala.util.Using
>>>>
>>>> object HelloScala {
>>>> def main(args: Array[String]) {
>>>> Context.printHeader(HelloScala.getClass)
>>>>
>>>> Using.Manager { use => // Scala's version of try-with-resources
>>>> val structuredExecutor =
>>>> use(StructuredExecutor.open("HelloScala"))
>>>>
>>>> val futureResults = (0 to 15).map{ item =>
>>>> println(s"item = $item, Thread ID = ${Thread.currentThread}")
>>>> structuredExecutor.fork{ () =>
>>>> println(s"\ttask = $item, Thread ID =
>>>> ${Thread.currentThread}")
>>>> item
>>>> }
>>>> }
>>>> println(futureResults.map(_.get))
>>>> structuredExecutor.join
>>>> }
>>>> }
>>>> }
>>>>
>>>> Yes, Loom does run on Scala 2.13... I cannot get Scala 3 to work with
>>>> Maven
>>>> yet... might have to resort to SBT... ��
>>>>
>>>> I wonder if ScopeLocal works correctly in Scala? �� I guess it should...
>>>>
>>>> Don't have to worry about lazy Stream termination here ��
>>>>
>>>> Don't need Future::resultNow either
>>>>
>>>> For some reason, if you don't call join(), close() does not throw an
>>>> exception �� -- that's not right?
>>>>
>>>> The Kotlin version was much harder to write and I will post later...
>>>>
>>>> A fun way to end the week... ��
>>>>
>>>> Cheers, Eric
>>>>
>>>>
>>>> "C:\Program Files (Open)\jdk-18\bin\java.exe" --enable-preview
>>>> -Dfile.encoding=windows-1252 -jar
>>>> C:\Users\ERIC\Documents\git\loom-lab\laboratory\target\laboratory.jar
>>>> Hello net.kolotyluk.loom.HelloScala$
>>>> PID = 23312
>>>> CPU Cores = 12
>>>> Heap Size = 6442450944 bytes
>>>>
>>>> ______________________________________________________________________________
>>>>
>>>> item = 0, Thread ID = Thread[#1,main,5,main]
>>>> item = 1, Thread ID = Thread[#1,main,5,main]
>>>> item = 2, Thread ID = Thread[#1,main,5,main]
>>>> item = 3, Thread ID = Thread[#1,main,5,main]
>>>> item = 4, Thread ID = Thread[#1,main,5,main]
>>>> item = 5, Thread ID = Thread[#1,main,5,main]
>>>> item = 6, Thread ID = Thread[#1,main,5,main]
>>>> item = 7, Thread ID = Thread[#1,main,5,main]
>>>> item = 8, Thread ID = Thread[#1,main,5,main]
>>>> item = 9, Thread ID = Thread[#1,main,5,main]
>>>> item = 10, Thread ID = Thread[#1,main,5,main]
>>>> item = 11, Thread ID = Thread[#1,main,5,main]
>>>> item = 12, Thread ID = Thread[#1,main,5,main]
>>>> task = 0, Thread ID =
>>>> VirtualThread[#15]/runnable at ForkJoinPool-1-worker-1
>>>> item = 13, Thread ID = Thread[#1,main,5,main]
>>>> item = 14, Thread ID = Thread[#1,main,5,main]
>>>> task = 1, Thread ID =
>>>> VirtualThread[#17]/runnable at ForkJoinPool-1-worker-2
>>>> task = 2, Thread ID =
>>>> VirtualThread[#18]/runnable at ForkJoinPool-1-worker-3
>>>> task = 4, Thread ID =
>>>> VirtualThread[#20]/runnable at ForkJoinPool-1-worker-3
>>>> task = 5, Thread ID =
>>>> VirtualThread[#22]/runnable at ForkJoinPool-1-worker-2
>>>> task = 7, Thread ID =
>>>> VirtualThread[#24]/runnable at ForkJoinPool-1-worker-2
>>>> task = 6, Thread ID =
>>>> VirtualThread[#23]/runnable at ForkJoinPool-1-worker-3
>>>> task = 8, Thread ID =
>>>> VirtualThread[#25]/runnable at ForkJoinPool-1-worker-3
>>>> task = 9, Thread ID =
>>>> VirtualThread[#27]/runnable at ForkJoinPool-1-worker-2
>>>> task = 3, Thread ID =
>>>> VirtualThread[#19]/runnable at ForkJoinPool-1-worker-4
>>>> task = 10, Thread ID =
>>>> VirtualThread[#28]/runnable at ForkJoinPool-1-worker-3
>>>> task = 11, Thread ID =
>>>> VirtualThread[#29]/runnable at ForkJoinPool-1-worker-2
>>>> task = 12, Thread ID =
>>>> VirtualThread[#31]/runnable at ForkJoinPool-1-worker-4
>>>> task = 13, Thread ID =
>>>> VirtualThread[#34]/runnable at ForkJoinPool-1-worker-5
>>>> item = 15, Thread ID = Thread[#1,main,5,main]
>>>> task = 14, Thread ID =
>>>> VirtualThread[#35]/runnable at ForkJoinPool-1-worker-5
>>>> task = 15, Thread ID =
>>>> VirtualThread[#42]/runnable at ForkJoinPool-1-worker-11
>>>> Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
>>>>
>>>> Process finished with exit code 0
>>>>
>>>
More information about the loom-dev
mailing list