hg: panama/panama/jdk: Summary: push of version-2 prototype of Arrays2.0. Work remains, comment is requested.

[Samuel Audet]

Hello, David,

On 10/08/2014 11:41 AM, David Chase wrote:
> And it’s useful feedback to hear “I love the idea of bulk operations with built-in
> fork/join and blocking” or “I really really need the ability to lock ranges of an array”
> or “volatile access to array elements is very important to me”.

About the above, I believe I would have something to contribute at this 
point. My main field of expertise is basically computer vision, and the 
most commonly used environment for development is MATLAB. Apart from the 
fact that everyone else uses it and is sort of a lingua franca, it is 
useful mainly for three reasons:
1. Convenient IDE with command line interface and GUI API for data 
visualization
2. Fast operations on matrices, commonly used to process images
3. Extensive libraries for image processing, statistics, etc

However, whenever we need to do something that does not fit into the 
MATLAB way of doing things, aka "vectorization", performance suffers 
dramatically, and that is a real problem when processing images, so 
people usually fall back to C/C++. There is also R or Python with NumPy 
and SciPy, but they suffer from the same issue as MATLAB when we start 
needing to do anything slightly out of the ordinary: Things slow down to 
a crawl, thus the inevitable fallback to C/C++, even for simple prototyping!

In my opinion, Java can offer a better development environment for tasks 
related to computer vision. So, why is everyone not processing images 
with Java? What is missing from Java? In fact, not much really, but 
things need to come together in a coherent and supported manner. 
Hopefully, Java 9 is going to come with a REPL and help cover the first 
point above. As for fast matrix operations, there has been a few 
packages in the past for that, and I hope "Arrays 2.0" can fill part of 
that gap, but things like http://nd4j.org/ still pop up now and then, 
and for computer vision, something like OpenCV is often sufficient, so 
that is why I came up with my own low-overhead wrappers in JavaCV, which 
incidentally has been rebased on what I called the JavaCPP Presets in an 
attempt to cover the third point, a condition that applies to other 
fields too and also one of the goals of Project Panama, if I understand 
correctly.

As far as computer vision is concerned, Java offers a more balanced 
foundation than any other solution out there. (Although now with .NET 
going open source...) It is high-level enough to perform simple 
operations almost as easily as in MATLAB or Python, while not suffering 
from their extreme drops in performance that basically forces everyone 
to become C/C++ coders. But we need to build on that foundation...

To illustrate my point, I included a new "indexer" package in the latest 
release of JavaCPP that allows easy-to-use but efficient access to 
multidimensional native arrays:
	http://bytedeco.org/news/2014/12/23/third-release/
(The sample code could be further simplified by using new functionality 
in Java SE 8, but I am limiting myself to Java SE 6, and now that I 
think about it we should also provide unsigned versions of ByteIndexer 
and ShortIndexer.) Anyway, that is not something that we could do in 
MATLAB or Python with any sort of efficiency (although MathWorks is 
continuously improving their JIT compiler, while PyPy and their efforts 
on software transactional memory seem vaguely promising), but loops and 
lambda expressions like that are pretty much the bread and butter of 
image processing, so they need to be fast. Technically, it is possible, 
because this is fast:
	http://www.scalanlp.org/
However, it requires Scala, which is not nearly as popular as the Java 
language itself. C/C++ is fast and popular, but it does not offer a 
REPL. Physics guys have been working on this forever:
	https://root.cern.ch/
And still, either that particular project or the concept of interpreted 
C/C++ has not caught up in any other fields that I know of, for 
technical reasons (awkwardness, complexity, portability, etc), I gather, 
but I do not believe those same reasons apply to Java.

So, Java? Yes, please! :) Let me know how I can help.

Samuel