My experience using Java to do CLI Scripting

[forax at univ-mlv.fr]

----- Original Message -----
> From: "Stuart Marks" <stuart.marks at oracle.com>
> To: "Remi Forax" <forax at univ-mlv.fr>, "Ron Pressler" <ron.pressler at oracle.com>, "David Alayachew"
> <davidalayachew at gmail.com>
> Cc: "cay horstmann" <cay.horstmann at gmail.com>, "core-libs-dev" <core-libs-dev at openjdk.org>
> Sent: Tuesday, April 15, 2025 12:10:54 AM
> Subject: Re: My experience using Java to do CLI Scripting

> On 4/14/25 1:48 PM, Remi Forax wrote:
>>> From: "Ron Pressler" <ron.pressler at oracle.com>
>>> This does what you want (and could even be combined to a single expression):
>>>
>>>     Process p = new ProcessBuilder("ls", "-al").start();
>>>     String result = p.inputReader().lines().collect(Collectors.joining("\n"));
>>>
>>> and it’s even nicer in the cases where you may want to process the output line
>>> by line, as many scripts do.
>> 
>> Hi Ron,
>> i think you need to close the inputReader
> 
> A few points here...
> 
> It might be possible to get away without any Process-related cleanup. For one,
> there's no "close" method on Process or ProcessBuilder. The destroy() method
> will
> certainly clean up, but you don't want that; and waitFor() merely waits for
> termination but doesn't actually clean anything up.
> 
> At least in the Unix ProcessImpl class, there's a bunch of infrastructure to
> handle
> exit of the underlying process, drain input from its stdout/stderr, and close
> the
> pipes. (I didn't look on Windows.) So setting aside termination, IOException,
> and
> what to do with stderr, it seems possible to just get the inputReader() and do
> something with the characters it emits.

You mean by making the Process Closeable ?

> 
> Getting a stream of lines with lines() seems like a reasonable thing to do if
> you
> want to process the output line by line.

with the caveat that you have to close() the returned stream (like Files.lines()), something people will often forget.


> 
> There are other possibilities, taking a nod from Files, which has methods
> readAllLines() and readString(). Putting similar methods on some class in the
> Reader
> family might help considerably here. Or you could call Reader.transferTo(Writer)
> that will send the characters to any Writer that might have a useful
> destination.

+1 for readAllInputLines() and readInputString() on Process.

> 
> s'marks

regards,
Rémi


> 
> 
> 
> 
> 
>> 
>>    public static String output(String... args) {
>>      ProcessBuilder processBuilder = new ProcessBuilder(args);
>>      try {
>>        Process process = processBuilder.start();
>>        try (BufferedReader reader = process.inputReader()) {
>>          return reader.lines().collect(Collectors.joining("\n"));
>>        }
>>      } catch (IOException e) {
>>        throw new IOError(e);
>>      }
>>    }
>> 
>> 
>>>
>>> — Ron
>>>
>> 
>> regards,
>> Rémi
>> 
>>>
>>>
>>>> On 14 Apr 2025, at 20:06, Cay Horstmann <cay.horstmann at gmail.com> wrote:
>>>>
>>>> Absolutely, ProcessBuilder/Process is the right approach.
>>>>
>>>> I realize that all those bells and whistles in the Process API are there for a
>>>> reason, but the API is a bit clunky for the happy day path that one usually has
>>>> in a script: running a process until it terminates and getting its output. It
>>>> is trivial to write a couple of helper methods, but it might be nice if the
>>>> Process API could help out. Something like
>>>>
>>>> Process p = Process.waitFor("ls", "-al");
>>>> String result = p.output();
>>>>
>>>> Cheers,
>>>>
>>>> Cay
>>>>
>>>> PS. This isn't pretty in Python either:
>>>> https://docs.python.org/3/library/subprocess.html#subprocess.run
>>>>
>>>>
>>>> Il 12/04/25 17:02, Ron Pressler ha scritto:
>>>>> Hi.
>>>>> Let’s focus on ProcessBuilder and Process (as I think that’s where you want to
>>>>> focus, and why I think this discussion is more appropriate for core-libs-dev).
>>>>> Can you try to show more concretely what the pain point is in your actual code?
>>>>> The ProcessBuilder example is long because it does multiple things, each of
>>>>> which may or may not be relevant to your use case. That doing five different
>>>>> things requires five lines of code doesn’t help us see where your specific pain
>>>>> point is.
>>>>> — Ron
>>>>>> On 12 Apr 2025, at 15:39, David Alayachew <davidalayachew at gmail.com> wrote:
>>>>>>
>>>>>> Hello Amber Dev Team and Kulla Dev Team,
>>>>>>
>>>>>> (I made a reddit post too, if you prefer to interact there instead --
>>>>>> https://www.reddit.com/r/java/comments/1jx87ys/)
>>>>>> The following JEP's have released recently.
>>>>>>      • JEP 495: Simple Source Files and Instance Main Methods
>>>>>>      • JEP 330: Launch Single-File Source-Code Programs
>>>>>>      • JEP 222: jshell: The Java Shell (Read-Eval-Print Loop)
>>>>>> These have made it really easy for me to do CLI scripting in Java, as opposed to
>>>>>> Bash. However, I've run into some pain points, as I've relied more and more on
>>>>>> Java.
>>>>>> For starters, the hand off from Java --> Bash is kind of ugly. Bash --> Java is
>>>>>> not bad, due to void main(final String[] args), as well as Bash's xargs. But
>>>>>> Java --> Bash is ugly, and here is an example demonstrating how/why.
>>>>>> I use AWS CLI to manage my dev environment. It's super powerful, and is all
>>>>>> available directly from the CLI, using simple Bash or even CMD.
>>>>>> Let's say I use AWS CLI to gather some ad-hoc information about my entire dev
>>>>>> environment. How do I manage the multiple handoffs back and forth between AWS
>>>>>> CLI and Java?
>>>>>> There are no good answers.
>>>>>>      • Store the results into a file, then use JShell/java(c) to process the output
>>>>>>      file from Bash/AWS CLI.
>>>>>>          • There's multiple handoffs back and forth between AWS CLI and Java. So, every
>>>>>>          handoff from Java ---> AWS CLI means generating a new file, thus increasing the
>>>>>>          complexity and cruft. It's unideal.
>>>>>>      • Use Java's ProcessBuilder and Process classes.
>>>>>>          • This works, but is heavy-handed. Look at the examples in those links. That is
>>>>>>          multiple lines of code to represent a single bash command. It does appear to be
>>>>>>          the idiomatic way, though.
>>>>>>      • Do all upstream processing with AWS CLI in Bash directly, then do only a
>>>>>>      single handoff to Java, once I have done all I need to with AWS CLI.
>>>>>>          • This is definitely the least painful, but it also means I don't use much Java
>>>>>>          at all. And any changes in upstream processing must be done in Bash to avoid
>>>>>>          handoff headaches from AWS CLI ---> Java.
>>>>>>      • Download the AWS SDK Jar files and just do it all in Java.
>>>>>>          • Ignoring the fact that some things are much harder to do via the AWS Java
>>>>>>          SDK's, there's actually some functionality that just isn't available via the
>>>>>>          Java ones. I'd have to recreate it myself, and it would be a significant lift.
>>>>>> Option 4 is best when I am building an application, but for ad-hoc checks that I
>>>>>> want to do on the fly (my most common use-case by far), I have been using
>>>>>> Option 3.
>>>>>> I just wish I could use more Java. It's a FAR BETTERtool than Bash, but I can't
>>>>>> justify the level of effort for ad-hoc use cases because of the poor hand off
>>>>>> from Java --> Bash. And since AWS CLI is only available via Bash/CMD, I'm stuck
>>>>>> with a bunch of not-good choices.
>>>>>> CLI Scripting in Java is great, but I wanted to highlight this pain point to
>>>>>> spread awareness.
>>>>>> Can you relate?
>>>>>>
>>>>
>>>> --
>>>>
> >>> Cay S. Horstmann | https://horstmann.com