Library additions for shell scripting

Jens Lideström jens at lidestrom.se
Sun Oct 8 10:16:05 UTC 2023


Hello, Core lib developers :)

Java is pretty decent for shell scripts style programs now days!

I'd like to suggest a couple of additions to the standard library to 
make it better.

## Background

By "shell script style programs" I mean programs that don't have a GUI 
and mostly perform these tasks: Move files around, read and write files, 
executes commands, execute external programs and fetch network files.

Simple shell scripts are often written in Bash or Batch. Slightly more 
complex scripts are often written in for example Python.

For an organisation that works mostly with Java it's an advantage to be 
able to instead use Java for these slightly complex scripts, so that 
developers can use their existing Java skills instead of learning an 
alternative language.

Historically Java has been a rather poor choice for shell scripts, 
because of need for a separate compile step, and the verbosity of the 
language and the relevant libraries.

Now days Java is have become reasonable choice, after the following 
additions:

* New file API based on Path and Files in Java 7.
* Addition and improvement of ProcessBuilder in Java 5-9.
* Addition of java.net.http.HttpClient in Java 11.
* Ability to run single file programs without a separate compilation 
step in Java 11.
* Features to write more succinct code, such as collections factories 
and records.

However: A few common tasks are still hard or unnecessary fiddly. Many 
of these tasks can be performed with external libraries such as Guava or 
Apache Commons, but for shell script style programs it's hard to include 
external libraries.

I think this could be helped by a few relatively minor additions to the 
standard library.

## Tasks that would benefit from improvements

Ordered approximately according to how important I think they are.

### 1. Wait for termination of an external process and get its output

* Wait for the termination of an external process, with a timeout, and 
then capturing its output as a string.

* The best current solution I have been able find is to use a Future to 
wait for the output.

* Example solution: 
https://stackoverflow.com/questions/41491327/how-to-get-the-output-from-a-process-and-set-a-timeout/73486466#73486466

* This can probably be fixed by some addition to ProcessBuilder.

### 2. Recursively delete a directory

* Recursively delete a directory, including its files and nested 
directories.

* The best current solution I have been able find: 
https://www.baeldung.com/java-delete-directory

### 3. Pack and unpack a zip file

* Create a zip file with the arbitrary contents of a directory, 
including nested directories.

* Unpack a full zip file including all files and nested sub-directories, 
then write it to disk.

* The best current solution I have been able find involves manually 
looping over all files or zip entries: 
https://www.baeldung.com/java-compress-and-uncompress#unzip

* This can often a done by calling an external command, but a pure Java 
solution would be better.

### 4. Report download process in HttpClient

* Some way to report how much of a large file that has been downloaded 
using HttpClient.

* A very minor improvement.

* The best current solution I have been able find involves creating a 
custom BodyHandler: https://stackoverflow.com/a/77250200/452775

### 5. List directory contents eagerly

* Currently there is `Files#list`, but the returned steam needs to be 
closed using a try-with-resources.

* This is minor but re-occurring annoyance. An eager version of this 
method would be a quality-of-life improvement.

* Guava has `MoreFiles#listFiles` for this.


### 6. Parse and generate JSON

* This is currently explored as JEP 198: https://openjdk.org/jeps/198

* I think a very simple solution would be sufficient for shell script 
style programs.


## Conclusion

I think a few small additions to the standard library could solve these 
problems and be useful both to shell script style programs as well as 
other Java programs.

Best Regargs,
Jens Lideström, Java and MUMPS developer


More information about the core-libs-dev mailing list