Hi Roger, Well if there is some developer interest in it and it would consolidate and make code more robust in the JDK then it seems worth pursuing. Thanks, Brian On Jul 19, 2018, at 1:30 PM, Roger Riggs <Roger.Riggs@Oracle.com> wrote:

No particular novel use cases. There are quite a few places in OpenJDK tools, etc that naively use string.split(pathSeparator) that will misbehave on Windows in the edge cases and may not be paying attention to the empty path component case. It would provide a measure of regularity and predictability to path parsing. There was a query on stack Overflow about such an API.

Hi Jon, I agree that supporting Paths leads people in the right direction. Two considerations led me to propose the simpler String version first. 1.  Early parsing of paths can occur before the file systems that support Path have been initialized. 2.  There is an additional exception that can occur when creating a Path (InvalidPathException) and the caller may want to handle it individually. Adding another method that returns List<Path> is a straightforward addition. Keeping the string version may make adoption easier. Thanks, Roger On 7/23/18 1:53 PM, Jonathan Gibbons wrote:

Roger,

Having written internal library code for both javac and jtreg to do what you suggest, I would support such an API.

I note that as well as parsing system properties, it would also be a useful API for command-line tools that accept paths as options.

However, I would suggest that it is more appropriate to return a List<Path> than a List<String>.  It is better to bias the API towards an appropriate type (i.e. Path) than to leave it to the user to create a stream to do the conversion.   Those folk that still want a File can still create the stream to do the conversion if they so desire.

By converting to Paths as part of the API, it becomes a clearer point to specify and document the possibility of InvalidPathException.

-- Jon

On 7/19/18 12:39 PM, Roger Riggs wrote:

...

Hi,

Parsing of paths such as the values of properties java.class.path and java.library.path do not have  direct support in the Java API. There are several implementations within OpenJDK and a few variations exist with regard to treatment of empty path elements and on Windows support for quoting.

Would it be useful to have an API such as the following:

    /**      * Returns a list of paths parsed from a string separated by {@link File#pathSeparator}.      * All characters except the {@link File#pathSeparator} are allowed.      * Some operating systems support quoting segments of the string      * potentially including {@code pathSeparator} characters.      * Empty paths, identified by leading, trailing, and adjacent separator characters,      * can be omitted or replaced with a non-null provided path.      *      * @implNote      * On Windows, zero or more characters between double-quotes ({@code "}) are included      * literally in the path, regardless of whether they are valid in a path.      *      * @param path a string containing paths separated by the {@link File#pathSeparator}      * @param emptyPath a path to replace an empty path or {@code null} to ignore empty paths      * @return an immutable list of strings for each path; {@code non-null}      */     public static List<String> parsePathStrings(String path, String emptyPath) { ...}

There are of course, possible variations and convenience versions but the API should be kept to the essentials. With java.util.streams it becomes easy to process each component or to map the strings to Files or Path.

Suggestions welcome, Roger

Please review the API and implementation of an API to parse Path strings. Two methods are added to java.nio.file.Paths to parse a string using the path separator delimiter and return either List<String> or List<Path>.  Empty path elements are ignored. For compatibility with current URLClassPath behavior the internal implementation handles replacement of empty paths. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-8207690_parsing_api_for_classpath_... CSR:   https://bugs.openjdk.java.net/browse/JDK-8208208 Thanks, Roger

On 10/09/2018 19:28, Jonathan Gibbons wrote:

Roger,

You've run into the standard naming ambiguity problem of "when is a path a search path, with elements separated by File.pathSeparator (e.g. class path, source path), and when is it a file path, with elements separated by File.separator (e.g. a nio.file.Path identifying a file or directory)?"

This shows up most obviously in the method confusingly named "pathToPaths".

In other contexts (javac, jtreg, etc) I've tried to use the term "search path" to describe the string that is a sequence of elements separated by File.pathSeparator. I agree the naming is confusing as this API is about parsing a search path into its components. The Instrumentation API is another example where "Search" was put into the method name.

-Alan

Roger, I had in mind your first "disambiguate" suggetion (searchPathToStrings, searchPathToPaths) but I note your initial comment "Is the search path the input string or the return value? Well its both, but with different structure. " Since we have Paths.get how about List<Path> Paths.getList(String searchPath) I'd be less inclined to give formal support to converting to a List<String>. Those folk that want that could/should be using String.split(File.pathSeparator) or something like that. -- Jon On 09/10/2018 01:34 PM, Roger Riggs wrote:

Hi Jon, Alan,

No surprise and suggestions welcome! :)

Since we don't have return type overloads, the name has to encode the return type (or the important aspects of it). Is the search path the input string or the return value? Well its both, but with different structure.

Remove the ambiguity:

* a) Drop one of the APIs, leaving only the return of List<String> and push the conversion to Path to the consumer. e.g. List<Path> files = toSearchPath(String).stream().map(s -> Paths.of(s)).collect(Collectors.toList()); * b) Drop the other API, always doing the conversion to Path and throwing exceptions. e.g. List<Path> toSearchPath(String); The Caller can get the string or file by calling toString() or toFile(); potentially wasting the effort to check the path syntax.

Disambiguate:

* List<String> searchpathToStrings(s); List<Path> searchpathToPaths(s); or var files = Paths.searchpathToStrings("a;b;c"); var searchpath = Paths.searchpathToPaths("x;y;z"); * List<Path> toPathList(String searchpath); List<String> toStringList(String searchpath) Without static imports looks like: var files = Paths.toStringList("a;b;c").stream().... var searchpath = Paths.toPathList("x;y;z"); * ???

Thanks, Roger

On 9/10/2018 2:28 PM, Jonathan Gibbons wrote:

...

Roger,

You've run into the standard naming ambiguity problem of "when is a path a search path, with elements separated by File.pathSeparator (e.g. class path, source path), and when is it a file path, with elements separated by File.separator (e.g. a nio.file.Path identifying a file or directory)?"

This shows up most obviously in the method confusingly named "pathToPaths".

In other contexts (javac, jtreg, etc) I've tried to use the term "search path" to describe the string that is a sequence of elements separated by File.pathSeparator.

-- Jon

On 9/10/18 11:16 AM, Roger Riggs wrote:

...

Please review the API and implementation of an API to parse Path strings. Two methods are added to java.nio.file.Paths to parse a string using the path separator delimiter and return either List<String> or List<Path>. Empty path elements are ignored.

For compatibility with current URLClassPath behavior the internal implementation handles replacement of empty paths.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-8207690_parsing_api_for_classpath_...

CSR: https://bugs.openjdk.java.net/browse/JDK-8208208

Thanks, Roger

OK, I understand the need for quoting, although I wonder how many folk have bothered with that up to now (looking at javac, jtreg!) ;-) I'd stay with List<Path> getList(String searchPath) and then either have people use getList(searchPath).stream().map(Path::toString).collect(Collectors.toList()) or if we want the extra method, then List<String> getListAsStrings(String searchPath) -- Jon On 09/10/2018 01:55 PM, Roger Riggs wrote:

Hi Jon,

On 9/10/2018 4:47 PM, Jonathan Gibbons wrote:

...

Roger,

I had in mind your first "disambiguate" suggetion (searchPathToStrings, searchPathToPaths) but I note your initial comment "Is the search path the input string or the return value? Well its both, but with different structure. "

Since we have Paths.get

how about List<Path> Paths.getList(String searchPath) Not bad, but...

Perhaps, List<String> Paths.getListAsStrings(String searchPath)...

...

I'd be less inclined to give formal support to converting to a List<String>. Those folk that want that could/should be using String.split(File.pathSeparator) or something like that.

Nope! there's quoting on Windows that gets short changed with that work around.

Other opinions?, Suggestions?

Thanks, Roger

...

-- Jon

On 09/10/2018 01:34 PM, Roger Riggs wrote:

...

Hi Jon, Alan,

No surprise and suggestions welcome! :)

Since we don't have return type overloads, the name has to encode the return type (or the important aspects of it). Is the search path the input string or the return value? Well its both, but with different structure.

Remove the ambiguity:

* a) Drop one of the APIs, leaving only the return of List<String> and push the conversion to Path to the consumer. e.g. List<Path> files = toSearchPath(String).stream().map(s -> Paths.of(s)).collect(Collectors.toList()); * b) Drop the other API, always doing the conversion to Path and throwing exceptions. e.g. List<Path> toSearchPath(String); The Caller can get the string or file by calling toString() or toFile(); potentially wasting the effort to check the path syntax.

Disambiguate:

* List<String> searchpathToStrings(s); List<Path> searchpathToPaths(s); or var files = Paths.searchpathToStrings("a;b;c"); var searchpath = Paths.searchpathToPaths("x;y;z"); * List<Path> toPathList(String searchpath); List<String> toStringList(String searchpath) Without static imports looks like: var files = Paths.toStringList("a;b;c").stream().... var searchpath = Paths.toPathList("x;y;z"); * ???

Thanks, Roger

On 9/10/2018 2:28 PM, Jonathan Gibbons wrote:

...

Roger,

You've run into the standard naming ambiguity problem of "when is a path a search path, with elements separated by File.pathSeparator (e.g. class path, source path), and when is it a file path, with elements separated by File.separator (e.g. a nio.file.Path identifying a file or directory)?"

This shows up most obviously in the method confusingly named "pathToPaths".

In other contexts (javac, jtreg, etc) I've tried to use the term "search path" to describe the string that is a sequence of elements separated by File.pathSeparator.

-- Jon

On 9/10/18 11:16 AM, Roger Riggs wrote:

...

Please review the API and implementation of an API to parse Path strings. Two methods are added to java.nio.file.Paths to parse a string using the path separator delimiter and return either List<String> or List<Path>. Empty path elements are ignored.

For compatibility with current URLClassPath behavior the internal implementation handles replacement of empty paths.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-8207690_parsing_api_for_classpath_...

CSR: https://bugs.openjdk.java.net/browse/JDK-8208208

Thanks, Roger

On 11 Sep 2018, at 10:24, Alan Bateman <Alan.Bateman@oracle.com> wrote:

On 10/09/2018 21:55, Roger Riggs wrote:

...

Nope! there's quoting on Windows that gets short changed with that work around.

Other opinions?, Suggestions? One suggestion is reduce this down to one method that returns a stream rather than a collection. It could work lazily if you want. Something like:

Stream<String> splitSearchPath(String input)

I agree with Alan, this seems like the lowest-order primitive. I refreshed my memory of search path splitting from the `jmod` tool, where such an API would be useful ( if we weren’t using joptsimple ). There are two separate use cases, ClassPathConverter, and DirPathConverter ( in JmodTask ), both of which resolve each String path value against the current working directory, and do some context specific validation. -Chris.

On 11 Sep 2018, at 14:18, Roger Riggs <roger.riggs@oracle.com> wrote:

Hi Chris,

On 9/11/18 7:50 AM, Chris Hegarty wrote:

...

...

On 11 Sep 2018, at 10:24, Alan Bateman <Alan.Bateman@oracle.com> wrote:

On 10/09/2018 21:55, Roger Riggs wrote:

...

Nope! there's quoting on Windows that gets short changed with that work around.

Other opinions?, Suggestions? One suggestion is reduce this down to one method that returns a stream rather than a collection. It could work lazily if you want. Something like:

Stream<String> splitSearchPath(String input) I agree with Alan, this seems like the lowest-order primitive. How can that be? The Stream creates a lot more objects and takes even more to use it.

I guess it depends on the use-case that you are trying to address. In the cases that I sited, a Stream would likely be more appropriate since the calling code requires to iterate over the entries, rather than store them as-is. You are probably thinking of different use-cases.

It can't be stored or re-used and has higher overhead that can't easily be understood.

.collect(toList()) - which, again, depending on use-case may be more or less common than entries.stream().

...

I refreshed my memory of search path splitting from the `jmod` tool, where such an API would be useful ( if we weren’t using joptsimple ). There are two separate use cases, ClassPathConverter, and DirPathConverter ( in JmodTask ), both of which resolve each String path value against the current working directory, and do some context specific validation. Command line tools are more likely to resolve against the current directory List<String> is least common but still pushes more to the developer.

Exactly. My comments are from the perspective of writing a command line tool, since that is the only direct experience I have with such functionality. You may have different use- cases in mind. -Chris.

Hi, My original intention was to provide minimal support for parsing a property value and handling OS specific behaviors (quoting). java.nio.file.Paths is a bit of backwater (now that Path.of was added) and its factories are dedicated to handling Paths. I considered a new type representing a search path but that seemed to expand the scope beyond the minimum. Search path logic is not one of the high profile functions used by lots of developers and there seem to be many variations on how they are used. It is also a slippery slope since it could easily expand to add many useful functions that applications already deal with themselves.  - Searching the path for a particular file or directory  - Supporting patterns for matching files on the path, using filters or glob patterns  - Caching of previous lookups or directory contents  - Optimizing the search path, ignoring duplicate entries, etc. If that's the direction this should go it expands the scope quite a bit. Thanks, Roger On 9/11/18 12:49 PM, mark.reinhold@oracle.com wrote:

2018/9/11 4:50:49 -0700, chris.hegarty@oracle.com:

...

...

On 11 Sep 2018, at 10:24, Alan Bateman <Alan.Bateman@oracle.com> wrote: On 10/09/2018 21:55, Roger Riggs wrote:

...

Nope! there's quoting on Windows that gets short changed with that work around.

Other opinions?, Suggestions? One suggestion is reduce this down to one method that returns a stream rather than a collection. It could work lazily if you want. Something like:

Stream<String> splitSearchPath(String input) I agree with Alan, this seems like the lowest-order primitive. Agreed, but it begs a question that applies to all these alternatives: Why does this API belong in java.nio.file.Paths?

In other words, search paths are a different abstraction from filesystem paths. Should they have their own class, even if it only winds up with a couple of static utility methods? java.nio.file.SearchPath?

public class SearchPath { public static Stream<String> splitToStream(String searchPath); public static List<Path> splitToPaths(String searchPath); public static String join(Collection<String>); }

- Mark

On 11/09/2018 14:09, Roger Riggs wrote:

Streams are fine and easy to get from a List; but overall are a much higher overhead than the list itself. And they can't be stored or re-used. Which would mean re-parsing the string again.

I think returning Stream<String> will provide the most flexibility. The example usages in the webrev mostly want the elements in an array or they want to perform an action for each element and that is easily done with pipelines such as: splitSearchPath(input).map(Path::of).toArray(Path[]::new) splitSearchPath(input).map(Path::of).forEach(...) If something wants to elements in a List or needs to cache the result then they can use a collector. I agree we might need something special to optimize JDK startup but I don't think these cases should influence the API. -Alan

On Tue, 11 Sep 2018 at 10:25, Alan Bateman <Alan.Bateman@oracle.com> wrote:

On 10/09/2018 21:55, Roger Riggs wrote:

...

Nope! there's quoting on Windows that gets short changed with that work around.

Other opinions?, Suggestions? One suggestion is reduce this down to one method that returns a stream rather than a collection. It could work lazily if you want. Something like:

Stream<String> splitSearchPath(String input)

is a lot more flexible (in my view) and the places where we use Path objects can use .map(Path::of).

This is a broader question for new methods in the JDK, not just this one. I don't think anyone has come up with a "style guide" for when to use Stream returning as opposed to list-returning methods. What I can say is that I think List is the better fit here, because: - the result is not large - the result may well be stored somewhere (method should guarantee result is immutable) - a List is simpler conceptually than Stream as a result type (eg. serial vs parallel) Personally, I only tend to return Stream if I the particular method is returning a very large data set. Stephen

On 11/09/2018 17:04, Stephen Colebourne wrote:

: This is a broader question for new methods in the JDK, not just this one. I don't think anyone has come up with a "style guide" for when to use Stream returning as opposed to list-returning methods.

What I can say is that I think List is the better fit here, because: - the result is not large - the result may well be stored somewhere (method should guarantee result is immutable) - a List is simpler conceptually than Stream as a result type (eg. serial vs parallel)

Personally, I only tend to return Stream if I the particular method is returning a very large data set.

There are several discussion points around this aspect of API design. One of the most important questions to ask is whether a stream or collection is more useful to the caller. For the API under discussion then we have several examples in the JDK that process the class path or module path. One example involves mapping the elements of the class path to file URLs and into an array to create a URLClassLoader. Another maps the elements to Path objects and into an array to create a ModuleFinder. Another maps the elements to Path objects and performs an action on each element. Stuart brings up the empty path case which, depending on context, may need to be filtered. The examples so far iterate over the elements once and an intermediate/cached collection isn't needed. On the other hand, I think Roger's main use-case is representing the value of the java.class.path property as a List<Path> which will may involve caching an unmodifiable list. -Alan

+1 -- Jon On 9/13/18 10:21 AM, Roger Riggs wrote:

Hi,

Defining a SearchPath class does have a number of benefits as Mark outlined.

Consider:

public class SearchPath {     public static SearchPath of(String searchPath) {...} public static SearchPath of(List<String> elements) {...} public Stream<String> stream() {...} public List<String> asList() {...}     public String toString() {...}     private SearchPath() {}; }

A SearchPath can be constructed from various forms of search path elements and can create other forms as needed. As a class it would be extensible and can start small and grow as needed.

Examples:         List<String> list = SearchPath.of("a:b:c").asList();

    Path p = SearchPath.of("x:y:z").stream()          .filter(Predicate.not(String::isEmpty))          .map(Path::of)              .filter(Files::isDirectory)              .filter(q -> Files.exists(q.resolve("x.jar)))          .findFirst()          .orElseThrow();

If that seems like a reasonable base, I (or some other volunteer) can flesh it out with the suggestions.

Thanks, Roger

On 9/13/2018 3:33 AM, Alan Bateman wrote:

...

On 11/09/2018 17:04, Stephen Colebourne wrote:

...

: This is a broader question for new methods in the JDK, not just this one. I don't think anyone has come up with a "style guide" for when to use Stream returning as opposed to list-returning methods.

What I can say is that I think List is the better fit here, because: - the result is not large - the result may well be stored somewhere (method should guarantee result is immutable) - a List is simpler conceptually than Stream as a result type (eg. serial vs parallel)

Personally, I only tend to return Stream if I the particular method is returning a very large data set.

There are several discussion points around this aspect of API design. One of the most important questions to ask is whether a stream or collection is more useful to the caller. For the API under discussion then we have several examples in the JDK that process the class path or module path. One example involves mapping the elements of the class path to file URLs and into an array to create a URLClassLoader. Another maps the elements to Path objects and into an array to create a ModuleFinder. Another maps the elements to Path objects and performs an action on each element. Stuart brings up the empty path case which, depending on context, may need to be filtered. The examples so far iterate over the elements once and an intermediate/cached collection isn't needed. On the other hand, I think Roger's main use-case is representing the value of the java.class.path property as a List<Path> which will may involve caching an unmodifiable list.

-Alan

Is the wildcard character allowed in the input? I'm thinking of the wildcard support in classpath. --Max

On Sep 11, 2018, at 2:16 AM, Roger Riggs <Roger.Riggs@oracle.com> wrote:

Please review the API and implementation of an API to parse Path strings. Two methods are added to java.nio.file.Paths to parse a string using the path separator delimiter and return either List<String> or List<Path>. Empty path elements are ignored.

For compatibility with current URLClassPath behavior the internal implementation handles replacement of empty paths.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-8207690_parsing_api_for_classpath_...

CSR: https://bugs.openjdk.java.net/browse/JDK-8208208

Thanks, Roger

https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html... This is not about the wildcard parsed by the shell. The bug synopsis mentions "classpath" and I am just curious if it covers this feature. I don't have my own user case for it. --Max

On Sep 11, 2018, at 10:48 PM, Roger Riggs <roger.riggs@oracle.com> wrote:

Hi,

What would the use case be for that?

A search path is usually just a sequence of file paths; wildcard expansion is not includes. Wildcards are significant only in shell contexts, though the expansion is done by the shell on Unix and by the application on Windows; it would add a fair bit of complexity.

Roger

On 9/11/18 10:44 AM, Weijun Wang wrote:

...

Is the wildcard character allowed in the input?

I'm thinking of the wildcard support in classpath.

--Max

...

On Sep 11, 2018, at 2:16 AM, Roger Riggs <Roger.Riggs@oracle.com> wrote:

Please review the API and implementation of an API to parse Path strings. Two methods are added to java.nio.file.Paths to parse a string using the path separator delimiter and return either List<String> or List<Path>. Empty path elements are ignored.

For compatibility with current URLClassPath behavior the internal implementation handles replacement of empty paths.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-8207690_parsing_api_for_classpath_...

CSR: https://bugs.openjdk.java.net/browse/JDK-8208208

Thanks, Roger

Clarified. So classpath can call this new method first and then expand * to all JAR files. This is fine. Thanks Max

On Sep 11, 2018, at 11:05 PM, Roger Riggs <roger.riggs@oracle.com> wrote:

Hi Max,

Right, that is a description of the shell environment on Windows and how the java launcher behaves. On Windows the launcher expands wildcards when evaluating classpath from the environment and the command line args. After they are expanded the java.class.path property is set. The proposed API can be used with other paths and does not expand wildcards.

Regards, Roger

On 9/11/18 10:53 AM, Weijun Wang wrote:

...

https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html...

This is not about the wildcard parsed by the shell. The bug synopsis mentions "classpath" and I am just curious if it covers this feature. I don't have my own user case for it.

--Max

...

On Sep 11, 2018, at 10:48 PM, Roger Riggs <roger.riggs@oracle.com> wrote:

Hi,

What would the use case be for that?

A search path is usually just a sequence of file paths; wildcard expansion is not includes. Wildcards are significant only in shell contexts, though the expansion is done by the shell on Unix and by the application on Windows; it would add a fair bit of complexity.

Roger

On 9/11/18 10:44 AM, Weijun Wang wrote:

...

Is the wildcard character allowed in the input?

I'm thinking of the wildcard support in classpath.

--Max

...

On Sep 11, 2018, at 2:16 AM, Roger Riggs <Roger.Riggs@oracle.com> wrote:

Please review the API and implementation of an API to parse Path strings. Two methods are added to java.nio.file.Paths to parse a string using the path separator delimiter and return either List<String> or List<Path>. Empty path elements are ignored.

For compatibility with current URLClassPath behavior the internal implementation handles replacement of empty paths.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-8207690_parsing_api_for_classpath_...

CSR: https://bugs.openjdk.java.net/browse/JDK-8208208

Thanks, Roger

Hi Roger,

110 * Returns a list of path strings parsed from a string with empty paths removed.

The Unix shell and the Java launcher's -classpath processing treat an empty path entry as the current working directory. (I don't know what happens on Windows.) Removing empty paths thus would seem to change the semantics of search paths, at least on some systems. s'marks On 9/10/18 11:16 AM, Roger Riggs wrote:

Please review the API and implementation of an API to parse Path strings. Two methods are added to java.nio.file.Paths to parse a string using the path separator delimiter and return either List<String> or List<Path>.  Empty path elements are ignored.

For compatibility with current URLClassPath behavior the internal implementation handles replacement of empty paths.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-8207690_parsing_api_for_classpath_...

CSR:   https://bugs.openjdk.java.net/browse/JDK-8208208

Thanks, Roger

On 09/12/2018 04:30 PM, Roger Riggs wrote:

Hi Stuart,

The implementation retains the previous handling of empty path elements for URLClassPath in the implementation.  The spec for the new methods is explicit about dropping empty elements.

For a library API, it is inadvisable to support implicit context such as the current working directory. I received some offline comments about proposing too many methods and dropped the variants that supported replacing empty path elements with an explicit path.

Keeping the empty path elements would simplify the spec though.

...and it's easy to .filter() them out if the parsing method returns Stream<String>... Regards, Peter

Thanks, Roger

On 9/11/18 4:54 PM, Stuart Marks wrote:

...

Hi Roger,

...

 110 * Returns a list of path strings parsed from a string with empty paths removed.

The Unix shell and the Java launcher's -classpath processing treat an empty path entry as the current working directory. (I don't know what happens on Windows.) Removing empty paths thus would seem to change the semantics of search paths, at least on some systems.

s'marks