Fwd: Files.walk() is unusable because of AccessDeniedException

[timo.kinnunen at gmail.com]

Let’s not forget that in the use case where there is an installed security manager, the API clearly states how those access checks are supported. The results of access checks are communicated by the security manager using unchecked exceptions. As a result the Files::walk implementation has to carefully catch just the right unchecked exceptions in the right places so that a failed access check doesn’t escape and terminate the walk with an exception. That it does so is no accident or a random coincidence.

But that gives me an idea. If the Files::walk implementation categorically refuses to support any other access checks than security manager’s, then a workaround for this could come as a custom security manager. The security manager would deny any attempts to access a file that isn’t accessible and grant the rest.

This prototype implementation seems to work in some quick tests:

package utils.security;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.AccessMode;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

public class DenyAccessWhereNoAccessGrantedSecurityManager extends SecurityManager {
	public static void install() {
		try {
			Path path = Files.createTempFile("all", ".policy");
			Files.write(path, "grant {\n  permission java.security.AllPermission;\n};".getBytes(StandardCharsets.UTF_8));
			path.toFile().deleteOnExit();
			System.setProperty("java.security.policy", path.toString());
			System.setSecurityManager(new DenyAccessWhereNoAccessGrantedSecurityManager());
		} catch(IOException e) {
			throw new AssertionError("Cannot install security manager, aborting", e);
		}
	}
	private final ThreadLocal<Checker> readChecker = new Checker(AccessMode.READ);
	private final ThreadLocal<Checker> writeChecker = new Checker(AccessMode.WRITE);
	private class Checker extends ThreadLocal<Checker> {
		private final AccessMode mode;
		private boolean used;
		public Checker(AccessMode mode) {
			this.mode = mode;
		}
		protected @Override Checker initialValue() {
			return new Checker(mode);
		}
		public boolean check(String file) {
			if(!used && (used = true)) {
				try {
					FileSystems.getDefault().provider().checkAccess(Paths.get(file), mode);
				} catch(IOException e) {
					throw new SecurityException(e);
				} finally {
					((used = false) || mode == AccessMode.READ ? readChecker : writeChecker).remove();
				}
			}
			return !used;
		}
	}
	public @Override void checkRead(String file) {
		if(readChecker.get().check(file)) super.checkRead(file);
	}
	public @Override void checkRead(String file, Object context) {
		if(readChecker.get().check(file)) super.checkRead(file, context);
	}
	public @Override void checkWrite(String file) {
		if(writeChecker.get().check(file)) super.checkWrite(file);
	}
}

All that code boils down to a function doing binary classification with the response vocabulary consisting of either 1. throwing an exception, or 2. doing nothing.

With the workaround in place this stream lists my files and public folders successfully, skipping the rest. Without, it attempts to walk into some other user’s home folder and falls over when that doesn’t work:

package utils.security.test;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import utils.security.DenyAccessWhereNoAccessGrantedSecurityManager;

public class FileAccessMain {
	public static void main(String[] args) throws IOException {
		DenyAccessWhereNoAccessGrantedSecurityManager.install();
		Path path = Paths.get(System.getProperty("user.home")).getParent().toAbsolutePath();
		Files.walk(path, 2).sorted().forEach(System.out::println);
	}
}

As the security manager is totally dumb itself, it must be that this big change in behavior is all coming from the logic contained in the Files::walk implementation. A binary classifier like the one above should be easy enough to add to the code in Files::walk. This would then make it unnecessary to have a custom security manager. Which I probably implemented incorrectly anyways.






-- 
Have a nice day, 
Timo

Sent from Mail for Windows 10

From: Henry Jen
Sent: Wednesday, May 25, 2016 08:24
To: Andrew Haley; Gilles Habran
Cc: core-libs-dev at openjdk.java.net
Subject: Re: Fwd: Files.walk() is unusable because of AccessDeniedException

I think there is a work-around, use list() and flatMap() should get you what you needed.

The API is designed to walk a tree where you suppose to have access with. If OS level cause an IOException, that need to be dealt with. Acknowledged that exception handling is not a strong suite in Stream API, developer will need to do some work.

Files.find() also allows you to get entries and filter out by permission. What you can do is make sure you have permission on the top level, then call find with maxDepth 1 to only get entries on that directory.

Combined with flatMap(), you should be able to get what you want. Try the following code to see if it works for you.

import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.stream.Stream;
import java.io.IOException;

public class ListCanRead {
    static Stream<Path> walkReadable(Path p) {
        if (Files.isReadable(p)) {
            if (Files.isDirectory(p)) {
                try {
                    return Stream.concat(Stream.of(p), Files.list(p));
                } catch (IOException ioe) {
                    return Stream.of(p);
                }
            } else {
                return Stream.of(p);
            }
        }
        return Stream.of(p);
    }

    public static void main(String[] args) throws IOException {
        System.out.println("List directory: " + args[0]);
        walkReadable(Paths.get(args[0])).flatMap(ListCanRead::walkReadable)
            .forEach(System.out::println);

        Files.walk(Paths.get(args[0]))
            .forEach(System.out::println); // Could throw AccessDeniedException
    }
}

Cheers,
Henry

On May 24, 2016 at 4:48:30 PM, Gilles Habran (gilleshabran at gmail.com) wrote:
> Good morning,
>  
> well I would like to be able to manage the outcome of the processing myself
> and not get an exception in a part where I have no control.
>  
> For example, I would expect to get an exception when I tried to read a file
> where I don't have the permission. I would not expect to get an exception
> when Java creates the Stream.
>  
> Maybe I am the only one to have a problem with this ? I don't know but it
> feels strange to be forced to execute a software with root permissions
> where I don't even plan to read file I cannot read because of lack of
> permissions.
>  
> For me, we should be able to test the attributes of a file and depending on
> the result, read the file or not. If we read the file without permission,
> we get an exception, if not, we can go to the next file without error.
>  
> If that's unclear, please let me know, I'll try to give more informations
> or examples.
>  
> Thank you.
>  
> On 24 May 2016 at 10:19, Andrew Haley wrote:
>  
> > On 05/20/2016 10:38 AM, Gilles Habran wrote:
> > > why is my message still waiting for approval after a month ?
> >
> > What is it you want Java to do? You can't walk the directory
> > because you don't have permission. sudo should work.
> >
> > Andrew.
> >
> >
>