Eliminating the security overhead when not running with a security manager

Tue Nov 21 00:48:18 UTC 2017

One thing that springs to mind.  Some allowance would have to be made
for domain combiners and JAAS Subject propagation: this mechanism also
uses access control contexts, to its own great detriment.  I would say
that the JAAS domain combiner strategy should be dropped in favor of a
simple thread local holding the Subject, regardless of what else is
decided.

I do personally believe that using a security manager as some kind of
proof against untrusted Java code is a mistake.  However I think one
important use case of security managers is where the Java code being
run actually _is_ trusted, but you want to avoid "weird machine"
effects where server code could be exploited (by way of malformed
requests or whatever) to perform actions that are unexpected but also
technically allowed (for example, if your file management abstraction
is only ever intended to modify "/app/user-storage/-" then you don't
want it to be able to access "/app-server/secret-passwords", and if
that happens, you maybe want to start yelling a lot).  In this case
it's an extra layer of protection.  I don't see this use case going
away or being invalidated in any way; an application server request
_is_ code, in a way, that must be interpreted and "executed" after a
fashion in order to do something useful for the user.

In terms of approach: back when the discussion around StackWalker
design was still going on, it seemed to me that the performance costs
of the security manager could be mostly shifted to the security
manager itself (thus solving the problem in a different way), if it
would walk the stack to do access checking directly, like this:

• An "acc" field on Thread is used to hold the inherited access control context
• AccessController.doPrivileged(task) becomes empty other than to
clear the acc field and delegate to the called block (and restore the
acc field after)
• AccessController.doPrivileged(task, acc) becomes empty other than to
set the acc field to the given ACC argument and delegate to the called
block (and restore the acc field after)
• AccessController.checkPermission() iterates the call stack until it
hits one-frame-lower-than-doPrivileged or the bottom of the stack,
checking access along the way for each protection domain (with some
simple duplicate elimination as it walks), and once it hits the end,
it calls checkPermission on the inherited acc field in Thread (if
any).
• Creating a new thread still captures the caller acc but then copies
it into the new thread's inherited acc field
• Construction/compilation of the ACC could possibly be moved to
userspace, which will reduce any memory overhead of having it baked in
to the native JVM code when it's not actually used
• The doPrivileged(action, acc, perms) case could be handled with
another Thread field which could be checked before or after the
primary check (in this case all the doPrivileged methods would have to
stash and set this field as well, and restore it after)
• A "shared secret" is added to let AccessController access the two
Thread cache field(s)

This way the cost of doPrivileged is greatly minimized (two Thread
field reads and writes before the block, and two Thread field writes
after the block, assuming the aforementioned second field for array
permissions) without affecting security manager functionality should
it happen to be enabled in software.  Also it's easy to improve the
security manager debug output, such that you could get a report on
_all_ classes or modules or codeSources or whatever which would do not
grant a given permission (instead of just the first one), which is
greatly useful in reducing iterations in the design and debug stages.

When the security manager is actually enabled, only testing would
reveal the comparative performance of a stack walker based approach
versus the current native ACC optimization code (maybe C2 can work
some magic here).  There's probably some kind of disgusting weirdness
around method handles and reflection things on the call stack that
would have to be examined as well.  Finally, it will (as always) be a
challenge to craft a Stream-consuming Function that doesn't allocate
scads of objects when actually iterating the thread stack.  Maybe we
could allow an Iterator-based StackWalker method.  Maybe I'll keep
dreaming.

Anyway I never got a chance to prototype this, but it might be a fun
option worth exploring.  I found the idea of moving this stuff all to
user space to be very appealing (due in no small part to the idea that
it could potentially be examined and analyzed by a much larger
audience, being Java code). It also hints at the possibility of a
fully "user space" replacement of the security manager concept (much
of the remaining cost lives in the structure of AccessControlContext,
which is based on an array of ProtectionDomain objects; this is
definitely non-ideal and could possibly be hidden behind a smarter
abstraction).

I really firmly believe that domain combiners should be ultimately
eliminated regardless of whatever else happens as they add a lot of
complexity in an area where you really don't want complexity, and I
think we've already seen specific negative effects of this that have
needed fixing in various ways (recently even).

On Mon, Nov 20, 2017 at 7:18 AM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
>
> One of the long standing issues with the security manager support is that
> the overhead when there is no security manager is non-zero. Two specific
> examples are (i) the overhead of defineClass (assuming the defining loader
> is a SecureClassLoader) as it involves a callback to get the permissions for
> the protection domain and (ii) the overhead of AccessController.doPrivileged
> in order to execute an action on a privileged stack.
>
> The bigger question is of course whether it is interesting to run with a
> security manager in 2017. It's clearly still important for environments that
> download potentially malicious code to execute in a sandox but that isn't
> most applications these days. We have seen a few cases where applications
> set a security manager in order to enforce some policy, like preventing
> plugins calling System.exit but these are cases that would be better served
> with other solutions.
>
> I would like to explore changes to the API and implementation that would
> allow us to eliminate some of the overhead when not running with a security
> manager. Long term then it would be interesting to explore degrading and
> eventually dropping the security manager but that is beyond the scope of
> what I would like to discuss here. Sean Mullan and Jeff Nisewanger ran an
> interesting BOF at JavaOne 2017 on this topic and I believe are planning to
> do a survey at some point to understand the current usage of the security
> manager.
>
> For now I would like to explore what it would take to eliminate some of the
> overhead. A big challenge is the System.setSecurityManager API as allows a
> security manager to be set in a running VM. This means that classes loaded
> before a security manager is set may be involved in permission checks that
> happen after a security manager is set. Similarly, privileged actions may
> have started before the security manager is set. The complications around
> this go away if there was some way to know that a security manager can never
> be turned on in a running system.
>
> To get started, I've put a prototype of an initial proposal on the cr site
> [1]. The summary of the proposal is:
>
> 1. A "disallow security manager" mode that is opt-in to disallow
> setSecurityManager being used to set a security manager in a running VM.
> Opt-in means it has not impact on existing code.
>
> 2. Deprecates setSecurityManager to discourage further usage and allow for
> the possibility of making "disallow security manager" the default in the
> future. If that were to become the default then it would have no impact on
> deployments that set the security manager on the command line at startup.
>
> 3. When running in "disallow security manager" mode,
> AccessController.doPrivileged invokes actions directly and
> SecureClassLoader.defineClass skips the callback to get permissions for the
> protection domain. This part helps startup and runtime performance.
>
> It's important not to get hung up on the details in the webrev, the
> important thing is understand if the security folks on this mailing list are
> open to a run mode that prevents a security manager being set in a running
> system.
>
> -Alan.
>
> [1] http://cr.openjdk.java.net/~alanb/8191053/webrev/

-- 
- DML