Further Defenses for the Security Manager

[Michael Maass]

Is there a standard corpus of applets that Oracle would use to test this 
kind of feature? I am need of an applet corpus and I am wondering if 
there is something out there already that you folks use.

Michael

On 12/19/2014 07:47 PM, Jeff Nisewanger wrote:
> Thank you for contacting us and sharing the initial results of your research.
>
> You raised two basic topics.
>
> First, you discussed the possibility of adding additional restrictions on the ability to change the system Security Manager multiple times during application execution. This is normally already subject to a security check but there might be malicious exploits that bypass the existing check.
>
> It turns out that we are already independently investigating this topic. We agree that such a restriction likely requires additional assistance from the JVM implementation for several reasons.
>
> As you mentioned, an important aspect to such a change is to minimize backwards compatibility problems. Your research on the use of security managers by existing Java applications is interesting and useful.
>
> You also suggested the general idea that whenever a new class is loaded we should check the granted permissions of the classes containing the methods on the thread call stack.
>
> You propose to ensure that each such class on the call stack has already itself been granted a superset of the permissions that would be granted to the newly loaded class. However, you would not enforce this general rule for common cases such as classes loaded by the bootstrap loader or classes within the list of restricted package names.
>
> Does your proposal take into account any calls to AccessController.doPrivileged() on the thread call stack? Could a thread shortcut this proposed check by causing it's action to take place on another thread (create a new thread to initiate the class loading)?
>
> The Java Virtual Machine specification allows for class loading to be initiated in a flexible manner that may have only a tangential relationship to the members of a thread's call stack. Examples include:
>
> 1. When a class is being loaded the JVM resolves the names of the superclass or super interfaces
>
> 2. Constant pool resolution during the execution of a bytecode instruction in a class that refers symbolically to a named class type.
>
> 3. Bytecode verification of another class which is in the process of being loaded. To ensure type-safety, the verifier may sometimes load other classes recursively that are symbolically referenced by a bytecode instruction (before any of the code in the class being verified is ever executed normally).
>
> 4. Direct calls to Class.forName(), ClassLoader.loadClass() or perhaps somehow maliciously to ClassLoader.defineClass().
>
> Also, according to the specification, the JVM is free to spontaneously "pre-load" classes whenever it feels like it even if those classes have no apparent relationship with the thread chosen to load them.
>
> For this reason, Java class loading is generally independent of thread context except for the immediate class in which a constant pool resolution is being performed which is subject, for example, to the language-level access controls (public, protected, etc.) of the class it is referencing.
>
> Thanks,
> Jeff
>
>
>> On Oct 29, 2014, at 6:35 AM, Michael Maass <mmaass at andrew.cmu.edu> wrote:
>>
>> Hello All,
>>
>> I've spent the last 6 months working with some colleagues on a project that aimed to stop an exploitation avenue that has been popular with recent Java exploits: disabling the security manager. We think that what ended up with may be worthy of a JEP and/or a prototype implementation in the JVM; we're looking for feedback from the experts, you folks.
>>
>> We initially set out to determine how benign applications interact with the security manager and to determine how those interactions differ from malicious ones. The idea was that an investigation aimed at making these determinations could show us ways to constrain operations on the manager that are backwards compatible with benign applications while blocking exploits.
>>
>> After investigating the open source Java landscape we learned to distinguish between (1) "self-protecting" and "defenseless" security managers and (2) the operations that may be performed in the presence of each. A defenseless manager enforces a policy that uses at least one permission in a set of permissions we identified that allow anyone to change the manager itself or the policy it enforces (a self-protecting manager is the opposite of a defenseless manager). Given this definition, we found that enforcing the following rules stopped known exploits without impeding (aside from overhead) the execution of benign applications:
>>
>> 1. If a self-protecting manager is set, a class may not load or cause the loading of a class more privileged than itself unless the loaded class is in a protected package (listed in the package.access property in java.security.Security).
>>
>> 2. If a self-protecting manager is set, an application may not change the security manager or the enforced policy in any way.
>>
>> We built a JVMTI agent to enforce these rules (available here: https://github.com/SecurityManagerCodeBase/JavaSandboxFortifier). Unfortunately, to use the JVMTI events that allow us to enforce rule 2, the JIT must be off, which drastically slows down the execution of applets in the presence of our agent. Rule 1 adds about 1-2% overhead. We believe overhead would drop enough for adoption if the JVM were to natively enforce these rules instead of using an agent, but we do not have the resources or the expertise to try this ourselves.
>>
>> Any thoughts on whether or not this type of change is worth pursing in the form of a JEP and/or prototype in the JVM? If so, is anyone willing and able to help?
>>
>> Thanks,
>>
>> Michael Maass, PhD Candidate
>> Software Engineering
>> Institute for Software Research
>> School of Computer Science
>> Carnegie Mellon University