Re: Huge security hole in Sun's JVM

Jeff Nisewanger (Jeff.Nisewanger@eng.sun.com)
Fri, 7 May 1999 16:34:24 -0700 (PDT)

Message-Id: <199905072336.QAA21099@shorter.eng.sun.com>
Date: Fri, 7 May 1999 16:34:24 -0700 (PDT)
From: Jeff Nisewanger <Jeff.Nisewanger@eng.sun.com>
Subject: Re: Huge security hole in Sun's JVM
To: java-security@java.sun.com, pugh@cs.umd.edu

> Sun's JVMs have a huge security hole. The JVM allows
> an outside class to invoke a private constructor or
> a private static method, and to access a private
> variable. I confirmed this bug in the following JVM's:
>
> Sun Sparc Solaris 1.1.8
> Sun Sparc Solaris 1.2.1 production
> Sun Sparc Solaris 1.2.1 reference
>
> WinTel 1.1.8
> WinTel 1.2.1 classic
> WinTel 1.2.1 HotSpot

This is not a bug. What you are seeing is that in some
circumstances Java classes are purposefully not run through full
language access control checks when loaded directly from the local
disk. Illegal accesses by downloaded Java code will be caught.

Java classes loaded directly by the "bootstrap class loader"
builtin to the virtual machine are not run through the access checks
(or the bytecode verifier). The theory is that local classes on the
disk are completely trusted and therefore do not need the overhead of
access checking whereas downloaded classes are not automatically
trusted.

For 1.1.x this trust extends to all classes on the classpath
including core Java classes and application classes. For 1.2.x this
normally includes only the core Java classes from the JRE rt.jar. Instead,
1.2.x application classes are loaded from a seperate class loader and
are bytecode verified and normally subject to language access
control checks.

An exception to the normal language access control checks is
made for local application classes in 1.2.x if the class attempting the
access and the target class were both loaded by the application class
loader and are in the same "protection domain". This narrow exception provides
backwards compatibility for existing applications impacted by
bugid 4160064 listed in Bug Parade.

In all cases in both 1.1.x and 1.2.x, downloaded Java bytecodes
are fully verified and the complete language access control checks are
performed.

Your example will not work if transformed into an applet and
downloaded. Also, in 1.2.x it will not work as an application if you
place the 2 class files into seperate protection domains by, for
example, placing each class into it's own jar file. Finally, the
illegal access will be caught in 1.2.x if you use the -Xverify:all flag
as in "java -Xverify:all A2".

Jeff