Java Security: From HotJava to Netscape & Beyond

Drew Dean, Edward W. Felten, Dan S. Wallach

Department of Computer Science,

Princeton University

May, 1998

This article examines the Java language and both the HotJava and Netscape browsers which support it, and find a significant number of flaws which compromise their security.

.

Presented by Jenny Liu15 May 2001

Summary Introduction

Java Security Mechanisms

Taxonomy of Java Bugs - Denial of service - Information Available to Applets

Security Analysis: how structure of some systems contribute to the existence of bugs

- Policy enforcement failures

Representative Application and its security requirements

- Networking

Conclusion

Introduction Sun introduced the notion of downloading a program

(called an applet) which runs inside the Web browser in mid-1995.

Applets cannot directly make system call - The securityManager class is meant to define an

interface for access control, and it is used to approve dangerous operations.

In order to be secure, systems must limit applets’ - access to system resources - interference with other applets - communication with other systems - ability to learn about local environments

Java Security Mechanisms All access controls in HotJava were done on an

ad hoc basis – insufficient.

“SecurityManager” defines and implements a security policy, centralizing all access control decisions.

Java uses its “type system” to provide protection for the security manager.

- embeddable in other software

- protection boundaries can be crossed without a context switch

Examples of known security flows in HotJava and Netscape

Denial of Service

Definition: A condition in which a system can no longer respond to normal requests

Computer Desktop Encyclopedia, 1999

Some examples of attacks are:

- busy-waiting to consume CPU cycles

- allocating memory until the system runs out

Denial of Service (Continuous) Two twists that can make denial of service attacks

more difficult to cope with:

- Masking the source of the attack: An attack can be programmed to occur after some time

delay, causing the failure to occur when the user is viewing a different Web page.

- Degradation of service: Definition: Significantly reducing the performance of the

browser without stopping it.

An attack can cause degradation of service rather than “outright” denial of service.

Information Available to Applets

What the applet can learn about the user’s environment to send over the channel could be an issue in security.

System.getenv() in HotJava has no security checks. Therefore user’s login name, machine name, contents of all environment variables are easy to learn.

By probing environment variables, some valuable information can be discovered, which is then useful to attackers desiring to break into a user’s machine.

“In JDK and Netscape, System.getenv() was replaced with “system properties”, many of which are not supposed to be accessible by applets.”

Analysis of bugs’ possible cause – Policy enforcement failures

“There must be an explicit and well-defined security policy enforced by the system” – Orange Book’s Fundamental Computer Security Requirements.

But… “the present documents on Netscape & HotJava do not formally define a security policy”.

We trust the operating system’s security policy will be enforced well on Java as general programming language.

However, we can not trust Java as a system for running untrusted applets obtained from the Web. So we require that Java define and implement a protected subsystem with an appropriate security policy.

Major problem in defining a security policy :

For Java Applets:

Policy needs to be “flexible” enough to not unduly limit applets, while still preserving the user’s integrity and privacy

Representative Application - Networking “Untrusted applets should be able to use network

services only under restricted circumstances.”

Java runtime library should support the protocols in current use today: HTTP(the Web), FTP(file transfer), Gopher, SMTP(email), NNTP(Usenet news), Finger(user information), etc.

FTP presents the most difficulties: – it has the server open a connection back to the client for

each data transfer, requiring the client to call listen() and accept().

All FTP servers are required to support passive mode, where the client actively opens all the connections.

FTP clients should be carefully designed to ensure that an applet does not use it to harm a third party.

Conclusion This article demonstrated an array of attacks that allow the

security of both HotJava and Netscape to be compromised.

“While many of the specific flaws have been patched, the overall structure of the systems leads us to believe that flaws will continue to be found”.

A well-defined, formal security policy would help to allow more functionality for untrusted applets without compromising the user’s integrity and privacy.

We conclude that the Java system in its current form can not easily be made secure. Therefore redesign of the language, the bytecode format, and runtime system appear to be the steps toward a higher-assurance system.

Questions & some thinking: Why does the system have to carefully manage

system calls?

Can you think of some examples of system resources which the systems should limit applets’ access to?