Distributed Logging in Javawith Constrained Resource Usage

Sunil Brown Varghese, Daniel Andresen

Dept. of Computing and Information Sciences

Kansas State University

Overview

Introduction to Logging Distributed Logging Distributed Logging with RMI Conclusions

Introduction

Everyone logs in one way or other… print(), out, System.out.print() are examples

of console logging Large systems like aircraft monitoring

systems, transaction–based financial processes may log into multiple targets

Advantages of Logging

Feedback to developers during development cycles

Provide system diagnostic alerts from production systems

Provide information for statistical analysis of working of a system

Delivery based on levels of importance could improve efficiency in development cycle

Logging options

The Log4J Project Logging Toolkit for Java from IBM Java 2 Standard Edition (v1.4.0 onward)

Distributed Logging

Distributed Logging

Log4j has good support for sending log messages to multiple log servers by TCP/IP socket connections

J2SE provides the Java Logging framework by default and supports distribution of log messages by socket connections (though listeners are left to the developer)

Java Logging Framework

Observation effects the Observed

If generated at a high rate, distributed log messages may swamp the processor and the network connections whereby decreasing system performance and the accuracy of the log analysis

Solution: Limit the use of distributed log messages to critical and non-repetitive sections of code and buffer messages

Distributed Logging with RMI

Transmission of Log messages with RMI calls allows the log objects to be recorded at remote sites as is

Log messages may be modified easily ( say based on network load, exception objects may be removed from log)

Representation

Extending the Java Logging Framework

RMI Handler extended from Handler class ensuring compatibility with Logger and security in-built into Java Logging

The logging levels may be varied dynamically against bandwidth usage by providing level information in an XML file

Class View – Client Side

Class View – Server Side

Sample Levels.xml file

<?xml version='1.0'?> <param-list> <cutoff>1600</cutoff> <cutoff>1400</cutoff> <cutoff>1200</cutoff> <cutoff>1000</cutoff> <cutoff>800</cutoff> <cutoff>600</cutoff> <cutoff>500</cutoff> </param-list>

RMI Log Server

Instantiates a logger and assigns the RMI Handler and other Handlers based on an XML file

Server binds to the specified machine and port (binding data from input XML file)

This framework allows a chain of RMI Servers to be formed connected by RMI Handlers in a tree fashion

Bandwidth Computation

TCP/IP based weighted average method is used to update bandwidth

Bandwidth is sampled for a Sampling Time and it is reset for a fresh start

If there is no bandwidth updates for an Inactivity Tolerance period, the Bandwidth is reset

RMI Log Buffer

The RMI Handler may buffer the log messages so as to reduce the number of RMI calls needed. This could dramatically raise the efficiency of transmission

The RMI Buffer ensures message transmission under normal RMI Handler operation and abnormal termination

RMI Log Server GUI

The GUI connects to RMI Log Server and provides time graphs of bandwidth statistics

The cutoffs along with bandwidth variance are shown

RMI Log Server GUI

RMI Log Server GUI Connection

As there isn’t a unique naming scheme for RMI Handlers connected to a logger, the URL and port information is used as RMI Handler identifier

RMI Log Server GUI Connection

Limitations / Assumptions

Leveling thresholds at the RMI Handler level are to be set intuitively by the user

Log message sizes were taken as constants for testing purposes

The overhead of making an RMI call is heavier than sending data through TCP/IP sockets

Conclusions

RMI Logging allows the recreation of log messages in remote sites as long as all sub-components are serializable

Allows the exception objects to be stored in remote repositories

RMI Handler can monitor its bandwidth usage and discard log messages of low level

Future work

RMI Handler may maintain a server list so that if one RMI Log Server fails, it retransmits to secondary servers

The GUI interface can be developed further to remotely control RMI Log Servers

Questions ?