log4perl mike schilli, yahoo! oscon, 07/24/2008. logging – why? debug during development go back...
TRANSCRIPT
Log4perl
Mike Schilli, Yahoo!
OSCON, 07/24/2008
Logging – why?
• Debug during development
• Go back in time and figure out what happened.
• Measure performance
• Trace activity on live systems
Why Log4perl and not one of the 20 Logging modules on CPAN?
• No CPAN dependencies
• Easy to use, but scales with complexity
• Unique Features
Log::Log4perl Availability
• cpan> install Log::Log4perl(Only requires core modules)
• Included in major Linux distrossudo apt-get install liblog-log4perl
• Requires Perl 5.00503 or better
• Windows: ppm package available in ActiveState archives or from log4perl.com
Use Log4perl as a Remote Control to your System.
Log::Log4perl Remote Controls
Loggers
Layouts
Appenders
Levels
Log::Log4perl Remote Controls
Loggers
Layouts
Appenders
Locate it in the system
Format it
Write it out
Levels Log/Suppress Priority/Level
Log::Log4perl Remote Controls
Loggers
Layouts
Appenders
DEBUG “Starting up”; ERROR “Cannot open $file”;
“Starting up” =>2007-06-21 07:30:33 Foo.pm-123 Starting up
DatabaseLog File …
Levels
Turn logging on in main or Net::Amazon
Sounds complicated?
Actually, it’s easy …
Easy Log4perl
#!/usr/bin/perl –w
use strict;
use Log::Log4perl qw(:easy);
DEBUG “Starting up”;
Don’t like macros? Use get_logger()
#!/usr/bin/perl –w
use strict;
use Log::Log4perl qw(get_logger);
my $logger = get_logger();
$logger->debug(“Starting up”);
Like it clean? Use Moose!
package Ferrari;use Moose;with “MooseX::Log::Log4perl”;
sub drive { my($self) = @_; $self->log->debug(“Wroom!!”);}
Easy Log4perl
#!/usr/bin/perl –w
use strict;
use Log::Log4perl qw(:easy);
DEBUG “Starting up”;
Easy Log4perl
$ ./hello
$
Easy Log4perl
#!/usr/bin/perl –wuse strict;use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init( $DEBUG );
DEBUG “Starting up”;
Easy Log4perl
$ ./hello
2008/07/08 18:37:12 Starting up
$
Easy Log4perl
#!/usr/bin/perl –wuse strict;use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init( $DEBUG );
DEBUG “Starting up”; # … something happensERROR “Horrible error!”;
Easy Log4perl
$ ./hello
2008/07/08 18:37:12 Starting up
2008/07/08 18:37:12 Horrible error!
$
Easy Log4perl
#!/usr/bin/perl –wuse strict;use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init( $ERROR );
DEBUG “Starting up”;ERROR “Horrible error!”;
Easy Log4perl
$ ./hello
2008/07/08 18:37:12 Horrible error!
$
Remote Control #1: Levels
You get the concept:
Log LevelConfigured
MessagePriority
FATAL
ERROR
WARNING
INFO
DEBUG
TRACE
FATAL
ERROR
WARNING
INFO
DEBUG
TRACE
Chatty configuration
Log LevelConfigured
MessagePriority
FATAL
ERROR
WARNING
INFO
DEBUG
TRACE TRACE
Silent configuration
ERROR
Log LevelConfigured
MessagePriority
FATAL
ERROR
WARNING
INFO
DEBUG
TRACE
Log Levels
• Choose them wisely– TRACE(“$bytes bytes transferred”);– DEBUG(“HTTP get OK”);– INFO(“Starting up”);– WARN(“HTTP get failed, retrying”);– ERROR(“Out of retries!”);– FATAL(“Panic! Shutting down.”);
Remote Control #2: Layouts
Location-Awareness
• Log4perl’s Loggers are aware of their location:
package Foo;
use Log::Log4perl qw(:easy);
sub foo {
DEBUG “Starting up”;
}
Location-Awareness
package Foo;
use Log::Log4perl qw(:easy);
sub foo {
DEBUG “Starting up”;
}
$ ./hello
2008/07/13 19:32:39 Starting up
Location-Awareness
package Foo;
use Log::Log4perl qw(:easy);
sub foo {
DEBUG “Starting up”;
}
$ ./hello
637 Foo::foo ./Foo.pm-4> Starting up
Location-Awareness
package main;use Log::Log4perl (:easy);Log::Log4perl->easy_init({ level => $DEBUG, layout => “%r %M %F-%L> %m%n”,});
Foo::foo(); # unchanged!
$ ./hello637 Foo::foo ./Foo.pm-4> Starting up
Configuration Files
If this becomes unwieldy:
Log::Log4perl->easy_init({ level => $DEBUG, layout => “%r %M %F-%L>%m%n”,});
Configuration Files
#l4p.confl4p.logger = DEBUG, Screen
l4p.appender.Screen = Log::Log4perl::Appender::Screenl4p.appender.Screen.Layout = PatternLayoutl4p.appender.Screen.Layout.ConversionPattern = \ %r %M %F-%L> %m%n
Log::Log4perl->init( “l4p.conf” );
Advantages of Config Files
• Can be edited– indepentently of the script– while the script runs– by people without access to the code
Remote Control #3: Categories (Loggers)
Turn on Logging EverywhereScript
Modulesl4p.logger = DEBUG, Screen
Using Categories
l4p.logger.Net.Amazon = \ DEBUG, Screen
Net::Amazon
Script
Modules
Using Categories
l4p.logger.main = \ DEBUG, Screen
main
Net::Amazon
Script
Modules
Using Categories
l4p.logger.main = DEBUG, Screenl4p.logger.Net.Amazon = \ DEBUG, Screen
Modules
Script
Net::Amazon
main
Categories
#l4p.confl4p.logger.main = DEBUG, Screenl4p.logger.Net.Amazon = DEBUG, Screen
l4p.appender.Screen = Log::Log4perl::Appender::Screenl4p.appender.Screen.Layout = PatternLayoutl4p.appender.Screen.Layout.ConversionPattern = \ %r %M %F-%L> %m%n
Net::Amazon
main
Category Inheritance
#l4p.confl4p.logger.Net = DEBUG, Screen
l4p.appender.Screen = Log::Log4perl::Appender::Screenl4p.appender.Screen.Layout = PatternLayoutl4p.appender.Screen.Layout.ConversionPattern = \ %r %M %F-%L> %m%n
Net::Google
Net::Amazon
main
Remote Control #5: Appenders
Root Logger
#l4p.confl4p.logger = DEBUG, Screen
l4p.appender.Screen = Log::Log4perl::Appender::Screenl4p.appender.Screen.Layout = PatternLayoutl4p.appender.Screen.Layout.ConversionPattern = \ %r %M %F-%L> %m%n
Net::Google
Net::Amazon
main
Multiple Appenders
#l4p.confl4p.logger.main = DEBUG, Screenl4p.logger.Net.Amazon = DEBUG, File
l4p.appender.Screen = Log::Log4perl::Appender::Screenl4p.appender.Screen.Layout = SimpleLayout
l4p.appender.File = Log::Log4perl::Appender::Filel4p.appender.File.filename = /var/log/myapp.logl4p.appender.File.Layout = PatternLayoutl4p.appender.File.Layout.ConversionPattern = \ %r %M %F-%L> %m%n
Net::Amazon
main Screen
File
Multiple Appenders (different log levels)
#l4p.confl4p.logger.main = DEBUG, Screenl4p.logger.Net.Amazon = ERROR, File
l4p.appender.Screen = Log::Log4perl::Appender::Screenl4p.appender.Screen.Layout = SimpleLayout
l4p.appender.File = Log::Log4perl::Appender::Filel4p.appender.File.filename = /var/log/myapp.logl4p.appender.File.Layout = PatternLayoutl4p.appender.File.Layout.ConversionPattern = \ %r %M %F-%L> %m%n
Net::Amazon
main Screen
File
Multiple Appenders
#l4p.confl4p.logger.main = DEBUG, Screen, File
l4p.appender.Screen = Log::Log4perl::Appender::Screenl4p.appender.Screen.Layout = SimpleLayout
l4p.appender.File = Log::Log4perl::Appender::Filel4p.appender.File.filename = /var/log/myapp.logl4p.appender.File.Layout = PatternLayoutl4p.appender.File.Layout.ConversionPattern = \ %r %M %F-%L> %m%n
Net::Amazon
main Screen
File
Log4perl Flow
Application sends a log message (Category, Priority)
Log4perl Configuration decides go/no go, based on Category and Priority
Appender Appender Appender
Layout
?
Log4perl and Log4j
• Log::Log4perl ports Log4j to Perl
• Log4j: de facto Java logging standard, by Ceki Gülcü– Latest development: ‘logback’
• http://logging.apache.org/log4j
• Log::Log4perl adds perlisms demanded by users
Log4perl History
• 0.01 release 2002• Current release 1.17 (07/2008)• Authors: Mike Schilli, Kevin Goess• Used by major banks, target.com,
fedex.com, Yahoo!, Google, …• Several CPAN modules support it (e.g.
Catalyst, Net::Amazon, …)• Every major Linux distro has it (Ubuntu,
Suse, Fedora …)
Log4perl Release History
Design Goals
• Easy to use in small scripts
• Scales easily with growing architecture
• Log4perl-enabled CPAN modules can be used with and without Log4perl initialization
• Optimized for Speed
• Open Architecture, extensible
Combine Remote Controls
• Write code once, and L4p-enabled scripts/modules can be used– with any logging configuration you desire– by changing loggers, appenders and layouts
independently– similar to dtrace probes
• No need to change your source code to change the logging behavior
Init and Watch
• Log::Log4perl->init(“l4p.conf”);• Log::Log4perl->init(\$conf_string);
• Log::Log4perl->init_and_watch(“l4p.conf”, 30);• Log::Log4perl->init_and_watch(“l4p.conf”, ‘HUP’ );
Demo
Appenders
• Appenders are output sinks
• Get called if a message passes category and level hurdles
Appenders
• Log::Log4perl::Appender::Screen Log::Log4perl::Appender::File Log::Log4perl::Appender::Socket Log::Log4perl::Appender::DBI Log::Log4perl::Appender::Synchronized Log::Log4perl::Appender::RRDs
• Log::Dispatch provides even more:
Log::Dispatch Appenders
• Log::Dispatch::ApacheLog Log::Dispatch::DBI, Log::Dispatch::Email, Log::Dispatch::Email::MIMELite Log::Dispatch::File Log::Dispatch::FileRotate Log::Dispatch::Screen Log::Dispatch::Syslog, Log::Dispatch::Tk
Example: Rotating Log File Appender
• Keep 5 days of logfiles, then delete:
log4perl.category = WARN, Logfile log4perl.appender.Logfile = Log::Dispatch::FileRotate log4perl.appender.Logfile.filename = test.log log4perl.appender.Logfile.max = 5 log4perl.appender.Logfile.DatePattern = yyyy-MM-dd log4perl.appender.Logfile.TZ = PST log4perl.appender.Logfile.layout = \ Log::Log4perl::Layout::SimpleLayout
Rotating Files
• Rotating File Appender:– Log::Dispatch::FileRotate– Single request pays for rotation– Rotation runs as particular user
• External Rotators (e.g. newsyslog):– recreate flag makes sure file appender adjusts– recreate_check_interval saves on stat() calls
Layouts
• SimpleLayout
$log->debug(“Sending Mail”);
DEBUG – Sending Mail
log4perl.appender.Screen = \ Log::Log4perl::Appender::Screenlog4perl.appender.Screen.layout = SimpleLayout
Layouts
• PatternLayout
$log->debug(“Sending Mail”);
2004/10/17 18:47:25 l4ptest.pl:25> Sending Mail
log4perl.appender.Logfile.layout = \ Log::Log4perl::Layout::PatternLayout log4perl.appender.Logfile.layout.ConversionPattern = \ %d %F{1}:%L> %m %n
Layouts
• PatternLayout%T stack trace%c Category of the logging event. %C Fully qualified package (or class) name of the caller %d Current date in yyyy/MM/dd hh:mm:ss format %F File where the logging event occurred %H Hostname %l calling method + file + line %L Line number within the file where the log statement was issued %m The message to be logged %M Method or function where the logging request was issued %n Newline (OS-independent) %p Priority of the logging event %P pid of the current process %r Number of milliseconds elapsed from program start %x The elements of the NDC stack%X{key} The entry 'key' of the MDC%% A literal percent (%) sign
Layouts
• Still not enough? Write your own:
log4perl.PatternLayout.cspec.U = sub { return "UID $<" } …log4perl.appender.Logfile.layout = \ Log::Log4perl::Layout::PatternLayout log4perl.appender.Logfile.layout.ConversionPattern = \ %d %U> %m %n
Speed
Benchmarks
• 1M/sec suppressed calls
• 50k/sec logged messages (memory appender)
• No degradation with subcategories
Avoid Wasting Cycles
for (@super_long_array) {
$logger->debug("Element: $_\n");
}
if($logger->is_debug()) {
for (@super_long_array) {
$logger->debug("Element: $_\n");
}
}
Dirty Tricks
Resurrect in a Single File
• The :resurrect target uncomments lines starting with ###l4p:
use Log4perl qw(:easy :resurrect);
sub foo {
# …
###l4p DEBUG “foo was here”;
}
Resurrect L4p in all Modules
• The Log::Log4perl::Resurrector touches all subsequently loaded modules (experimental!):
use Log4perl qw(:easy);
use Log::Log4perl::Resurrector;
use Foo::Bar;
# Deploy a module without requiring L4p at all!
package Foo::Bar;
###l4p use Log4perl qw(:easy);
sub foo {
###l4p DEBUG “This will be resurrected!”;
}
The Future
Wouldn’t it be nice …
• … if all modules on CPAN had a standard logging interface?
Log4perl-enable a CPAN Module
package Foo;
sub foo {
# … do something
}
1;
Log4perl-enable a CPAN Module
package Foo;
use Log::Log4perl qw(:easy);
sub foo {
# … do something
DEBUG “Fooing $bar”;
}
1;
… and voila, your CPAN module has a built-in remote.
Be Open
• Let other people debug your module in a standard way.
Q & A
Q & A
Thank You!
Log::Log4perl Project Page (includes slides of this talk):
http:/log4perl.com
Email me:Mike Schilli [email protected]
Don’t forget to submit a Review!
• Go to the OSCON schedule, click on this talk and then ‘rate this talk’
• Thanks!
Bonus Slides
Advanced Trickery
Infiltrating LWP
• Ever wondered what LWP is doing behind the scenes?
use LWP::UserAgent;use HTTP::Request::Common;
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init($DEBUG);Log::Log4perl->infiltrate_lwp();
my $ua = LWP::UserAgent->new();my $resp = $ua->request(GET "http://www.yahoo.com");
Infiltrating LWP
• Getting LWP to use Log4perl for logging:
2004/10/20 18:36:43 ()2004/10/20 18:36:43 ()2004/10/20 18:36:43 GET http://www.yahoo.com2004/10/20 18:36:43 Not proxied2004/10/20 18:36:43 ()2004/10/20 18:36:43 read 634 bytes2004/10/20 18:36:43 read 4096 bytes…2004/10/20 18:36:43 read 2488 bytes2004/10/20 18:36:43 Simple response: OK
Pitfalls
Beginner’s Pitfalls
Beginner’s Pitfalls
• Appender Additivity:– Once a lower level logger decides to fire, it will
forward the message to its appenders.– It will also forward the message
unconditionally to all of its parent logger’s appenders.
Beginner’s Pitfalls
• Appender Additivity (with dupes)
package Net::Amazon;DEBUG(“Debugging!”);
log4perl.category = FATAL, Screenlog4perl.category.Net.Amazon = DEBUG, Screen
log4perl.appender.Screen = \ Log::Log4perl::Appender::Screenlog4perl.appender.Screen.layout = SimpleLayout
DEBUG - Debugging! DEBUG - Debugging!
Appender Additivity
Net::Amazon
Net
root
DEBUG
FATAL
Screen Appender
File Appender
Beginner’s Pitfalls
• Appender Additivity (no dupes)
package Net::Amazon;DEBUG(“Debugging!”); Q
log4perl.category = ERROR, Screenlog4perl.category.Net.Amazon = DEBUG, Screenlog4perl.additivity.Net.Amazon = 0
log4perl.appender.Screen = \ Log::Log4perl::Appender::Screenlog4perl.appender.Screen.layout = SimpleLayout
DEBUG – Debugging!
Write Your Own Appender (1)
package ColorScreenAppender;our @ISA = qw(Log::Log4perl::Appender);use Term::ANSIColor;
sub new { my($class, %options) = @_; my $self = {%options, …}; bless $self, $class;}
sub log { my($self, %params) = @_; print colored($params{message}, $self->{color});}
Write Your Own Appender (2)
log4perl.logger = INFO, ColorApp log4perl.appender.ColorApp=ColorScreenAppender log4perl.appender.ColorApp.color = red log4perl.appender.ColorApp.layout = SimpleLayout
Filters
• Last minute chance for Appenders to block
• More expensive than Level/Category blocking
• Text/Level Match comes with L4p
• Write your own custom filters
Filters
log4perl.category = INFO, Screen log4perl.filter.MyFilter = \ Log::Log4perl::Filter::StringMatchlog4perl.filter.M1.StringToMatch = let this through
log4perl.appender.Screen = \ Log::Log4perl::Appender::Screen log4perl.appender.Screen.Filter = MyFilter log4perl.appender.Screen.layout = \ Log::Log4perl::Layout::SimpleLayout
Filters
log4perl.filter.M1 = Log::Log4perl::Filter::StringMatch log4perl.filter.M1.StringToMatch = let this through log4perl.filter.M1.AcceptOnMatch = true
log4perl.filter.M1 = Log::Log4perl::Filter::LevelMatch log4perl.filter.M1.LevelToMatch = INFO log4perl.filter.M1.AcceptOnMatch = true
Best Practices
• Don’t provide program name, function name in the message. Use Layouts instead.
• Log plenty. Don’t worry about space, use rotating log files for chatty output.
What you should log
• Program starting up, shutting down
• Function parameters
• Milestones in request processing
• Start/end of lenghty operations (e.g. DB access)
• result of called function (logged in calling function)
What you shouldn’t log
• Function/method names (although you want parameters)
• Date
• Process ID
• … everything else provided by the layout
Error Handling
use Log::Log4perl qw(:easy);
do_something or
LOGDIE “Horrible error!”;
• LOGCARP, LOGCLUCK, LOGCONFESS etc. also available.
Best Practices
• Rolling out new L4p-enabled Modulespackage My::Module;
use Log::Log4perl qw(:easy);
sub new { # … }sub foo { DEBUG “I’m here”; # … }sub bar { INFO “Now I’m here”; # … }
# Use w/o Log4perluse My::Module;
$o = My::Module->new();$o->foo();
# Log4perl enableduse My::Module;use Log::Log4perl qw(:easy);Log::Log4perl->easy_init($DEBUG);$o = My::Module->new();$o->foo();
Gotchas
• Avoid ‘append’ on NFS (creating lots of ‘read’ traffic)
• Don’t put expensive operations into the logger call which are executed before the logger decides
• Permission issues with rotating file appenders
mod_perl
• Init() in startup handler
• Current limitation: One init().
• Use init_once() if you’re not sure who inits first.
Combat overlapping Messages
• Either live with it or– Use the Synchronized appender– Use Appender.File.syswrite=1
Driving Log::Log4perl further
• Bug Fixes: Submit to [email protected]
• New Features: Propose features or send patches
• New Appenders: Release separately as a module in the
Log::Log4perl::Appender::xx namespace on CPAN
• … or contact me: [email protected]
Easy Mode Init with Files
use Log::Log4perl qw(:easy);
Log::Log4perl->easy_init({
level => $DEBUG,
file => “>file.log”,
category => “Bar::Twix”,
});
Multiline Layout
Instead of2007/04/04 23:59:01 This isa message withmultiple lines
get this:2007/04/04 23:59:01 This is2007/04/04 23:59:01 a message with2007/04/04 23:59:01 multiple lines
with PatternLayout::Multiline:log4perl.appender.Logfile. layout = \ Log::Log4perl::Layout::PatternLayout::Multilinelog4perl.appender.Logfile.layout.ConversionPattern = %d %m %n
Perl Hooks
• Add parameters known at init() time
log4perl.appender.Logfile.filename = sub { \
“mylog.$$.log” }
Loggers
• Have a– category (e.g. “Net::Amazon” )– priority (e.g. $DEBUG )
• Take a message (e.g. “Starting” )
Loggers
• Pass on their message if– logging has been configured for
• their category ( “Net::Amazon”) or• a parent category ( “Net”, “” )
– and (!) the message priority (e.g. $DEBUG) is greater or equal than the configured log level (e.g. $DEBUG) for the category
Limit Throughput: Levels and Categories
• Categories determine which parts of the system are addressed.my $log = get_logger(“Net::Amazon”);
or simplypackage Net::Amazon;
DEBUG “…”;
• Levels indicate the message priority.$log->debug(“Request sent ($bytes bytes)”;
Displaying Logs with Chainsaw
• Log4j features an XML layout and a socket appender
• XML output is displayed in a nice GUI called Chainsaw
• Log::Log4perl also features an XML layout and a socket appender
• … XML output is also displayed in a nice GUI called Chainsaw!
Displaying Logs with Chainsaw
Java / log4j Program
Java / log4j Program
Log::Log4perl enabled Perl Script
Log::Log4perl::Layout::XML
Log::Log4perl::Appender::Socket
Displaying Logs with Chainsaw