EntwicklerCamp 2016 IBM Notes – den EierlegendenWollMilchSau für alle und Immer

Debugging Java in Agents, Script Libraries, and XPages

Julian Robichaux, panagenda

EntwicklerCamp 2016

Who Am I?• Julian Robichaux

▪ Senior Application Developer, panagenda

▪ Notes developer since 1996

▪ nsftools.com since 2002

▪ Conference speaker since 2006

▪ IBM Champion since 2011

▪ @jrobichaux (since 2007)

EntwicklerCamp 2016

Why Are We Here?• Debugging Java in Notes/Domino!

▪ It’s easier than you think

▪ I do this all the time

▪ Eclipse is magical

• Java agents and XPages Java

• Tips at the end about creating/reading Java core dumps, finding bugs before the code runs, and more!

• N.B. this is from a presentation I gave with Mark Myers (LDC). HI MARK!

EntwicklerCamp 2016

What Will We Be Using?• IBM Domino® Server 9.0.1 FP4

• IBM Domino Designer (DDE) 9.0.1 FP4

• Java 1.6 (as included with Domino and DDE)

• XPages and Java agents

• Eclipse

EntwicklerCamp 2016

The Ninja Classes• The Java classes we use for demonstration are based [very

loosely] on Ninja prowess and skillpublic int attack (Ninja otherNinja, IMove move) throws NinjaException {

int damageInflicted = attemptMove(move); otherNinja.suffer(move); return damageInflicted;

}

EntwicklerCamp 2016

The Ninja Classes

EntwicklerCamp 2016

The Java Debug Console

EntwicklerCamp 2016

What Is Debugging?• Kind of a silly question, but one we need to address

• Many possible answers: ▪ Logging

▪ Capturing errors

▪ Reproducing problems

▪ Stepping through code

▪ Looking at crash dumps

EntwicklerCamp 2016

Classic Notes Java Debugging

System.out.println() !!

EntwicklerCamp 2016

The Lowly Java Console• Captures System.out and System.err messages from Java agent

code run on the Notes client

• Tools – Show Java Debug Console

• Problem with DDE: sometimes the debug console won’t open ▪ Solution: completely shut down the Notes client and DDE, restart

Notes, launch the debug console before you open DDE

EntwicklerCamp 2016

Debug Console Tricks• Type “v” for Java Version info

EntwicklerCamp 2016

Debug Console Tricks (cont.)• “m” for memory available, “f” for finalization, “g” for garbage

collection

EntwicklerCamp 2016

Debug Console Tricks (cont.)• “p” for system properties

EntwicklerCamp 2016

Debug Console Tricks (cont.)• “control-alt-e” for the Euro symbol

▪ Just some crazy thing my Windows XP machine does (might not work for you)

EntwicklerCamp 2016

That’s Great, But …• It’s not much more than seeing the output of “print” statements

▪ Only from Agents, not XPages

• Since Notes 7, we have the ability to attach a real Java debugger to the client ▪ Including, but not limited to, Eclipse

▪ Set breakpoints, step through code, inspect variables, etc.

EntwicklerCamp 2016

Eclipse or DDE?• DDE comes with the Eclipse debugger, so you have a choice

• Sometimes it’s more convenient to use standalone Eclipse, sometimes DDE

• I will show you both, starting with Eclipse

EntwicklerCamp 2016

Debugging Java agents and script libraries

EntwicklerCamp 2016

Setting up the Notes Client for Debugging• Tools – Java Debugging Preferences

EntwicklerCamp 2016

Setting up the Notes Client for Debugging (cont.)• Select the options and ports you want

▪ “Client Agents” are triggered agents

▪ “Locally Scheduled Agents” are just what they sound like

▪ “Http Preview” is Java code run from a Web page, served by the local nhttp process

EntwicklerCamp 2016

Setting up the Notes Client for Debugging (cont.)• Client restart is required when you make changes

EntwicklerCamp 2016

Check Your Firewall Settings• Nlnotes.exe must be allowed to use those ports in your

workstation firewall settings

EntwicklerCamp 2016

Check Your Firewall Settings (cont.)

EntwicklerCamp 2016

Create a Java Agent• Open a database in DDE, and go to the Code – Agents section • Click the “New Agent” button

EntwicklerCamp 2016

Create a Java Agent (cont.)• Give the agent a name, and make sure the “Type” is set to

“Java”

EntwicklerCamp 2016

Create a Java Agent (cont.)• Import the “NinjaClasses” script library into the agent

EntwicklerCamp 2016

Create a Java Agent (cont.)• Double-click the “JavaAgent.java” entry in the Agent Contents

list

EntwicklerCamp 2016

Create a Java Agent (cont.)• Add some code to the agent

EntwicklerCamp 2016

Create a Java Agent (cont.)• Close the “JavaAgent.java” tab, and choose “Yes” when you are

prompted to save your changes

EntwicklerCamp 2016

Create a Java Agent (cont.)• Make sure these Agent Properties are set:

▪ Target = “None” (standard agent setting)

▪ “Compile Java code with debugging information” checked

▪ See screenshot on next slide …

• The “Compile with debugging information” is very important, and is NOT checked by default ▪ Allows the debugger to know what line it’s on after the source code has been

compiled to bytecode

▪ Adds line numbers and method info to stack trace

▪ No performance loss

EntwicklerCamp 2016

Create a Java Agent (cont.)

EntwicklerCamp 2016

Create a Java Agent (cont.)• Click the “Export” button to export the agent code to a local file

EntwicklerCamp 2016

Create a Java Agent (cont.)• Save and close the Java agent

• It can now be run from the Actions menu of the database

• If you want, you can open the debug console and see the System.out message get printed when the agent runs! Amazing!

• Now, let’s set up the debugger …

EntwicklerCamp 2016

Install Eclipse• Make sure Java (standalone) is installed on your workstation

▪ Java 1.5 or higher (I use 1.6)

▪ JRE will work, but you may as well use a full JDK

• Download a version of Eclipse 3.4 (Ganymede) or higher ▪ www.eclipse.org/downloads

• This is simply a ZIP file that must be unzippped ▪ Do NOT use native Windows unzip. 7-Zip is a good choice.

• www.7-zip.org

▪ It is better to install Eclipse in a short folder name (like c:\eclipse)

EntwicklerCamp 2016

A Word About Versions• I usually use Java 1.6 and Eclipse 3.4, because that matches the

versions of Java and Eclipse in the Notes 9 client ▪ when I write Eclipse plugins I don’t have compatibility issues

• All these things should work with newer versions of Java and Eclipse too! ▪ Eclipse 4 is much sexier than Eclipse 3

EntwicklerCamp 2016

Launch Eclipse• Double-click “eclipse.exe”

EntwicklerCamp 2016

Launch Eclipse (cont.)• Create a new Workspace for our debugger project

EntwicklerCamp 2016

Launch Eclipse (cont.)

EntwicklerCamp 2016

Create a New Java Project• File – New – Java Project

EntwicklerCamp 2016

Create a New Java Project (cont.)• Name the project, click “Finish”

EntwicklerCamp 2016

Create a New Java Project (cont.)• Right-click the new project in Package Explorer and choose the

menu option “Import”

EntwicklerCamp 2016

Create a New Java Project (cont.)• Use an import source of:

General – File System

EntwicklerCamp 2016

Create a New Java Project (cont.)• Select the directory you exported the Java agent to earlier, and

all the files in that directory

• Import into folder: “ProjectName/src”

• Click “Finish”

EntwicklerCamp 2016

Create a New Java Project (cont.)■ The project

will now look like this:

EntwicklerCamp 2016

Add Missing JAR Files to Java Build Path• Not strictly necessary, but gets rid of all those red “X”’s • Choose the menu item Project – Properties

EntwicklerCamp 2016

Add Missing JAR Files to Java Build Path (cont.)• Select “Java Build Path” from the left-hand list

• Go to the “Libraries” tab

• Click the “Add External JARs…” button ▪ Select Notes.jar from your Notes\jvm\lib\ext directory

▪ Select Ninja.jar from wherever you have that stored

• NOTE: or you can import the Ninja class source code directly into your project

• Click “OK” to save the changes

EntwicklerCamp 2016

Add Missing JAR Files to Java Build Path (cont.)

EntwicklerCamp 2016

Show Line Numbers• Eclipse doesn’t usually show line numbers by default

• Right-click the “gutter” just to the left of the code (where line numbers should be) and choose the menu option “Show line numbers”

EntwicklerCamp 2016

Open the Debug Perspective• Window – Open Perspective – Debug

EntwicklerCamp 2016

Open the Debug Perspective (cont.)

EntwicklerCamp 2016

Can We Start Debugging Now?• Almost there …

EntwicklerCamp 2016

Set a Breakpoint in the Code• Double-click a line number to toggle the breakpoint at that line

▪ The breakpoint will be shown as a small blue ball in the gutter area to the left of the line numbers

EntwicklerCamp 2016

Create a Debug Configuration• Click the down-arrow next to the small “bug” icon in the task

bar

• Then choose “Debug Configurations…” from the drop-down menu

EntwicklerCamp 2016

Create a Debug Configuration (cont.)• Right-click “Remote Java Application” in the Debug

Configuration list, and choose “New”

EntwicklerCamp 2016

Create a Debug Configuration (cont.)• Use the following settings:

▪ Name: JavaAgent

▪ Connection Type: Standard (Socket Attach)

▪ Host: localhost

▪ Port: 8701

• Click “Apply” to save your changes

EntwicklerCamp 2016

Create a Debug Configuration (cont.)

EntwicklerCamp 2016

Start Debugging• Make sure the Notes client is running

• Click the “Debug” button at the bottom of the Debug Configuration dialog ▪ If the debugger attaches

successfully, you will see something like this:

EntwicklerCamp 2016

Errors Starting the Debugger?• Errors in Workspace

▪ You can ignore this if you get it. It just means that there are unresolved dependencies or something in the code you imported. Not important for the debugging we’re doing.

EntwicklerCamp 2016

Errors Starting the Debugger? (cont.)• Failed to connect to remote VM

▪ Make sure the Notes client is running, and that you set the debugging properties properly, and that you restarted the client after you set the properties

▪ Has the Notes JVM started yet? Launch the Java Debug Console to be sure.

▪ Check your firewall

▪ Check your port numbers

▪ Use TCPView from Sysinternals or Windows 8 Resource Monitor to determine which ports are currently in use

EntwicklerCamp 2016

Run the Agent!• Open the database in the Notes client

• Choose the menu option Actions – NinjaDebugTest

• When the code hits the breakpoint, the debugger should suspend the agent thread

EntwicklerCamp 2016

Debugging the Live Agent

EntwicklerCamp 2016

Basic Debugging Actions

Step Into, Step Over, Step Return

Disconnect

Resume Use Step Filters

EntwicklerCamp 2016

Step Filters• List of classes or packages to ignore when you do a “Step Into”

▪ So you don’t step into classes you have no source code for

• Good packages to add: ▪ com.ibm.*

▪ COM.ibm.* (if available, case-sensitive)

▪ lotus.domino.*

▪ org.apache.*

EntwicklerCamp 2016

Step Filters (cont.)

EntwicklerCamp 2016

Inspecting Variables• Variables and objects for the current code context can be found

in the “Variables” view of the Debug perspective ▪ By default, this will be in the top right of the perspective

▪ Double-click the tab to maximize it temporarily

EntwicklerCamp 2016

Inspecting Variables (cont.)• If you select a variable or object in the list, its string value will be

displayed in the bottom pane of the Variables view

EntwicklerCamp 2016

Inspecting Variables (cont.)• If you click on a scalar value in the Variables list, you can modify

that value for the current run of the code

EntwicklerCamp 2016

Evaluating Code• The “Display” console allows you to execute small bits of code

to evaluate them against the current context and see the result

• To open the Display view, choose the menu option Window – Show View – Display

EntwicklerCamp 2016

Evaluating Code (cont.)• In the Display console, you can type arbitrary expressions to be

evaluated ▪ Type the expression in the console view

▪ Select the entire expression (as though you were about to copy to the clipboard)

▪ Click the “Display result of evaluating selected text” button in the Display console toolbar

EntwicklerCamp 2016

Evaluating Code (cont.)• If the expression you run changes a value in an object, that

value will remain changed when the code continues

EntwicklerCamp 2016

When You’re Done …• Don’t forget to disconnect!

EntwicklerCamp 2016

Other Fun Tricks• The Eclipse debugger has a lot of other interesting (and useful)

bits of functionality ▪ Conditional breakpoints

▪ Exception breakpoints

▪ Detail formatters

▪ Watch values

▪ Copy stack

▪ And more …

• Many good resources on the Internet

EntwicklerCamp 2016

Issues and Problems• You MUST export source code from Notes, and keep it in sync with Eclipse

▪ But wait, you can do this within DDE too! No export needed.

▪ We will see this next …

• Script libraries must also be exported in order to debug attached code

• Once the debugger is running, it will hit the line number breakpoint for ALL agents that run, not just the one you want to debug

• Firewalls can get in the way

• Use Sysinternals TCPView or Windows 8 Resource Monitor to check ports

EntwicklerCamp 2016

TIME CHECK 16:45?

EntwicklerCamp 2016

Debugging Java in XPages

EntwicklerCamp 2016

Debugging Java in XPages• Java can be called from SSJS on an XPage

• If it’s your own custom Java class (as source code), it can be debugged

• NOTE: debugging JavaScript is outside the scope of this session; we’re only talking about Java here

EntwicklerCamp 2016

Notes 8.5 vs 9.0• The steps in this section are for Notes/DDE 9.0

• If you are still using DDE 8.5, there are some differences: ▪ the Java design element did not exist until 8.5.3

▪ the JAR file design element did not exist until 9.0

▪ you can still add Java code and JAR files for XPages in older versions of 8.5, but you have to use the Package Explorer view and modify the Build Path

▪ http://xpageswiki.com/apps/wiki-xpages/wiki-xpages.nsf/dx/Work_with_Java

EntwicklerCamp 2016

Import the Ninja JAR File

EntwicklerCamp 2016

Create an XPage-able Java Class

EntwicklerCamp 2016

Create an XPage-able Java Class (cont.)• Give the new class a Package

and a Name:

EntwicklerCamp 2016

Create an XPage-able Java Class (cont.)• The new class file should now be open in DDE for editing

EntwicklerCamp 2016

Create an XPage-able Java Class (cont.)• Add some code to the class and save it

EntwicklerCamp 2016

Create a New XPage• Back in the Applications view in DDE, go to the “XPages” section

of your database and click the “New XPage” button

EntwicklerCamp 2016

Create a New XPage (cont.)• Give the XPage a name, and click “OK”

EntwicklerCamp 2016

Create a New XPage (cont.)• Drag a new Label control to the XPage, and give it a “Label”

value of “Random Ninja Move:”

EntwicklerCamp 2016

Create a New XPage (cont.)• Drag a Computed Field control to the XPage, and give it a

JavaScript value as shown below

EntwicklerCamp 2016

Create a New XPage (cont.)• Save the XPage and test it in a browser

EntwicklerCamp 2016

Setting Up Debugging• Add the following lines to your Domino server’s Notes.ini

▪ This will require at least a “restart task http” to take effect

▪ The address=9700 variable is the port that debuggers will attach to

▪ As always, check your firewall rules

JavaEnableDebug=1 JavaDebugOptions=transport=dt_socket,server=y,suspend=n,address=9700

EntwicklerCamp 2016

Setting Up Debugging (cont.)• Open your test Java class in DDE again

• Click the down arrow beside the GREEN “bug” icon in the toolbar, and choose “Debug Configurations…”

EntwicklerCamp 2016

Setting Up Debugging (cont.)• In the Debug Configurations dialog, right-click the Remote Java

Application item and choose “New”

EntwicklerCamp 2016

Setting Up Debugging (cont.)• Adjust the “host” and “port” as needed, and click “Apply”

This is the port from the notes.ini JavaDebugOptions

EntwicklerCamp 2016

Setting Up Debugging (cont.)• Make sure you’ve restarted the Domino server or the HTTP task

after you added the Notes.ini debug entries • Click the “Debug” button in the Debug Configuration dialog

EntwicklerCamp 2016

Setting Up Debugging (cont.)• Double-click a line number in the Java code to add or remove

the breakpoint at that line ▪ The breakpoint will be

shown as a small blue ball in the gutter area to the left

EntwicklerCamp 2016

Debug the Code!• Refresh the XPage in a browser. When your breakpoint gets

triggered, you will see a dialog in DDE asking if you want to switch to the Debug Perspective. Choose “Yes.”

EntwicklerCamp 2016

Debug the Code!

EntwicklerCamp 2016

Standard Eclipse Debugging• From there, you can use standard Eclipse debugging as we

demonstrated earlier • Troubleshooting is the same too:

▪ Check the firewall port settings

▪ Check active ports in TCPView or similar

▪ Etc.

• Don’t forget to disconnect! • You can switch from the Debug perspective back to the “normal”

DDE perspective with Window – Open Perspective – Domino Designer

EntwicklerCamp 2016

Things to Watch Out For• The debug port will only allow one connection at a time

• The debug port is incredibly insecure – ANYONE with access to your server can connect to it

• There is no good way to specify (to the server) which piece of code you want to debug. If you set a breakpoint at line 10, then ALL Java XPages code will pause at line 10.

• As a result of these things, do NOT enable debugging on a production server

EntwicklerCamp 2016

Which One To Use? Eclipse or DDE?

EntwicklerCamp 2016

Nice Things About DDE• Fewer steps involved

• Code is always in sync (no need to export to Eclipse)

• You are already using it!

EntwicklerCamp 2016

Nice Things About Eclipse• Newer versions have newer features and fixes

▪ for debugging these are usually minor

• Runs in a separate memory space ▪ useful for debugging big agents

• If you build JAR files in Eclipse for deployment to Notes/Domino, you can debug from the same place you write the JAR file

• Eclipse is a good tool to have (it can do so many more things)

EntwicklerCamp 2016

Performance Analysis and Troubleshooting

EntwicklerCamp 2016

Code Optimization• Make your code work first, then worry about speed

▪ You always want working code to fall back on

• Is it fast enough? Leave it alone.

• Making code shorter doesn’t make it faster

• “Clever” code is harder to troubleshoot

• Don’t assume you know where the bottleneck is

“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil” - Donald Knuth

EntwicklerCamp 2016

Static Code Analyzers• FindBugs is a great tool for finding potential problems • Bugs grouped by categories and subcategories, like:

▪ Security

▪ Scary

▪ Performance

▪ Dodgy Code

• New cloud option for development teams ▪ Rank bugs, assign fixes

• http://findbugs.sourceforge.net

• http://www.cs.umd.edu/~pugh/MistakesThatMatter.pdf

EntwicklerCamp 2016

FindBugs

EntwicklerCamp 2016

FindBugs

EntwicklerCamp 2016

Common Bottlenecks• Some other guy’s server • Slow database queries • Memory issues

▪ Leaks (less common than you think)

▪ Too much cached information, especially large objects, strings, XML

• Loading and unloading resources • Non-lazy initialization of objects • Slow or broken network connection

EntwicklerCamp 2016

Simulating Poor Network Connections• http://www.charlesproxy.com • http://jagt.github.io/clumsy • Network Link Conditioner (OSX)

▪ XCode > Open Developer Tool > More Developer Tools

EntwicklerCamp 2016

Analyzing Java Memory Usage• Used for:

▪ Tracking down memory leaks

▪ Finding high-memory-use objects (and arrays)

▪ Finding objects that are unexpectedly still in memory

• YourKit Java profiling app ▪ http://yourkit.com

▪ http://notesin9.com/index.php/2012/11/29/notesin9-091-xpages-memory-profiling-part-1

• Eclipse Memory Analysis Tool ▪ http://www.eclipse.org/mat/downloads.php

▪ http://lazynotesguy.net/blog/2013/08/30/wheres-my-memory-gone-peeking-inside-jvms-heap-part-1-installation

▪ http://lazynotesguy.net/blog/2013/10/04/peeking-inside-jvms-heap-part-2-usage

EntwicklerCamp 2016

Analyzing Java Memory Usage• IBM Thread and Dump Analyzer (for javacore files)

▪ https://www.ibm.com/developerworks/community/groups/service/html/communityview?communityUuid=2245aa39-fa5c-4475-b891-14c205f7333c

▪ http://www-01.ibm.com/support/docview.wss?uid=swg27011855&aid=1

• IBM Heap Analyzer (for heapdump files) ▪ https://www.ibm.com/developerworks/community/alphaworks/tech/

heapanalyzer

▪ http://www-01.ibm.com/support/docview.wss?uid=swg27006624&aid=1

EntwicklerCamp 2016

Generating Heap Dumps with Notes/Domino• Domino XPages

▪ tell http xsp heapdump

▪ XPages Toolbox ( http://www.openntf.org/p/XPages%20Toolbox )

• Directly from Java Agents or Classes ▪ com.ibm.jvm.Dump.HeapDump();

▪ writes to notes\framework\heapdump.###.phd when run from an agent

▪ writes to the Domino program directory when run from an XPage

• there will be a message on the server console with the file location

EntwicklerCamp 2016

Generating Heap Dumps with Notes/Domino• Notes Client (Expeditor)

▪ this will give you information about Eclipse plugins

• open a Command Prompt while the Notes client is running

• change to the notes\framework\rcp\ directory

• enter one of the following commands:

• rcplauncher -dcommand_only -com.ibm.rcp.core.logger#dump heap • rcplauncher -dcommand_only -com.ibm.rcp.core.logger#dump threads

▪ writes to notes\data\workspace\logs by default

EntwicklerCamp 2016

Conclusion

EntwicklerCamp 2016

Additional Resources• Eclipse debugging

▪ www.ibm.com/developerworks/library/os-ecbug

▪ www.openntf.org/blogs/openntf.nsf/d6plinks/NHEF-8CCDTQ

▪ eclipsetutorial.sourceforge.net/debugger.html

▪ www.vogella.com/tutorials/EclipseDebugging/article.html

▪ www.cavdar.net/2008/09/13/5-tips-for-debugging-java-code-in-eclipse

▪ blog.codecentric.de/en/2013/04/again-10-tips-on-java-debugging-with-eclipse

EntwicklerCamp 2016 IBM Notes – den EierlegendenWollMilchSau für alle und Immer

Thank you

twitter: @jrobichauxemail: julian.robichaux@panagenda.com