advanced weblogic monitoring: jmx and wlsdm automation

40
1 - 1 M.FEVZİ KORKUTATA Integration and Infrastructure Architect Recent Projects: •Java Server Loader (JSL) www.admineer.com/blog •WebLogic Monitoring Infrastructure Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA) Advanced WebLogic Monitoring: JMX MBean Development & WLSDM Automation

Upload: m-fevzi-korkutata

Post on 16-Feb-2017

856 views

Category:

Technology


2 download

TRANSCRIPT

1 - 1

M.FEVZİ KORKUTATAIntegration and Infrastructure

Architect

Recent Projects:

•Java Server Loader (JSL)•www.admineer.com/blog•WebLogic Monitoring Infrastructure

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

Advanced WebLogic Monitoring:

JMX MBean Development &

WLSDM Automation

1 - 2

Advanced WebLogic Monitoring: JMX MBean Development & WLSDM Automation

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

Domain Structure – 1:

1.Applied Tools & Technologies2.Traditional WebLogic Monitoring3.Advanced WebLogic Monitoring4.JMX and MBean Objects5.Important WebLogic JMX MBean Objects6.Monitoring WebLogic Managed Servers7.Monitoring WebLogic Deployments8.Monitoring WebLogic Data Sources9.Monitoring WebLogic JMS Servers10. Monitoring WebLogic Thread Pool (HOGGERs/STUCKs)

1 - 3

Advanced WebLogic Monitoring: JMX MBean Development & WLSDM Automation

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

Domain Structure – 2:

11.JMC (Java Mission Control), JFR and WebLogic12.Monitoring WebLogic Backend (SQL, EJB, WebService... etc.)13.JMX MBean Development14.Register and Deploy JMX MBeans to WebLogic15.DEMO: Monitoring WebLogic Domain Resources 16.DEMO: Live Load Test by using JSL (Java Server Loader) and WebLogic Behavior17.DEMO: JMX MBean Registration & Deployment18.DEMO: Trigger Custom Scripts by using JMX and WLSDM19.Summary20.Questions & Answers

1 - 4

1. Applied Tools & Technologies

• Oracle WebLogic Server (12c: 12.1.3)• JDK: Java 7 (jdk1.7.0_80)• JMC: Java Mission Control (5.5)• Oracle MySQL Database (5.6)• Apache James Mail Server• WLSDM: WebLogic Smart Dashboard and Monitoring (v.2.3.1)

by WLSDM• JSL: Java Server Loader (v.1.0.9) by ADMINEER (Fevzi Korkutata)

• SuperStore DEMO Web Application by Fevzi Korkutata

• WLSDMMonitoringSamples JMX Application by WLSDM

• StuckThreadForFree Web Application by Frank Munz

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 5

1. Applied Tools & Technologies

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 6

2. Traditional WebLogic Monitoring

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

• Problem Occurred..! Really?

• Has «real» ALARM? Do we need?

• Why? Root Cause? Don’t know.

• Service Lost! Never mind...

• Unique and Ultimate Solution(!)

(Re)Start Managed Servers Manual Human Task

1 - 7

3. Advanced WebLogic Monitoring

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

• Problem Occurred !!! How come?

• Notify Instantly• Understand what’s going on via mobile• No Service Lost: Proactive WebLogic Domain Settings

(Timeout, Panic action... etc.)• I.E. Inform 3rd party and PartnerLinks automatically

Create automation scripts, notifications... (Restart managed servers, datasources... etc. automatically...)

• Identify Root-Cause, take precautions, prevent recurrence...

1 - 8

4. JMX and MBean Objects - 1

• JMX: Java Management Extension– Defines a standard infrastructure to manage a device from

Java programs– Decouples the managed device from management tools

• MBeans: The building blocks of JMX

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 9

4. JMX and MBean Objects - 2

JMX Connection/Monitoring Tools:

•WebLogic Monitoring Dashboard•JVisualVM•JMC (Java Mission Control)•WLSDM

JVM Remote JMX Connection Parameters:-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=7099 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 10

4.1. WebLogic, JMX, MBean Object, MBean Instance - 1

• WebLogic server uses JMX extensively internally to handle the configruation and state of the system.

• MBean Object vs. MBean Instance

• WebLogic Sever naming conventions encode its MBean object names as follows:

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

com.bea:Name=name,Type=type[,TypeOfParentMBean=NameOfParentMBean] [,TypeOfParentMBean1=NameOfParentMBean1]...

1 - 11

4.1. WebLogic, JMX, MBean Object, MBean Instance - 2

• com.bea: is the JMX domain name.

• For WebLogic Server MBeans, the JMX domain is always com.bea. If you create custom MBeans for your applications, name them with your own JMX domain.

• Example:

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 12

5. Important WebLogic JMX MBean Objects

• Separate MBean Types for Monitoring and Configuring

– Run-time Mbeans:

– Configuration Mbeans:

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 13

6. Monitoring WebLogic Managed Servers

• Health and State Monitoring

• Potential HEALTH problems?• Potential STATE problems?

• Panic Action:– WebLogic 12.1.3 Default Action Ignore– WebLogic 12.2.1 Default Action Exit

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

jvmOutOfMemory ??? WebLogic Behavior

1 - 14

6. Server JMX MBean Metrics

• OpenSocketsCurrentCount @ ServerRuntimeMBean

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 15

6. Server JMX MBean Metrics

• HoggingThreadCount @ ThreadPoolRuntimeMBean

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 16

6. HoggingThreadCountNotification

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 17

6. Server JMX MBean Metrics

• ActiveExecuteThreads @ ThreadPoolRuntimeMBean

• PendingUserRequestCount @ ThreadPoolRuntimeMBean

• ActiveTransactionsTotalCount @ JTARuntimeMBean

• HeapFreePercent @ JVMRuntimeMBean

• Throughput @ ThreadPoolRuntimeMBean

• TransactionRolledBackTotalCount @ JTARuntimeMBean

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 18

7. Monitoring WebLogic Deployments

• Deployments: Health and State Monitoring

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 19

7. Monitoring WebLogic Deployments

• OpenSessionsCurrentCount @ WebAppComponentRuntimeMBean

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 20

7. Monitoring WebLogic Deployments

• CompletedRequests @ WorkManagerRuntimeMBean

• PendingRequests @ WorkManagerRuntimeMBean

• StuckThreadCount @ WorkManagerRuntimeMBean

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 21

8. Monitoring WebLogic Data Sources

• Data Sources: Health and State Monitoring

• Data Source Suspended ???

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 22

8. Monitoring WebLogic Data Sources

• LeakedConnectionCount @ JDBCDataSourceRuntimeMBean

• if (LeakedConnectionCount > 0) ???

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 23

8. Monitoring WebLogic Data Sources

• ActiveConnectionsCurrentCount @ JDBCDataSourceRuntimeMBean

• ConnectionDelayTime @ JDBCDataSourceRuntimeMBean

• FailedReserveRequestCount @ JDBCDataSourceRuntimeMBean

• WaitingForConnectionCurrentCount @ JDBCDataSourceRuntimeMBean

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 24

9. Monitoring WebLogic JMS Servers

• JMS Health and State Monitoring

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 25

9. Monitoring WebLogic JMS Servers

• MessagesPendingCount @ JMSServerRuntimeMBean

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 26

10. Monitoring WebLogic Thread Pool(HOGGERs / STUCKs)

• HoggingThreadCount @ ThreadPoolRuntimeMBean

• StuckThreadCount @ WorkManagerRuntimeMBean

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

Administrator’s Nightmare

1 - 27

11. JMC (Java Mission Control), JFR and WebLogic

• JRockit vs. JDK

• JVM JMC Parameters:– -XX:+UnlockCommercialFeatures -XX:+FlightRecorder

• JDK 1.7_040 Higher JMC

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 28

11. JMC (Java Mission Control), JFR and WebLogic

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 29

12. Monitoring WebLogic Backend (SQL, EJB, Web Services... etc.)

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 30

12. Monitoring WebLogic Backend (SQL, EJB, Web Services... etc.)

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 31

13. JMX MBean Development

• Mbean Interface: DiskSpaceWatcherMBean

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 32

13. JMX MBean Development

• Mbean Class: DiskSpaceWatcher

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 33

13. JMX MBean Development

• Register Mbean to WebLogic• Listener Class: DiskSpaceWatcherListener.java

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 34

14. Deploy JMX MBeans to WebLogic

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 35

15. DEMO: Monitoring WebLogic Domain Resources

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 36

16. DEMO: Live Load Test by using JSL (Java Server Loader)

and WebLogic Behavior

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 37

17. DEMO: JMX MBean Registration & Deployment

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 38

18. DEMO: Trigger Custom Scripts by using JMX and WLSDM

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 39

19. SUMMARY

• Understanding JMX MBean Metrics increase WebLogic Administration knowledge

• WebLogic Mbean values assign JVM System Resource Values (Memory, System CPU, JVM CPU...). So, monitoring these WebLogic Mbeans ic crucial for mission critical WebLogic domains.

• Proactive monitoring and automations is mandatory for DevOps management.

• Try to understand your WebLogic domains’ characteristics and behavior

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)

1 - 40

THANK YOU..!

Questions & Answers

Fevzi Korkutata, Oracle ACE Associate (Middleware & SOA)