5/19/2005page 1 using an opensource stack to manage linux peter karnazes, phd cmg meeting...

5/19/2005

Using an OpenSource Stack to Manage Linux

Peter Karnazes, PhDCMG MeetingPhiladelphiaMay 19, 2006

5/19/2005

Agenda

• Open Source Software

• IT Data Center Management Problem

• IT Management Tools Characteristics– Commercial Tools

– Open Source Tools

• Why are Companies Moving to Open Source

• Open Source Management Stack – Example

• OMC – Open Management Consortium

• Summary

5/19/2005

Open Source Software

How successful is the

Open Source movement???

5/19/2005

Open Source Products

Many open source products are already successful

Operating System Linux

Web Server Apache

Application Server JBoss

Development Tools Eclipse (3DVE)

Database Application MySQL

5/19/2005

Open Source Software Maturity

2004 2005 2006 2007

OPERATINGSYSTEM

WEBSERVER

EMAILSERVER

DEVELOPMENTTOOLS

APPLICATIONSERVER

RDBMS

BPMENGINE

VIRTUALMACHINES

CONTENTMANAGEMENT

WEBBROWSER

EMAILCLIENT

CRM

SCM

DESKTOPPRODUCTIVITYSUITE

ERP

IT MANAGEMENTTOOLS

INFRASTRUCTURE

APPLICATIONS

5/19/2005

Open Source Customers

Companies using Open SourceIT Management Tools

TimeWarner Cable Charles River Associates

Citizens National Bank Dairyland Power

Linksys DHL

Shell Cingular Wireless

Siemens TicketMaster

5/19/2005

IT Data Center Management

• The Problem– How to manage and monitor a Linux based enterprise

class corporate IT data center and networks given the mission critical requirements

• Traditional solution– Use a proprietary commercial solution

• Today’s alternative– Convert to Open Source management solutions

5/19/2005

Commercial ToolsSolutions

• Leading commercial proprietary solutions– IBM Tivoli

– HP OPenView

– BMC

– CA (Computer Associates)

• Commercial solutions suffer from “functionality overkill”– Overload of features not required by most users

5/19/2005

Commercial ToolsDeployment

• Deployment issues

– Over-deployment -> “turn on everything”

• All the functionality is not actually required

• Requires additional staff to maintain

• Frequently results in poorly tuned alarms and notification schemes

– Under-deployment

• Licensing fees are paid for unused functionality

• Fees are recurring with annual licensing contracts

5/19/2005

• Licensing Fees– Fees can approach 7 figures

• Deployment and system administration can cost 5 times as much as initial licensing fee– Installation can take several months

• Proprietary solutions require specialized programming skills– Companies must maintain deep expertise in proprietary

software or hire pay high cost consulting fees to maintain and upgrade their environment

Commercial ToolsCosts

5/19/2005

• Standards based– Open interfaces

• Modular and configurable– Built on component architectures

• Designed to be easily modifiable– Code source is transparent

• No licensing fee*

Open Source ToolsCharacteristics

5/19/2005

• Managing custom-built applications– Not suited for out-of-the-box solutions

– Configuration required for specific parameters

– Open interfaces make custom management a good fit

• Easy integration– Open source solutions make good master IT managers

– Easier integration of various monitoring and performance management systems

Open Source ToolsAdvantages

5/19/2005

Open Source ToolsAdd-in Functionality

• Software enhancement– High probability that many needed add-in functions

are available online in the open source community

– Development uses open, standard protocols and API’s• Training personnel in standard API’s is a good long-term

investment

• There is a large pool of source developers (as opposed to proprietary specialists) available to hire or as contractors

5/19/2005

• Bug Fixes– The greatest advantage of open source software is the

global testing and peer review process controlled by the open source project

• Source code is available for determining the root cause of the each bug

• A world-wide network of IT professionals submit fixes and enhancements back to the project

• Open source project leaders review and have the fix tested before releasing the fix back into open source

Open Source ToolsCollaborative Development

5/19/2005

Open Source ToolsLower Cost

• Lower cost, both up front and long term TCO, is the key driver for Open Source IT monitoring and management

– Low Acquisition Cost• Core software is free

• Up front cost is for enhancements, services and support

5/19/2005


– Lower Deployment Cost• Companies install only what they need

• Deployments tend to be completed more quickly

– Low system administration overhead• Installations are not burdened by gratuitous features

• Vendor specific training is not required

• Staffs become self-reliant over time

5/19/2005


–Low scaling costs

• Adding new resources (applications, servers, and networks) require new management system add-ons– Commercial products

» Agents or SPIs involve more expenditures if they even exist

» Or they may require development by the vender or a third party

– Open source» No additional cost is incurred

» Features are usually available from the open source community

5/19/2005

Leading Open Source IT Management Tools

• Nagios

• JFFNMS

• OpenNMS

• Big Sister

• Netdisco

• Zabbix

• MRTG

• Ntop

• Syslog NG

• RRDtool

• Cacti

• Nmap

• NeDi

• Cfegine

• Nessus

• Snort

• Kismet

• Webmin

5/19/2005

• Network– Monitoring– Management– Traffic Monitoring– Traffic Analysis– Performance Graphing– Discovery– Inventorying– Configuration– Intrusion Detection– Wireless Detection

• Systems– Monitoring– Performance Graphing

• Remote Security Scanning

• Analytical Graphing

Leading Open Source IT Management Functions

5/19/2005

Why are Companies moving to Open Source

• Merger or Acquisition

– Simpler low cost solution required

– Newly merged IT resources

• New CIO Committed to Open Source

– New management understands the advantages of open source

• Corporate Mandate to Lower IT Costs

– IT operations face severe spending reductions

• Corporate Mandate for Open Source Software

– Company policy adopts an open source software policy

– Governments mandate exclusive use of open source software

5/19/2005

How to Create an Open Source Management Solution

• The Open Source community provides the pieces…

• The solution must be integrated by– A solution supplier

– Your IT organization

– Third party consultants

5/19/2005

Example – IT Management Solution Supplier

• Groundwork creates a totally integrated IT infrastructure monitoring and management solution using open source components

– Core components from Nagios and MySQL

– PHP portal based web interface

– Management level reporting

– Configuration tools

– Monitoring tools

5/19/2005

Groundwork Monitor

PERFORMANCE AND AVAILABILITY REPORTS

INTEGRATEDCONSOLE

MONITORING ANDCONFIGURATION

UIEXTENSIONS

PORTAL FRAMEWORK (Guava)

WEB PORTAL UI

GROUNDWORK FOUNDATION

APPLICATION PROGRAMMING INTERFACES (APIs)

COMMON DATA MODEL

ADAPTERS

DATABASE(MySQL)

AVAILABILITYMONITORING

Nagios

ALARMPROCESSING

Nagios

PERFORMANCEMONITORING

RRDtool

SNMP TRAPSSNMP TT

SYSTEM LOGSSyslog NG

OPEN SOURCE TOOLS 3rd PARTY SYSTEMS

NETWORK SYSTEMSAPPLICATION MGT

CONFIGURATION MGTSERVICE DESK

NETWORKS SERVERS DATABASES APPLICATIONS OTHER DEVICES

5/19/2005

Open Management Consortium

5/19/2005

Open Management Consortium

• May 9, 2006 - Six Leading open source systems and network management vendors formed the OMC

• Nagios Ayamon

• NetDirector Emu Software

• openQRM Qlusters

• openSIMS Symbiot

• Webmin Zenoss

• Zenoss Zenoss

5/19/2005

Nagios

• Nagios is a host and service monitor designed to manage and track network and host resources

• Works well with Linux and most *NIX operating systems

• A Monitoring daemon runs intermittent checks on hosts and services – uses external plugins which return status information

5/19/2005

Nagios

• Monitors– Host resources

• processor load, disk and memory usage, running processes, log files…

– Network services• SMTP, POP3, HTTP, NNTP, PING…

– Environmental factors• Temperature, humidity…

– Plug-in design allows development of custom host and service checks

• Able to design custom event handlers

• Web interface for remote viewing of status and problems

5/19/2005

NetDirector

• NetDirector is framework for managing configurations of common open source network services.

• Works well with Linux and most *NIX operating systems

• Network Configuration changes for single or multiple servers – Can be immediate or scheduled

5/19/2005

NetDirector

Manages the following

• HTTP (Apache)

• DNS (BIND)

• DHCP

• File and Print Services (Samba, NFS)

• Email (Postfix, Sendmail)

• FTP (VSFTP)

• Users and Groups (mass add users to multiple servers at once)

• Network Interfaces

Other Features

• Rollbacks (Revision Control) Troubleshooting/Logs

• Schedule Changes

• Server configuration cloning

• Intuitive integrated support for managing clustered servers and disaster recovery configurations

• Track Past or Scheduled Changes to be done

• Track who made what changes and when

5/19/2005

openQRM

• openQRM is a data center management platform with the following features

–Data Center Management• Manages thousands of servers• Generates custome reports

–Automatic, policy based provisioning

–Maintenance Tasks

–Extensible with plug-ins

5/19/2005

Zenoss

• Zenoss is composed of several different open source components

– Current version is monitor focused

– Future versions will develop deeper “management” functionality

5/19/2005

openSIMS

• openSIMS - Open Source Security Infrastructure Management System – OpenSIMS provides a way for tying together the open source

tools used for security management into a common infrastructure. It leads toward having different networks use risk metrics to collaborate on attacker profiling and remediation.

– Current version is 0.9 – still in beta

5/19/2005

Summary

• Open Source has proved to be a viable source for enterprise class software

• IT Management Software is entering the mature stage of Open Source development

• Total Cost of Ownership for IT Management Software can be significantly reduced by transitioning a data center’s infrastructure to Open Source

5/19/2005

Thank You

Questions?

Peter Karnazes, PhDCMG MeetingPhiladelphiaMay 19, 2006

5/19/2005page 1 using an opensource stack to manage linux peter karnazes, phd cmg meeting...

Documents

open source movement

modifiablecode source

custom management

proprietary software

good master

monthsproprietary solutions

good longterm investmentthere

transparentno licensing