the fusioninventory projectguillomovitch/presentations/fosdem2012.pdf · glpi php plugin...

ProjectCoding challengesTesting challenges

The FusionInventory projectA perl hacker perspective

Guillaume Rousse

FOSDEM 2012

Guillaume Rousse The FusionInventory project

Outline

1 ProjectHistoryAgent

2 Coding challengesLegacy codeEndless code traps

3 Testing challengesTesting HTTP server interactionsTesting system interactionsTesting modules compilation everywhere

HistoryAgent

Outline

HistoryAgent

OverviewIT assets inventory solution

inventoryhelpdesk

ArchitecturePHP applicationmodular

HistoryAgent

OverviewIT assets inventory solution

inventoryhelpdesk

ArchitecturePHP applicationmodular

HistoryAgent

OCS-NG

OverviewAutonomous IT assets inventory solution

inventorysoftware deployment

ArchitectureServer side:

Perl agent interfacePHP user application

Agent side:Windows C agentUnix Perl agent

HistoryAgent

OCS-NG

OverviewAutonomous IT assets inventory solution

inventorysoftware deployment

ArchitectureServer side:

Perl agent interfacePHP user application

Agent side:Windows C agentUnix Perl agent

HistoryAgent

Tracker

OverviewGLPI extension to deal with agentless devices

device discoveryremote inventory

ArchitecturePHP GLPI pluginPerl agent

HistoryAgent

Tracker

OverviewGLPI extension to deal with agentless devices

device discoveryremote inventory

ArchitecturePHP GLPI pluginPerl agent

HistoryAgent

Yesterday

HistoryAgent

FusionInventory

OverviewGeneric GLPI extension

merge of Unix OCS agent with Tracker agentmodular design

ArchitectureGLPI PHP pluginmulti-platform Perl agent

HistoryAgent

FusionInventory

OverviewGeneric GLPI extension

merge of Unix OCS agent with Tracker agentmodular design

ArchitectureGLPI PHP pluginmulti-platform Perl agent

HistoryAgent

Presentation

Componentsenginetasks

Task sampleslocal inventorynetwork discoveryremote network inventoryremote vmware inventorywake on lansoftware deployment

HistoryAgent

Presentation

Componentsenginetasks

Task sampleslocal inventorynetwork discoveryremote network inventoryremote vmware inventorywake on lansoftware deployment

HistoryAgent

Local inventory task

Hardware componentsCPU, memory, etc...USB, PCI, SCSI busesconnected devices

Software componentsoperating systeminstalled software

Configuration

network configurationdisplay configurationenvironment variables

HistoryAgent

Configuration

HistoryAgent

Configuration

HistoryAgent

Network discovery

Network scanSNMP scannmap scannetbios scan

Device identificationSNMP fingerprinting

HistoryAgent

Network discovery

Network scanSNMP scannmap scannetbios scan

Device identificationSNMP fingerprinting

HistoryAgent

Network inventory

Printersink levelpage counters

Networking deviceports listvlans listconnected devices

HistoryAgent

Network inventory

Printersink levelpage counters

Networking deviceports listvlans listconnected devices

HistoryAgent

Server communication

Legacy OCS protocol

HTTP POST requestsXML format

FusionInventory protocolHTTP GET requestsJSON format

HistoryAgent

Server communication

Legacy OCS protocol

HTTP POST requestsXML format

FusionInventory protocolHTTP GET requestsJSON format

HistoryAgent

Not just GLPI

Other management solutionsRudderUranosOTRSPulse2

HistoryAgent

Timeline

Versions2.0.x initial release2.1.x windows support2.2.x refactoring

roadmapswitch from XML to JSONswitch to POEmemory footprint reduction

HistoryAgent

Timeline

Versions2.0.x initial release2.1.x windows support2.2.x refactoring

roadmapswitch from XML to JSONswitch to POEmemory footprint reduction

Legacy codeEndless code traps

Outline

Sysadmin-style code

Look ’ma, I’m a shell guru

# HPUX sof tware l i s t e x t r a c t i o n@sof tL is t = ‘ s w l i s t | grep −v ’ ^ PH ’ | grep −v ’ ^# ’ | t r −s " \ t " " " | t r −s " " ‘

What’s this syslog thing anyway ?

set logsock ( ’ un ix ’ ) ;openlog ( " fus ion inven to ry−agent " , ’ cons , p id ’ , $ENV{ ’USER ’ } ) ;sys log ( ’ debug ’ , ’ sys log backend enabled ’ ) ;c lose log ( ) ;

Sysadmin-style code

Look ’ma, I’m a shell guru

# HPUX sof tware l i s t e x t r a c t i o n@sof tL is t = ‘ s w l i s t | grep −v ’ ^ PH ’ | grep −v ’ ^# ’ | t r −s " \ t " " " | t r −s " " ‘

What’s this syslog thing anyway ?

set logsock ( ’ un ix ’ ) ;openlog ( " fus ion inven to ry−agent " , ’ cons , p id ’ , $ENV{ ’USER ’ } ) ;sys log ( ’ debug ’ , ’ sys log backend enabled ’ ) ;c lose log ( ) ;

Creative hacks

Dynamic function export

my $backendSharedFuncs = {can_run => sub {

. . .} ,

foreach my $package ( @packages ) {foreach my $func ( keys %{$backendSharedFuncs } ) {

$package−>{$func } = $backendSharedFuncs−>{$func } ;}

Portability concerns

Contextthe agent runs an HTTP listener threadit forks itself to perform its duty

# h t t p server main thread codesub DESTROY {

$_[0]−>{ l i s t e n e r }−> k i l l ( ’ KILL ’ ) i f $_[0]−>{ l i s t e n e r } ;}

# h t t p server l i s t e n e r thread codethreads−>se t_ th read_ex i t_on l y ( 1 ) ;$SIG { ’ KILL ’ } = sub { threads−>ex i t ( ) } ;

Trapfork() is emulated on Windows: the listener thread is lost

Underlying code concerns

Contextthe network discovery task is multithreadedeach thread reports its result to the GLPI server

TrapOpenSSL is not thread-safe: crash with HTTPS

Underlying code concerns

Contextthe network discovery task is multithreadedeach thread reports its result to the GLPI server

TrapOpenSSL is not thread-safe: crash with HTTPS

LWP HTTPS support

Multiple SSL socket implementationsNet::SSL: certification authorityIO::Socket::SSL: certification authority, server hostname

Multiple LWP versionsLWP 5.x: no server certificate checkingLWP 6.x: server certificate checkingLWP 5.x on fedora: LWP 6.x behaviour backport

LWP HTTPS support

Multiple SSL socket implementationsNet::SSL: certification authorityIO::Socket::SSL: certification authority, server hostname

Multiple LWP versionsLWP 5.x: no server certificate checkingLWP 6.x: server certificate checkingLWP 5.x on fedora: LWP 6.x behaviour backport

Testing HTTP server interactionsTesting system interactionsTesting modules compilation everywhere

Outline

Overview

GoalTo test interactions with an HTTP server

Examplesproxy usageSSL usagecertificates checking

Overview

GoalTo test interactions with an HTTP server

Examplesproxy usageSSL usagecertificates checking

Using a real server

Prosall features available

Conslimited availabilitylimited variability

Using a real server

Prosall features available

Conslimited availabilitylimited variability

Running a test server

Prosfull availabilityfull variability

Conslimited features

Used modulesHTTP::Server::SimpleHTTP::Server::Simple::AuthenIO::Socket::SSLHTTP::Proxy

Mocking the user agent

Test::MockObjectNot tested

Testing command output parsing

GoalTesting the code parsing some command output

Initial code

my @output = ‘command ‘ ;foreach my $ l i n e ( @output ) {

i f ( $ l i n e =~ / ^ foo : ( . ∗ ) / ) {$ inventory−>addItem ( $1 ) ;

Testing command output parsing

GoalTesting the code parsing some command output

Initial code

my @output = ‘command ‘ ;foreach my $ l i n e ( @output ) {

i f ( $ l i n e =~ / ^ foo : ( . ∗ ) / ) {$ inventory−>addItem ( $1 ) ;

Abstraction layer

getFileHandle

sub getF i leHandle {my %params = @_;

my $handle ;

SWITCH: {i f ( $params { f i l e } ) {

i f ( ! open $handle , ’ < ’ , $params { f i l e } ) {carp "Can ’ t open f i l e $params { f i l e } : $ERRNO" ;return ;

}l as t SWITCH;

}i f ( $params {command } ) {

i f ( ! open $handle , ’−| ’ , $params {command} . " 2 >/dev / n u l l " ) {carp "Can ’ t run command $params {command } : $ERRNO" ;return ;

}l as t SWITCH;

}die " n e i t h e r command nor f i l e parameter given " ;

return $handle ;}

Code adaptation

Intermediate code

$inventory−>addItems ( $_ ) foreach get I tems ( ) ;

sub get I tems {my @output = ‘command ‘ ;my @items ;foreach my $ l i n e ( @output ) {

push @items , $1 i f $ l i n e =~ / ^ foo : ( . ∗ ) / ;}return @items

Code adaptation

Final code

$inventory−>addItems ( $_ ) foreach get I tems (command => ’command ’ ) ;

sub get I tems {my $handle = getF i leHandle (@_) ;while (my $ l i n e = <$handle >) {

push @items , $1 i f $ l i n e =~ / ^ foo : ( . ∗ ) / ;}close $handle ;return @items ;

Result availability concerns

Exotic files and commandsAIX lsvpd

HPUX machinfo

Linux Alpha /proc/cpuinfo

Samples collecting campaingresources treeorganisational concerns (tracability, ...)

Windows issuesregistryWMI

HPUX machinfo

Testing environment adaptation

GoalTesting the code selecting the command to run

Initial code

my @packages =−x ’ / b in / rpm ’ ? getRPMPackagesList (command => ’ rpm −qa ’ ) :−x ’ / b in / dpkg ’ ? getDPKGPackagesList (command => ’ dpkg − l ’ ) :−x ’ / b in / equery ’ ? getEqueryPackagesList (command => ’ equery l i s t − i ’ ) :

( ) ;}

Testing environment adaptation

GoalTesting the code selecting the command to run

Initial code

my @packages =−x ’ / b in / rpm ’ ? getRPMPackagesList (command => ’ rpm −qa ’ ) :−x ’ / b in / dpkg ’ ? getDPKGPackagesList (command => ’ dpkg − l ’ ) :−x ’ / b in / equery ’ ? getEqueryPackagesList (command => ’ equery l i s t − i ’ ) :

( ) ;}

Abstraction layer

canRun

sub canRun {my ( $wanted ) = @_;

return −x $wanted ;}

Code adaptation

Final code

my @packages =canRun ( ’ / b in / rpm ’ ) ? getRPMPackagesList (command => ’ rpm −qa ’ ) :canRun ( ’ / b in / dpkg ’ ) ? getDPKGPackagesList (command => ’ dpkg − l ’ ) :canRun ( ’ / b in / equery ’ ) ? getEqueryPackagesList (command => ’ equery l i s t − i ’ ) :

( ) ;}

Function hijacking

mockGetRun

sub mockCanRun {my (%params ) = @_;

my $new = sub {my $wanted = $_ [ 0 ] ;return $params {commands}−>{$wanted } ;

no warnings ’ rede f i ne ’ ;∗Fus ionInventory : : Agent : : Tools : : canRun = $new ;

Function hijacking

mockGetFileHandle

sub mockGetFileHandle {my (%params ) = @_;

my $old = \& Fus ionInventory : : Agent : : Tools : : ge tF i leHandle ;

my $new = sub {my (%opt ions ) = @_;

my $ f i l e = $params {commands}−>{$wanted } ;

i f ( $ f i l e ) {pr in t STDERR " f i l e ’ $ f i l e ’ de l i ve red \ n " ;return $old−>(@_, f i l e => $ f i l e ) ;

} else {pr in t STDERR " noth ing de l i ve red \ n " ;return ;

no warnings ’ rede f i ne ’ ;∗Fus ionInventory : : Agent : : Tools : : ge tF i leHandle = $new ;

Mocking a full environment

package Fus ionInventory : : Test : : MockSystem : : Debian ;

use Fus ionInventory : : Test : : MockSystem ;

mockSystem (commands => {

’ dpkg ’ => ’ resources / packaging / dpkg ’} ,f i l e s => {

’ / e tc / debian_vers ion ’ => ’ resources / re lease / debian ’}

$> p e r l −MFusionInventory : : Test : : MockSystem : : Debian f u s i o n i n v e n t o r y

Mocking a full environment

package Fus ionInventory : : Test : : MockSystem : : Debian ;

use Fus ionInventory : : Test : : MockSystem ;

mockSystem (commands => {

’ dpkg ’ => ’ resources / packaging / dpkg ’} ,f i l e s => {

’ / e tc / debian_vers ion ’ => ’ resources / re lease / debian ’}

$> p e r l −MFusionInventory : : Test : : MockSystem : : Debian f u s i o n i n v e n t o r y

Compilation test

Test::More

use Test : : More ;

use_ok ( ’Some : : Module ’ ) ;use_ok ( ’ Another : : Module ’ ) ;

Test::Compile

use Test : : Compile ;

a l l _pm_f i l es_ok ( ) ;

Compilation test

Test::More

use Test : : More ;

use_ok ( ’Some : : Module ’ ) ;use_ok ( ’ Another : : Module ’ ) ;

Test::Compile

a l l _pm_f i l es_ok ( ) ;

Module availability issues

package Some : : Module ;

use Win32 : : T ieReg is t r y ;

Windows

package Another : : Module ;

use Sys : : Syslog ;

Module availability issues

package Some : : Module ;

use Win32 : : T ieReg is t r y ;

Windows

package Another : : Module ;

use Sys : : Syslog ;

Workaround

Filtering

my @fi les = $OSNAME eq ’MSWin32 ’ ?grep { ! / Another / } a l l _ p m _ f i l e s ( ’ l i b ’ ) :grep { ! /Some/ } a l l _ p m _ f i l e s ( ’ l i b ’ ) ;

a l l _pm_f i l es_ok ( @f i les ) ;

Loading modules at runtime

use UNIVERSAL : : require ;

Win32 : : T ieReg is t ry−>require ( ) ;Win32 : : T ieReg is t ry−>import ( ) ;

Workaround

Filtering

my @fi les = $OSNAME eq ’MSWin32 ’ ?grep { ! / Another / } a l l _ p m _ f i l e s ( ’ l i b ’ ) :grep { ! /Some/ } a l l _ p m _ f i l e s ( ’ l i b ’ ) ;

a l l _pm_f i l es_ok ( @f i les ) ;

Loading modules at runtime

use UNIVERSAL : : require ;

Win32 : : T ieReg is t ry−>require ( ) ;Win32 : : T ieReg is t ry−>import ( ) ;

Mocking module, trivial case

No exported symbol

use Win32 : : OLE : : Const ;

Testing code

$INC { ’ Win32 /OLE/ Const .pm ’ } = 1 ;

Mocking module, moderate case

Simple export

use Win32 : : OLE qw( i n CP_UTF8 ) ;

Mock module

package Win32 : : OLE;

use base ’ Expor ter ’ ;

our @EXPORT = qw( i n CP_UTF8 ) ;

Mocking module, complex case

Complex export

use Win32 : : T ieReg is t r y (D e l i m i t e r => ’ / ’ ,ArrayValues => 0 ,qw /KEY_READ/

Mock module

package Win32 : : T ieReg is t r y ;

our $Regis t ry ;

sub import {my $ca l lpkg = c a l l e r ( ) ;no s t r i c t ’ r e f s ’ ;

∗{ " $ca l lpkg \ : : Reg is t ry " } = \ $Regis t ry ;∗{ " $ca l lpkg \ : : KEY_READ" } = sub { } ;

Using mock modules

Testing code

i f ($OSNAME eq ’MSWin32 ’ ) {push @INC, ’ t / fake / un ix ’ ;

} else {push @INC, ’ t / fake / windows ’ ;

Conclusion

Contactswww.fusioninventory.org#fusioninventory on freenodefusioninventory-user@lists.alioth.debian.org

Questions ?

Conclusion

Contactswww.fusioninventory.org#fusioninventory on freenodefusioninventory-user@lists.alioth.debian.org

Questions ?

the fusioninventory projectguillomovitch/presentations/fosdem2012.pdf · glpi php plugin...

Documents

annexe 2 : exploitation du plugin fusioninventory ·...

la rousse foods

fusioninventory -...

lab rousse

fosdem2012 sayer-sems-sbc

fusioninventory openworldforum-paris-2011-september

algorithm forearm rousse

fusioninventory froscamp2010

the fusioninventory project - rmll / lsm...

fosdem2012 replication-features-of-2011

the fusioninventory project - rmll / lsm 2012

beverages - la rousse foods

glpi et fusioninventory, des solutions libres et innovantes

fusioninventory project fosdem 2011

projection spécial tête rousse

glpi et fusioninventory+++

mall rousse brochure

loadays 2011 asset management with fusioninventory and glpi

debian jessie (8.0) + glpi 9.0 + fusioninventory

david durieux goneri le...