hiding legacy software using perl and soap as glue alasdair allan university of exeter, exeter, u.k....

Hiding legacy softwareusing Perl and SOAP as glue Alasdair AllanUniversity of Exeter, Exeter, U.K.

AbstractThe worst nightmare a software developer can face is a major re-engineering of a mature system. We'd all much rather sit down and write new code than carry out software archaeology on a half understood system written by a dozen different people. This is where Perl can come in, wrapping legacy code in XS and SOAP means that you can hide the horror a mature system behind a clean interface. Perl makes an excellent glue language, and implementing new systems on top of your legacy code no longer means that you have to know the exotic internals of the legacy system. By using web services as building blocks new functionality can be written quickly and efficiently. Old code never dies, it just gets hidden.

Saturday 11th Dec. 2004 London Perl Workshop

2

Encapsulating knowledge

• Your code base is the fossil record of your organisation

• The worst mistake you can make is to try and re-implement your existing code base from scratch


3

Throwing code away…

• A very painful example from my own experience is “Chunking”

• This wasn’t even because we thought we could do it better, we simply thought it wasn’t relevant to the modern world

• We should have modified, not discarded


4

Second system effect

• Re-implementing things can be a lot more costly than you would expect

Second System Effect n. (sometimes, more euphoniously, `second-system syndrome') When one is designing the successor to a relatively small, elegant, and successful system, there is a tendency to become grandiose in one's success and design an elephantine feature-laden monstrosity.


5

Software archaeology

• Software naturally forms layers

• Underneath the convenience layer you’ll normally find a more powerful layer which is usually much harder to understand


6

Code reuse

There are two main approaches,

• Wrapping code• Software as services

Both you and the code will benefit from,

• Refactoring


7

Refactoring

Why should you refactor?

• To fix architectural problems

• To remove inefficiency

Just plain ugly is not a good reason.


8

Just plain ugly

• Code is often ugly for a reason

• Every bug fix makes the code harder to understand and much uglier

• Refactoring just to make the code pretty could remove added goodness


9

Wrapping code

• The traditional way to hide legacy code is to wrap it up behind a convenience layer

• In Perl this is usually done using XS and lately the Inline::* modules


10

Refactoring by stages

• Once you wrap components you can remove them in a piece meal fashion

• This is especially true if you use the “software as services” paradigm


11

The Book…

Extending and Embedding PerlTim Jenness & Simon CozensManning, ISBN 1930110820Costs £21.17 at Amazon.co.uk

“…the canonical book for this type of programming” -- Alasdair Allan


12

Perl XS

• It looks like a couple of talks on the Advanced track will be discussion XS in depth, so I’m not going to bother

• Thank goodness for that…


13

Inline Module

• Inline lets you write Perl subroutines in other programming languages like C, C++, Java, Python, Tcl and even Assembly.

• You don't need to compile anything. All the details are handled transparently so you can just run your Perl program like normal.


14

What’s going on?

1. Your Perl module reads the code from the appropriate place usually below the DATA handle

2. An MD5 checksum is calculated for the code in question

3. This checksum and other information are compared with the XS modules previously generated by Inline.


15

What’s going on?

4. If the checksum does not match, an XS module is generated based on the functions and arguments in the inlined code.

5. The module is built and and installed into a local directory

6. If the checksum matched, the relevant module is loaded


16

Inline::C

use Inline C; hello_world(’Your Name Here');

__END__ __C__ void hello_world(char* name) { printf("Hello %s!\n", name);}


17

Inline::Python

use Inline Python;print "9 + 16 = ", add(9, 16), "\n";

__END__ __Python__

def add(x,y): return x + y


18

Inline::Python and Objects

use Inline Python;my $obj = new Myclass();

__END__ __Python__

from mylibrary import myclass as Myclass


19

Inline::Javause Inline Java;my $obj = new Example( ‘some data’ );print $obj->get_data() . "\n”;

__END__ __Java__

public class Example { private String data = null;

public Example( String s){ data = s; }

public String get_data(){ return data ; }}


20

Inline::Java and STUDY

use Inline ( Java => 'STUDY', STUDY => ['java.util.HashMap'] );

my $hm = new java::util::HashMap(); $hm->put("key", "value"); my $val = $hm->get("key");

print $val . "\n";


21

Inline::*

• Inline::C

• Inline::Java

• Inline::Python

• Inline::Tcl

• and others…


22

Software as services

• A different approach to hiding legacy code is the “software as services” paradigm

• Add yet another layer on top of the wrapped legacy code so that the interface to the outside world becomes language neutral

• Perl is good at this…


23

SOAP::Lite

See www.soaplite.com for details

The latest release is V0.65 Beta 2, although this is not yet on CPAN. Adds, amongst otherthings, • MIME and DIME support


24

Google Web Services

See www.google.com/apis/,use SOAP::Lite;

my $key = "000000000000000000000000";my $wsdl = "http://api.google.com/GoogleSearch.wsdl";my $query = "foo";

my $google = SOAP::Lite->service( $wsdl );my $result = $google->doGoogleSearch( $key, $query, 0, 10, "false", "", "false", "", "latin1", "latin1");

my $results = $result->{'estimatedTotalResultsCount’};print “About $results returned\n”;


25

The Book…

Programming Web Services with PerlRandy Ray & Pavel KulchenkoO’Reilly, ISBN 0596002068

Costs £19.95 at Amazon.co.uk

Now slightly out of date, but still a good source

of information about the SOAP::Lite modules


26

The Mailing Lists…

See www.soaplite.com for details,

• Main list is on Yahoo! Groups, see groups.yahoo.com/group/soaplite/

• But there is also soaplite-announce and soaplite-devel lists hosted at SourceForge


27

Amazon Web Services

• A good example of the software as services paradigm


28

ItemSearch Service

sub make_request { my $keywords = shift; my $request_type = \SOAP::Data->value( SOAP::Data->name('Keywords')->value($keywords), SOAP::Data->name('SearchIndex')->value('Books'));

my $itemsearch_request = SOAP::Data->value( SOAP::Data->name('SubscriptionId’) ->value($subs_id), SOAP::Data->name('Request')->value($request_type));

my $aws_handle = SOAP::Lite->service("$aws_wsdl"); $aws_handle->ItemSearch($itemsearch_request);

my $som = $aws_handle->call(); return $som;}


29

Building our Complex Type

my $request_type = \SOAP::Data->value( SOAP::Data->name('Keywords')->value($keywords), SOAP::Data->name('SearchIndex')->value('Books'));

my $itemsearch_request = SOAP::Data->value( SOAP::Data->name('SubscriptionId’) ->value($subs_id), SOAP::Data->name('Request')->value($request_type));

Builds the following complex type

<SubscriptionId>$subs_id</SubscriptionId><Request> <Keywords>$keywords</Keywords> <SearchIndex>Books</SearchIndex></Request>


30

Calling the Service

my $aws_handle = SOAP::Lite->service("$aws_wsdl");$aws_handle->ItemSearch($itemsearch_request);

my $som = $aws_handle->call(); return $som;

Creates a SOAP object calls the Amazonservice using the WSDL found at the URL

Returns a SOAP::SOM object containing the results of the query


31

A simple SOAP Server

The simplest way is to use an existing web server,the following would end up as cgi-bin/service.cgi

#!/usr/bin/perl

use SOAP::Lite;use SOAP::Transport::HTTP;

use ANY_OTHER_MODULE_NEEDED;

SOAP::Transport::HTTP::CGI ->dispatch_to(‘/path/to/perl/modules/’) ->handle;


32

dispatch_to( )

A collection of Perl modules,

dastardly{aa}: ls modules/drwxr-xr-x 2 nobody users 4.0K Sep 16 2003 ./drwxr-xr-x 6 root root 4.0K Oct 11 21:56 ../-rw-rw-r-- 1 nobody users 118 Sep 16 2003 Echo.pm-rw-rw-r-- 1 nobody users 85 Sep 16 2003 Ping.pm

package Ping;

sub ping { $class = shift; return “ACK”;}


33

Easily consumed…

You don’t need to write WSDL to use the service,

use SOAP::Lite;

my $soap = new SOAP::Lite();$soap->uri(‘http://www.company.com/Ping/);$soap->proxy(‘http://www.company.com/cgi-bin/service.cgi’);

my $result;eval { $result = $soap->ping(); };if ( $@ ) { print $result->faultstring(); exit;}

print $result->result();


34

Asynchronous web services

• SOAP::Lite does not support .NET asynchronous callbacks

• But you can implement asynchronous web services using the module

• Use contextual web services and persistent state to keep track of things by hand

• This isn’t as hard as it sounds…


35

Authentication

• See WebService::TicketAuth for a full blown solution

• Alternatively you could do a light weight implement of authentication using HTTP cookies by sub-classing the relevant SOAP::Transport module.

• Look at www.astro.ex.ac.uk/people/aa/ for an implementation of the later


36

No more language neutrality

• If you can serialise it you can send it over a wire. Data::Dumper can be a powerful tool

my $dumper = new Data::Dumper([$object], [qw($object)]); my $serialised = $dumper->Dump();\

and at the far end,

my $object = eval $serialised; if ( $@ ) { print “Warning: Cannot de-serialise object\n”; }


37

SOAP::Lite has problems

• Interoperability - Although have a look at the SOAP::WSDL module if you need .NET interoperability

• Complex types - You’ll find that the SOAP::Data::Builder should simplify things

• WSDL - Perl is loosely typed, this means no automatic WSDL generation


38

So why did I talk about Inline?

• Because of the lack of “proper” WSDL support I’m currently using a mixture of Java and Perl services to hide legacy Fortran (and C)

• Perl serves as an excellent glue language for this task


39

Grid Services?

See www.sve.man.ac.uk/Research/AtoZ/ILCT

WSRF::Lite is the follow on work from OGSI::Lite,

it implements the Web Service Resource

Framework which has effectively replaced OGSI,

for details on WSRF visit www.globus.org/wsrf


40

Lots of code, broken compiler?

If you have a large mature code base but the

limits of the underlying infrastructure are starting

to cause problems, then perhaps you should

• Re-implement the compiler, not your code• No, seriously…


41

Re-implement the system?

The alternative is to rewrite the entire system in a new language but,

• Experience shows that such projects often (almost always?) fail

• You loose years (decades?) of customised business logic

• You make a lot of people very unhappy


42

Use Parrot?

• Implementing functional compilers for 4GL languages on top of Parrot has been done before (err, once)

• See “Building a Parrot Compiler” by Dan Sugalski at the O’Reilly’s OnLamp.com

• For simpler languages, or problems, you could target the compiler for Perl 5 as Dan did initially


43

Problems with Legacy Code

• Layering - the higher level routines will believe the output of the lower level ones.

• Bugs - You still have all the old bugs, although on the bright side you, you don’t have any new ones.

• Backdoors - The possibility of backdoors in the lower level code.


44

Conclusions

• Perl is an excellent glue language

• You can use it to hold together the unlikeliest collection of different pieces of code

• It may be ugly but this is better than loosing decades of “smartness”

• Your code has evolved, don’t lose that

hiding legacy software using perl and soap as glue alasdair allan university of exeter, exeter, u.k....

Documents