integration of portals and web services

04/19/2304/19/23 uri="http://grids.ucs.indiana.edu/ptliupaguri="http://grids.ucs.indiana.edu/ptliupages/presentations/ncsaexpeditionmar02" ees/presentations/ncsaexpeditionmar02" email="[email protected]"mail="[email protected]"

11

Integration of Portals and Web Services

PTLIU Laboratory for Community GridsGeoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn

Computer Science, Informatics, Physics

Indiana University, Bloomington IN 47404http://grids.ucs.indiana.edu/ptliupages/presentations/ncsaexpeditionmar02

[email protected]

NCSA Expedition Meeting March 15 2002

04/19/2304/19/23 uri="http://grids.ucs.indiana.edu/ptliupages/presentations/ncsaexpeditionmar02" email="[email protected]"uri="http://grids.ucs.indiana.edu/ptliupages/presentations/ncsaexpeditionmar02" email="[email protected]" 22

What is a Web Service I A web service is a computer program running on either the local

or remote machine with a set of well defined interfaces (ports) specified in XML (WSDL)

In principle, computer program can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be implemented in any way what so ever• Interfaces can be method calls, Java RMI Messages, CGI Web

invocations, totally compiled away (inlining) but

The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python

Web Services separate the meaning of a port (message) interface from its implementation

Enhances/Enables Re-usable component model of ANY electronic resource


What is a Web Service II Web Services have important implication that ALL

interfaces are XML messages based. In contrast Most Windows programs have interfaces defined as

interrupts due to user inputs Most software have interfaces defined as methods which

might be implemented as a message but this is often NOT explicit

Security Catalog

PaymentCredit Card

WarehouseshippingWSDL interfaces

WSDL interfaces


What is a Web Service III “Everything electronic” is a resource

• Computers; Programs; People• Data (from sensors to this presentation to email to

databases) “Everything electronic” is a distributed object All resources have interfaces which are defined in XML for

both properties (data-structure) and methods (service, function, subroutine) (Resources are Services)• We can assume that a data-structure property has

getproperty() and setproperty(value) methods to act as interface

All resources are linked by messages with structure, which must be specifiable in XML

All resources have a URI such as unique://a/b/c …….


Classic Grid Architecture

Database Database

Netsolve

Computing

SecurityCollaboration

CompositionContent Access

Resources

Clients Users and Devices

Middle TierBrokers Service Providers

Middle Tier becomes Web Services


etc.XML WS to WS Interfaces

(Virtual) XML Knowledge (User) Interface

Clients

(Virtual) XML Data Interface

Raw DataRawResources

Raw Data

WSWS

Web Service (WS)

WS

WSWS WS WSWS

Render to XML Display Format

(Virtual) XML Rendering Interface


WSDL Abstractions WSDL abstracts a program as an entity that does

something given one or more inputs with its results defined by streams on one or more outputs.

Functions are defined by method name and parametersmethodname(parm1,parm2, … parmN)• Where parameters are “Input” “Output” or both

In WSDL, we will have a Web Service which like a (Java or CORBA Program) can be thought of as a (distributed) object with many methods• Instead of a function call, the “calling routine” sends an

XML message to the Web Service specifying methodname and values of the parameters

• Note name of function is just another parameter


Details of WSDL Protocol Stack UDDI finds where programs are

• remote( (distributed) programs are just Web Services

WSFL links programs together(under revision?)

WSDL defines interface (methods, parameters, data formats)

SOAP defines structure of message including serialization of information

HTTP is negotiation/transport protocol

TCP/IP is layers 3-4 of OSI Physical Network is layer 1 of OSI

UDDI or WSILUDDI or WSIL

WSFLWSFL

WSDLWSDL

SOAP or RMISOAP or RMI

HTTP or SMTP or IIOP or

RMTP

HTTP or SMTP or IIOP or

RMTP

TCP/IPTCP/IP

Physical Network

Physical Network


Peer to Peer Grid

DatabaseDatabase

JXTA

JXTA

Peer to Peer Grid

Web Service Interfaces

Web Service Interfaces

Event/MessageBrokers

Integrate P2Pand Grid/WS

Event/MessageBrokers


Examples of Web Services I OGSA (Open Grid Service Architecture)

• Integrate Web Service and Grid Concepts and allows Globus to be implemented as Web Services

Audio-Video Conferencing as a Web Service• Integrates H323, SIP, JXTA (etc.) protocols by mapping to

single XML Interface

• Provides VRVS reflector model from Messaging Web Service Messaging or Event Web Service provides intelligent

routing and buffering of messages Performance as Web Service (yesterday)

• Allow access to distributed monitoring data, analysis, models, and final benchmarks with interoperable XML interfaces


Examples of Web Services II Computing as a Web service

• More on this today – job submittal, status, composition, data services, visualization etc.

Education as a Web Service• One of easiest to do as object standards well defined (IMS) and little

performance issues

• Grading, Homework submission, registration, assessment etc.

Universal Access and Web Services• As Web Services allow multiple implementation of a particular interface,

one can adjust to needs of particular clients (PDA v. versus, impaired sight etc.)

• Can build custom implementations of certain web services for particular communities but re-use others

Collaborative Web Services• As interfaces all message based, relatively easy to share Web Services


Portals and Web Services Web Services allow us to build a component model (see

CCA) for resources. Each resource naturally has a user interface (which

might be customized for user) Web Service <--> Portlet Natural to use a component model for portal building

displayed web page from collection of portlets• So can customize each portlet and customize which portlets

you want Apache Jetspeed seems good open source technology

supporting this model • JSP model is better than say a client-side Java integration in

that also message based so this is “Portal as a Web Service”


Component –based Portal

Web Services provide a component model for the middleware (see large “common component architecture” effort in Dept. of Energy)

Should match each WSDL component with a corresponding user interface component

Thus one “must use” a component model for the portal with again an XML specification (portalML) of portal component


Meta-Data and Web Services Enriching resources with meta-data is critical idea

• Enables one to identify and link resources around the globe

• Allows one to find out “meaning” of a Web service not just syntax of interface

Semantic Grid implies linkage of Grid/Web services enabled by meta-data leading to “digital brilliance” phase transition

We can experiment with Semantic Web techniques for specifying meta-data RDF DAML OIL

These encompass both straightforward enriched data as well as Artificial Intelligence assertions


RDF in a Nutshell

The essence of RDF is a statement which has a subject and predicate and an object

The “subject” has the property “predicate” with value “object” The Web Page has the title “RDF Model and Syntax” Mary had a husband called John Statements can specify metadata but more generally any facts are

expressible in this way One can associate multiple objects with a subject and so build up

a general node of a tree with parent as subject and objects as children.

DAML and OIL extend RDF with more powerful relations and ontologies

Subject ObjectPredicate


An Example of RDF and Dublin Core <rdf:RDF xmlns:rdf=“http://www.w3.org/1999/02/22-rdf-syntax-

ns#” xmlns:dc=“http://purl.org/metadata/dublin_core#”> <rdf:Description about=“http://www.dlib.org”> <dc:Title>D-Lib Program - Research in Digital

Libraries</dc:Title><dc:Description>The D-Lib program supports the community of people with research interests in digital libraries and electronic publishing. </dc:Description><dc:Publisher>Corporation For National Research Initiatives</dc:Publisher><dc:Date>1995-01-07</dc:Date>

<dc:Subject>• <rdf:Bag> <rdf:li> Research; statistical methods</rdf:li>

<rdf:li> Education, research, related topics</rdf:li><rdf:li> Library use Studies </rdf:li> </rdf:Bag>

</dc:Subject> <dc:Type>World Wide Web Home Page</dc:Type> <dc:Format>text/html</dc:Format> <dc:Language>en</dc:Language> </rdf:Description> </rdf:RDF>


What could one do? Get applications together and decide on set of common

web services• Service and methods within services• At “Grid Computing” and “Application” level

Implement generalized meta-data service at application level – generate, access• Role of RDF DAML OIL IMS etc.

Define both middle tier (WS) and user interface (portal) components

Agree on pilot projects demonstrating both interoperable WS/Portal components and complete applications built from WS/Portal components

Feed lessons into W3C and Grid processes


Collaborative Web Services First note that there are two distinct concepts Collaboration as a Web Service

• Such as “Audio-Video Conferencing” as a Web Service or “Text Chat as a Web Service”

Collaborative Web Services• Here we view a Web Service as specifying a

(distributed) object and wish to share an object• Object could be a Web page, a Job status form, a

scientific visualization, a PowerPoint slide etc. (not all of these are Web Services but all should be)


Why Web Services for Collaboration Well everything is meant to be a Web Service but also: Web Services are MUCH EASIER to make

collaborative than other objects because all input and output is defined by uniform XML messages• You need to teach your message service about collaboration!

• Note local applications are NOT Web Services – input is things like “user mouse click” represented by “method events” (UI program interrupts) not “XML message events”

The elegance of collaborative web services suggest that it could be easiest to make object X (such as PowerPoint or SVG) collaborative not by traditional direct methods but by converting to a Web Service


CollabWS: Collaboration as a Web Service

This includes Audio-Video conferencing, Instant Messengers, shared applications and tools

There are basic standards H323 and SIP There are peer to peer collaboration approaches like

JXTA from Sun There are various commercial products – Groove

Networks effectively defines some shared application interfaces

CollabWS defines a superset of collaboration properties and subservices in XML/WSDL

Then you provide several bindings of collabws.wsdl to new transports and old ones (binary H323)


H323 SIP

JXTA


SIPH323 JXTA

H323 JXTA

SIP


A Typical SIP Message REGISTER sip:registrar.biloxi.com

Via: SIP/2.0/UDP 10.4.1.4:5060To: Bob (sip:[email protected])From: Bob (sip:[email protected]);tag=456248Call-ID: [email protected]: 1826 REGISTERContact: (sip:[email protected])Expires: 7200Contact-Length: 0

Initially build a wrapper that accepts such messages and converts to …..


Collaboration as a Web service - now


SIP as XML <sipasxml>

<register> sip:registrar.biloxi.com</register><via> SIP/2.0/UDP 10.4.1.4:5060</via><to> Bob (sip:[email protected]) </to><from tag="456248" > Bob (sip:[email protected]) </from><call-id> [email protected]</call-id><cseq> 1826 REGISTER </cseq><contact> (sip:[email protected]) </contact><expires> 7200 </expires><contact-length> 0 </contact-length></sipasxml>

So this is either an internal format to which we map SIP messages or it is an external format used for Collaboration Web Service


Future Collaboration Web Service


Messaging/Events as a Web Service We can implement messaging subsystem (between

WSDL resources) with either direct messages or by a queued system where you publish messages to queues and subscribe as receiver to particular queues

There are many different publish/subscribe models• JMS is a cluster of central servers• JXTA is a very dynamic Peer to Peer model where pipes are

queues and topics (metadata) are service advertisements Implement JMS API with JXTA protocol – different

WSDL bindings here have different fault tolerance/reliability semantics• Could use JMS as long distance “carrier” between JXTA peers• JXTA provides higher performance than JMS for nearby

recipients


Event Web Service

Filter is mapping to PDA or slow communication channel (universal access) – see our PDA adaptor

Workflow natural as all messages “intercepted” by Event Web Service

Routing illustrated by JXTA Destination-Source matching illustrated by JMS

Web Service 1

(Virtual)Queue

Web Service 2

Destination Source Matching FilterRouting workflow

WSDLPorts

WSDLPorts


Broker Network

Database

Resource

Broker

Broker

Broker

Broker

Broker

Broker

Software multicastP2P: Brokers are clients


Narada JXTA/JMS Integration http://grids.ucs.indiana.edu/ptliupages/projects/narada/ Is a network of event brokers which can reliably deliver XML

specified events Using openJMS selection module, becomes a distributed or

conventional Java Message Service Linking special JXTA

proxies, it can link JXTA communities

Think of JXTA JMS andNarada as differentbindings toevent/messagingweb service


Education as a Web Service Can link to Science as a Web Service and substitute educational

modules “Learning Object” XML standards already exist from IMS/ADL

http://www.adlnet.org – need to update architecture Web Services for virtual university include: Registration Performance (grading) Authoring of Curriculum Online laboratories for real and virtual instruments Homework submission Quizzes of various types (multiple choice, random parameters) Assessment data access and analysis Synchronous Delivery of Curricula Scheduling of courses and mentoring sessions Asynchronous access, data-mining and knowledge discovery Learning Plan agents to guide students and teachers


CMS (Content Management) as a Web Service

For meta-data, build federated database as a Web Service• Storage, update, lookup of objects

• Registration of different control engines For content, if W3C DOM and Java API as for SVG

and “decent” XHTML browsers, then can wrap DOM as a web service for all interface functions• Can of course do this for IE5 but document object model

unreliable Build collaboration systems as sharing Web Service

interfaces (not COM or other object API as proposed by Groove Networks)


Integration of XML/Web Services into an Enterprise I

You are the CIO of a little known state’s (call it Xanadu) education system and been tasked to position it as a 2005 leader in technology enhanced education

Your initial review reveals a multitude of incompatible existing information systems distributed over Xanadu’s 50 campuses

Your cradle to grave community education mandate (which got the Governor elected) implies integration of all aspects of education in your information system

You note that performance issues are much less important in this application than interoperability (it currently takes months to get anything done so what's an extra millisecond or so at a WSDL interface) , so even in current primitive/immature state Web services look attractive


Enterprise Web Services II We start a requirements analysis and a catalog of existing “raw

resources”. This will lead to the clean internal XML(WSDL) interfaces XIF and to the external XML interfaces (XEF) to the raw resources

Xanadu Internal Framework (XIF) built on top of XML Messaging, Security etc. Services

We find that The Grid Forum community has defined computing interfaces which we will adopt to support infrastructure needed for use of simulations and data repositories in curriculum

We find that Microsoft, the Liberty Alliance and the Grid have Security schemes but are relieved that with a little bit of XSLT (transformations) we can map between them and can define authentication & access control as a Web Service

We join the IMS consortium and adopt their standards for learning meta-data and user related information

We adopt the AICC and ADL SCORM standards for “learning objects” (course pages)


Enterprise Web Services III We define Xanadu educational agents which use XIF plus

RDF/DAML-OIL to be able to capture educational goals We expect commercial standards compliant solutions will be

available for most other needed Educational web services We find that we must interface with a dreadful collection of

external raw resources coming from legacy PeopleSoft SAP Oracle and Microsoft products• As our new model is object (XML) and open standards based we build a

separate XII (Xanadu Information Infrastructure) supporting XIF objects We analyze structure of legacy systems and their relation to XIF

and define the external framework XEF which captures much of the raw resources• This XEF Schema is extensible to capture quirks of each legacy system• We build a set of Web services that convert XEF into XIF

We inform vendors that we will only install XIF compliant systems in all future purchases


WSDL from a Gateway View


7 Primitives in WSDL types: which provides data type definitions used to describe the

messages exchanged. message: which represents an abstract definition of the data being

transmitted. A message consists of logical parts, each of which is associated with a definition within some type system.

operation– an abstract description of an action supported by the service.

portType: which is a set of abstract operations. Each operation refers to an input message and output messages.

binding: which specifies concrete protocol and data format specifications for the operations and messages defined by a particular portType.

port: which specifies an address for a binding, thus defining a single communication endpoint.

service: which is used to aggregate a set of related ports


Discussion of 7 WSDL Primitives types specify data-structures which are equivalent to arguments of methods message specifies collections of types and is equivalent to set of

arguments in a method call. Note that it is an “abstract method” in Java terminology

operation is a a collection of input output and fault messages; there are 4 types of operation one-way(service just receives a message), request-response(RPC), solicit-response, notification (services pushes out a message)

portType represents a single channel that can support multiple operations. It is “abstract” as specified as a set of operations. It is equivalent to a “interface or abstract class” in Java

binding tells you transport and message format for a porttype (which can have multiple bindings to reflect say performance-portability trades)

port combines a binding and an endpoint network address (URL) and is like a “class instance”

service consists of multiple ports and is equivalent to a “program” in Java


Gateway and Web Services We will use the Gateway Computing Portal as an

example (http://www.gatewayportal.org)• It is largely built using CORBA with a Java Server Pages

front end• http://community.ucs.indiana.edu:8004/GCWS/BatchScriptGen/Main.jsp

Several capabilities have been interfaced using WSDL• Job Submission (11 Methods including execute local and

remote command, copy files etc. as well as Submit Job)• Manage WebFlow Session (67 Methods)• Generate Batch Script (just 1 method but two

implementations developed – one at SDSC and one at Indiana – with UDDI to manage)

• Each is one service – could have used finer grain services• Sample files are at

http://grids.ucs.indiana.edu/ptliupages/presentations/ggf4feb02


Gateway Architecture As needed DoD approved Kerberos/SecureID security,

does not usually link to standard Grid technology (Globus)

backend

Globusor submit to localqueue system

Corba middle tierSECIOP withJava modules

JSP SSL

backend

SOAP wrapperJSP Servlet

CORBA

WSDL

Will replacehistorical CORBABy EJB

Add CastorJava to XML

Pre WSDL Post WSDL


SOAP and Gateway Portal I Having specified service in WSDL, the run-time is implemented in SOAP which is “just” an XML header (info

needed by transport – empty here) and body Here is SOAP transported by HTTP message This is execLocalCommand WSDL operation to run one particular command (ls) on current WebFlow directory

Specify ls as

Argument of operation

HTTP Header

SOAP Envelope and body


And this is the result of ls sent back to client in SOAP over HTTP

SOAP and Gateway Portal II

HTTPHeader

SOAPEnvelopeand body


WSDL Message Example<message name="submitRequest"> <part name="xmljob" type="xsd:string"/></message><message name="submitResponse"> <part name="response" type="xsd:string"/></message>

For the batch script service, we pass the XML description of the job as a string and get back the script as a string.In general, any XML primitive or complex types can be used in messages.

We could improve our service by defining a BatchScript complex type.


WSDL portTypes Example <portType name="BatchScriptServicePortType"> <operation name="batchGen"> <output message="tns:submitResponse"

name="submitResponse"/> <input message="tns:submitRequest"

name="submitRequest"/> </operation> </portType>

A portType corresponds to a Java class, so if we compile this WSDL to make client stubs, we will generate a BatchScriptServiceBinding.java class.


WSDL SOAP Binding Example<binding name="BatchBinding" type="tns:BatchScriptServicePortType"> <soap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/> <operation name="batchGen"> <soap:operation soapAction=""/> <input> <soap:body use="encoded“ namespace="urn:BatchScriptService" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/> </input> <output> <soap:body use="encoded" namespace="urn:BatchScriptService“

encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/> </output> </operation></binding>

note binding’s “type” attribute points back

to the portType tag by name


WSDL Ports and Services<service name="BatchScriptService"> <documentation>BS stands for Batch Script </documentation> <port binding="BatchBinding”

name="BatchPort"> <soap:address location=

"http://yourserver/soap/servlet/rpcrouter/"/> </port> </service></definitions>

ports are concrete implementations of portTypes and point back to a particular binding (by name). They also point to the specific location of a server that implements the service.A service is a collection of one or more ports.


What is a Virtual XML Interface We can specify interfaces in XML but we are not required to

implement in XML. Example 1:We aren’t likely to change syntax of mail

Reply-to:Geoffrey Fox <[email protected]>To: Geoffrey Fox <[email protected]>Subject: A Test for Tutorial

A simple mail messageGeoffrey Fox [email protected] FAX 8128567972Phones Cell 315-254-6387 Home 812323919

But we could specify and indeed store in XML with transport done using conventional SMTP.

So conventional mail is easy to give a virtual XML specification for withname:value becoming<name>value</name> but this is not only way


Mail in XML <mailasxml uri=“gxos://mail/users/gcf/sent/2002/february/290” >

<smtpheaders><reply-to email=“[email protected]” >Geoffrey Fox</reply-to><to email=“[email protected]” >Geoffrey Fox</to> <subject>A Test for Tutorial</subject>

</smtpheaders><smtpbody> <message whitespace="collapse"> A simple mail message</message> <signature personuri=“ssn://123/45/6789” whitespace=“preserve” > Geoffrey Fox [email protected] FAX 8128567972 Phones Cell 315-254-6387 Home 812323919 </signature> </smtpbody>

</mailasxml> Such an interface could be used by “User Messaging as a Web service” which

could perhaps integrate News Groups, e-mail, text chat, instant messenger, voicemail


Virtual XML for MPI I In days gone by, we used to study:

MPI_SEND(buf, count, datatype, dest, tag, comm) [ IN buf] initial address of send buffer (choice) [ IN count] number of elements in send buffer (nonnegative integer) [ IN datatype] datatype of each send buffer element (handle) [ IN dest] rank of destination (integer) [ IN tag] message tag (integer) [ IN comm] communicator (handle)

Now our SPMD code is a Web service running in some large scale parallel machine – perhaps integrated with an Apache server running on node• Need to support distributed and shared memory

We want to define all interfaces in WSDL – at least all of them which could be “external” ports


Virtual XML for MPI II datatype becomes an XML Schema specifying structure of

transmitted data – could either be a URI for Schema or its realization• There would be a built-in schema suitable for handling standard

Fortran/C datatypes

buf is a handle to an XML instance of Schema datatype count is still an integer (xs:integer where xs is Schema

namespace xmlns:xs=http://www.w3.org/2001/XMLSchema)• For some bindings count is implied by buf

dest becomes a URI with special notation such as legacympi://procnum/local/nnn for processor numbers

tag becomes an instance of (new) mpitag Schema comm becomes an instance of (new) mpicommunicator Schema


Virtual XML for MPI III So one defines mpi.wsdl with all 100 or so methods

defined as (abstract) operations• We will define several bindings including the classic HPCC

method calls for Fortran C C++ and Java with either distributed memory or shared memory mode

• For C++ and Java we would implements bindings for either “object” or “packed consecutive byte” interpretation of buf

Note one can use both static and run-time compilation to map XML specified services into particular implementations

MPI_INIT() might be implemented with JXTA (peer-to-peer advertisments) to find nodes; the communicator might be transmitted by SOAP while buf transmission uses MPICH

This brings Web Services to MPI but also MPI to web services

integration of portals and web services

Documents