SOA for RPG ProgrammersWhat does it really mean to ME!?

by

Aaron BartellKrengel Tech

(creators of RPG-XML Suite, www.rpg-xml.com)

aaron@MowYourLawn.com

Agenda• First Things First – Defining SOA• Why SOA? Why now, and how?

• SOA real world scenarios

• When to use services and when not to

• Best Practices

• SOA in action – Java front-end to RPG back-end

Defining SOASOA = Service Oriented Architecture

– In strictly RPG terms "Service Oriented" is the practice of creating RPG programs with a separation of concern

• Break down programming to distinct features organized by function, for example, order or customer related business logic should be placed in it's own RPG *SRVPGM separate from other types of business logic.

– SOA has a lot to do with modular programming practices (i.e. RPG *SRVPGMs and *MODULEs)‏. SOA starts with modernizing your RPG by using service programs and sub procedures!!

– SOA can be implemented within a machine (RPG to RPG program communication) or outside of a machine (inbound or outbound program calls).

Defining SOA contd...

Wikipedia excerpt (with my comments in parentheses) - "The key to SOA is independent services (i.e. RPG programs or sub procedures) with defined interfaces (PLISTs and prototypes) that can be called to perform their tasks in a standard way (conform to RPG data types), without the service having foreknowledge of the calling application (develop more on the generic side to account for future use), and without the application having or needing knowledge of how the service actually performs its tasks.(make sure to understand what type of

stateful-ness you need to account for)"

Defining SOA contd...You don't know how your business logic will be used in the future,

but you can safeguard yourself by building modular.A challenge we currently face is the entire tech industry not being

sure of a solid business minded UI technology. Right now Adobe Flex, Sun JavaFX, a variety of HTML/Javascript frameworks and Microsoft Silverlight are all trying to create a solid user interface with features similar to what we have had in 5250 for years.

Never before has there been a better time to “build for change”.

Why SOA? Why now? And How?• Why?

– Over the years our IT shops have had varied needs to meet customers demands. That has required many different software applications to be written/installed on varied platforms and languages based on a packages requirements and/or capabilities (i.e. Language, OS, and DB dependencies).

– Spanning the gap from one machine to the next is the new ROI to save time, money and offer expedient service to end customers. The fact that you HAVE technology is no longer important - it is instead how well you utilize it.

• How– Best of breed technologies and methodologies are being created to facilitate cross

platform/language communication. • Some of the web service technologies still have kinks being worked out and

will be modified as usage increases.• Perception of how to use web service technologies vary greatly. Organization

http://ws-i.org is addressing this issue.• XML, XSD, HTTP, SOAP

– Tried and true technologies are being re-purposed and determined how they should fit into the SOA big picture.

• FTP, SQL Stored Procedures, EDI, SMTP, MQSeries, Data Queues, etc.

IF soa=xml_web_services GOTO class_and_learn_more

XML based web services popularized SOA but real SOA already existed in all shops. If you have one program calling another then you have already implemented SOA. Nearly everything that has been built on your machine by your company is a service – some just haven't been built with modularity and extensibility in mind because it hasn't been as important until now.

SOA involves MANY technologies including FTP, SQL Stored Procedures, EDI, etc, and is not just limited to XML based web services.

Save Yourself From SOA- Many times SOA is a band-aid for inconsistent IT architecture. - Instead of .NET front-ends look for native extensions to current investment. - Eliminates failure points by keeping things on a single machine. - Eliminate the need to do data synchronization. - Create huge efficiencies by employing a single server-side business language (RPG).

- When you control both ends of the spectrum (a.k.a black box application) then it behoves you to pick the technologies that will suit the best for the long term (i.e. how easy is it to maintain, do new framework releases cause re-writes, how easy is it to debug, what is the community support like, etc)‏

Log To Save Your HairWhen you start developing varied levels of a Service Oriented

Architecture it becomes ever so important to have good logging in place so you can quickly find bugs, issues, and rule your processes out as the culprit.

Create a single DB2 table to act as a central location for logs that can be easily queried and indexed. Make sure this log file is generic in nature so it can be used for many purposes. “Extensions” to the log table can be added through “association” DB2 tables.

Log To Save Your Hair contd...Example of a generic log DB2 table definition A UNIQUE

A R LOGR TEXT('Generic Log')‏ A PID 15P 0 COLHDG('Parent Id')‏

A UID 15P 0 COLHDG('Unique Id')‏

A CRTDT Z COLHDG('Record created')‏ A CD 10A COLHDG('Code')‏

A PGMNAM 30A COLHDG('Program Name')‏ A SVRTY 10 0 COLHDG('Severity')‏

A TXT 256A COLHDG('Text')‏

A K PID A K UID

Example of an “association” DB2 table (there would be many of these to associate the different systems to a single log table) ‏

A UNIQUE

A R LOGORDASCR TEXT('Order Assoc')‏ A PID 15P 0 COLHDG('Parent Id')‏

A ORDNBR 9A COLHDG('Order Number')‏

A K PID A K HTEORDNBR

Example SQL statement to associate the two tablesSELECT * FROM LOG as L

LEFT JOIN LOGORDASC as LOA on LOA.PID = L.PIDWHERE LOA.ORDNBR = '1121452'

RPG Modularization ExampleExample of an order program having written out date logic in it's code vs. separating it out into a *SRVPGM.

• Problems• Code is repeated many times in same program.• Takes many lines of code to accomplish a simple task making the program considerably longer.

Hiding Implementation DetailsBelow is an RPG sub procedure named IFS_cmpSame that determines if the contents of pFile1 and pFile2 are exactly the same.

Here is the IFS_cmpSame being used in another program. Note that this program has no knowledge that Qshell API's are actually being used behind the scenes, nor does it care. It is only concerned that it can pass in two files and get a boolean value returned stating whether or not they are the same.

Hiding implementation details and making services “easy to use” will foster adoption. Use the 80/20 rule and develop for the 80, then let the 20 do a work around. Don't try to boil the ocean by thinking you have to develop a piece of code that will last forever.

Develop For Future UseIt is important to look beyond today's need of a service and theorize how to make the service more generic for future use. Many times this can only come with practice (i.e. trial and error).

Before thinking about future use:

// tmstmpStr now has '2009-02-01-12.08.35.316470'tmstmpStr = timestampToChar( %timestamp() );

After thinking about future use:

// tmstmpStr now has '2009-02-01T12:08:35'tmstmpStr = timestampToChar('yyyy-MM-ddThh:mm:ss': %timestamp());

yyyy = Four digit yearMM = Monthdd = Day number of monthhh = Hour of daymm = Minutes of hourss = Seconds of minute

In general do not create services to extend technology but instead to extend business logic and/or DB2 data.

• Calling a web service to do things like string processing (i.e. Replace all occurrences of x in y)?• Creating a web service that generates a GUID (Globally Unique Identifier) ‏

Don’t replace existing services that already work well. For example, just because EDI might be older technology doesn’t mean it should be replaced by XML. Obviously business partner requirements or money savings might warrant the use of web services.

Use the right technology for the job (i.e. don’t use XML web services to duplicate table data from one machine to another - potentially a huge waste of bandwidth). Software vendors like DataMirror should be used instead.

Be careful of polling services that have the potential to cause scalability issues down the road. An example of this would be making a web service request and in the background the request is submitted to a batch process, and then your program makes repeated requests to see if the batch process is done. It would instead be better to implement “wait” based technologies like data queues.

When NOT to do services

When to offer a serviceIn short, whenever a business need can justify it.

• Does implementing service xyz make money and save time? Typical ROI approaches used for most any programming change/addition should be used to determine when a service is necessary.

• Business partner requirement (i.e. credit card provider might be phasing out traditional approach for processing and moving to an XML based architecture) ‏

• Extend RPG business logic and DB2•Knowledgeable RPG programmers in i5 shops are going to be hot ticket items in the future as SOA adoption takes place, as long as they know how to extend using a variety of services.

• Are the resources (software, employees, knowledge, tooling) available to meet the task at hand.

• Implementing web services might involve your shop writing communication code from the ground up or purchasing packages to give you a head start.

• Plan Ahead - Start doing Web Services (or services in general) NOW to gain an understanding of how it will affect your IT infrastructure in the next 5 years. Knowing how to “extend your System i” and being ready to do it might be the key to the i5 staying alive in your shop.

Best Practices• Be cognizant of data types that are or aren't widely supported

– For example, Java/.NET have "unlimited" string sizes while RPG has a max of 65535 (or 16MB if you are on V6R1+)

– Define boundaries of fields in XML Schema Definition documents.• Develop for scalability

– If you are offering an XML web service that returns a list of records from DB2 make sure to cap the total number of records that can be returned (e.g. 100 at a time) and then allow subsequent requests to come in and specify which relative page or record to start on. This is fairly easy to do with embedded SQL in RPG.

• Develop an appropriate level of security– A little security on a public service (i.e. a simple "shared secret") is better

than nothing when invoking a service. That keeps somebody from simply hitting a web service and executing business processes and instead heads them off at the gate if they have an invalid password.

– In the same breath be careful not apply every security mechanism in the toolbox – inspiring usage is important if you are looking to foster adoption. For instance, if you aren't storing XML containing sensitive data after it is parsed then the need for the XML Encryption technology is lessened if HTTPS is used during transport.

Best Practices contd...• Don't over architect - granular isn't always good

– Granular is good SOME of the time. Having 10,000 granular web services available isn't necessarily a good thing because orchestrating when/how/where to call each one could be a pain in the gluteus maximus. Instead it is SOMETIMES better to wrap up many functions into a single call.(i.e. Create a service named Item_check() and have it internally do validation checks, inventory checks, valid quantity, configuration checks, etc vs. having four services to call for each of the aforementioned checks)‏

• Service Level Agreements (SLA)‏– Often times you get to assume things about the other system. For instance do

they take good care to ensure services have decent "up time". What happens if the service is down, or better stated, what is their recovery plan?

• SOA Standards– As your organization starts developing services (either from RPG to RPG or

RPG to .NET/Java/etc), pay special attention to conventions used and make efforts to govern the different practices. Think of the benefits gained from following coding standards within RPG and you can imagine how formulating rules of engagement for SOA will save time down the road. An example might be a central dictionary definition that describes the name and data type of different bits of information (i.e. Invoice is always shorthanded to “Inv” and is always an integer 10 digits in length - this will keep front end systems written in .NET/Java from passing you huge numbers and/or strings down the road)‏

SOA Scenario• Let's say your company makes shoes and a reseller recently engaged both

you and your competitor asking if they could resell your product on their website. The implementation timeline is four months. The reseller wants you to offer a mechanism they can call to retrieve real-time product info. The reseller has a competent Java IT staff that is ready to collaborate on technology matters with the bid winning party. Your current environment (i.e. business logic and data access) is RPG on DB2 on the IBM i. You could go the route of introducing Java programming into your environment in an effort to mirror the technology efforts of the reseller which could require re-tooling of RPG programmers; or you could simply revitalize your existing RPG application logic to get to an endpoint in a fraction of the time. This may require re-engineering some business logic to be more modular, but being that you are in the RPG environment, that can happen MUCH more easily than picking up Java for the same purpose.

• ROI: By revitalizing existing processes (read RPG business logic and DB2/400 Data) you are able to win the bid because you can offer an acceptable time line for completion.

SOA In Action!

Java Toolbox

SQL

XML

The following is an application I developed for a System i Network article coming out soon. It demonstrates three ways to connect to System i5 RPG *SRVPGM's

using IBM's Java Toolbox, SQL External Stored Procedures, and XML web services.

This was built to gain understanding of the work involved with each approach and also the benefits/drawbacks of each.

MYLIB/CUSTSV (*SRVPGM)‏

Note the absence of pass-back parms. The Java Toolbox doesn't

support them.Note the use of LIKEDS. The Java Toolbox can't pass more than 7 parameters to a

*SRVPGM sub procedure, but passing data structures can help get around that problem.

Java Toolbox: cust_getRec

XML Web Service: cust_getRec

SQL External Stored Procs: cust_getRec

Our Unique Opportunity

As RPG programmers we have a unique opportunity presenting itself!

We must task ourselves to leverage technology and make our machine accessible to the outside world while trying to keep as much on a single platform as possible to gain efficiencies.

We must learn and understand the new technologies so RPG and the IBM i are not replaced with other environments like .NET which can effectively double the complexity of your application infrastructure and “programming stack”.

Articles on SOA technologies"Automating FTP"http://www.systeminetwork.com/artarchive/6628/Automating_FTP.html

"Open The Door To Non iSeries Data"http://www.systeminetwork.com/artarchive/14473/Open_the_Door_to_Non_iSeries_Data.html

"Program with AS/400 TCP/IP Sockets"http://www.systeminetwork.com/artarchive/1508/Program_with_AS_400_TCP_IP_Sockets.html

"5 Tips for Integrating Java with System i Resources"http://www.systeminetwork.com/artarchive/20882/index.html

"5 More Tips for Integrating Java with System i Resources"http://www.systeminetwork.com/artarchive/20916/5_More_Tips_for_Integrating_Java_with_System_i_Resources.html

"The Essential Guide to Using XML for Data Interchange"http://www.systeminetwork.com/artarchive/20761/The_Essential_Guide_to_Using_XML_for_Data_Interchange.html

"Will Web Services Serve You?"http://www.systeminetwork.com/artarchive/19651/Will_Web_Services_Serve_You_.html

"SOAP, WSDL, HTTP, XSD, What?"http://www.systeminetwork.com/artarchive/20934/SOAP__WDSL__HTTP__XSD__What_.html

Aaron Bartell aaronbartell@mowyourlawn.comlead developer of RPG-XML Suite (www.rpg-xml.com)‏

and owner of www.MowYourLawn.com and check out his latest effort at www.SoftwareSavesLives.com

We have reached the end!

.com/aaronbartell