xml u.s.e.r. 101 u.s.e.r. 101.pdf · xml u.s.e.r. 101 what you need to get started: xml editing...

XML U.S.E.R. 101

David Andruchuk Computer Systems Design Associates, Inc.

October 26, 2010

What can i do…..i can do XML

XML U.S.E.R. 101

U.S.E.R. equates to UDTFs, SQL, Excel, RPG; all the tools that are present in

your workplace environment.

Using what we already are familiar with on our IBM i and PC; SQL, RPG, DB2,

RAD and more will allow us to get comfortable with XML so that you can say

you Grok XML (To grok is to share the same reality or line of thinking with

another physical or conceptual entity, intermingling of intelligence that necessarily affects both the observer and the observed) .

XML U.S.E.R. 101

Table of Contents

Introduction Introduction to XML

Programmatic Approach Using IBM RAD for XML Using Excel for XML Using SQL for XML Using RPG for XML

XML U.S.E.R. 101

Introduction to XML

XML U.S.E.R. 101

XML has become a standard for cross-platform communications

XML has or will impact your business if you continue to be in business

XML has gained acceptance as a preferred data exchange technology in spite of the limitations because of the flexibility and human readable formatting

XML U.S.E.R. 101

What you need to get started:

XML Editing tool(s) WDSC, Rational Application Developer (RAD) (IBM)

XMLSpy (Altova)

Stylus Studio (Progress Software Corporation)

Microsoft Office 2003 or higher

XML Reference(s) WDSC Step by Step (MC Press)

XML for eServer i5 and iSeries (MC Press)

Qshell for iSeries (MC Press)

XSLT 2nd Edition (WROX)

W3Schools Online Web Tutorials (www.w3schools.org)

XML U.S.E.R. 101

What is XML?

– XML is short for Extensible Markup Language

Pros of XML?

– XML is flexible

– XML is self documenting

– XML can replace or extend legacy systems

Cons of XML?

– XML is verbose

– XML can significantly increases the size of your data

– XML works best with hierarchal data in a relational data base

XML U.S.E.R. 101

There are three main components of XML

XML Document

Document containing data

XML Document Type Definition (DTD)

Document that defines the document structure of the XML Document

XML Schema Definition (XSD)

Document that defines the document structure of the XML Document in an XML-based alternative to DTD

XML U.S.E.R. 101

Structure of XML documents

An XML document has a tree-like structure that is hierarchical

The document must contain one and only one root element

An element is the parent of all the elements it contains

The elements that are inside a parent element are called its children

Similarly, the elements that have the same parent element are called siblings

XML U.S.E.R. 101

XML U.S.E.R. 101

Structure of DTD documents

Contains the list of tags which are allowed within the XML document and their types and attributes

Defines how elements relate to one another within the document's tree structure and specifies which attributes may be used with which elements

XML U.S.E.R. 101

XML U.S.E.R. 101

Structure of XSD documents

Contains the legal building blocks of an XML document similar to DTD

Defines how elements relate to one another within the document's tree structure and specifies which attributes may be used with which elements

Schema supports all data types used in most programming languages such as string, decimal, integer, Boolean, date and time

Supports complex type elements that allows you to define an element type that can consist of sub-elements

XML U.S.E.R. 101

XML U.S.E.R. 101

Using IBM RAD for XML

XML U.S.E.R. 101

Using IBM RAD for XML

This product provides a comprehensive visual Extensible Markup

Language (XML) development environment. Tools help you

create, import, edit, validate, and generate XML, XSL, DTDs, and

XML schemas (XSD) files. Mapping tools help you establish

relationships between XML files and between XML files and

relational database tables.

(C) Copyright IBM Corporation 1992, 2005. All Rights Reserved.

XML U.S.E.R. 101

Opening an XML document with RAD XML document

Right click > open with > rationalsdp (eclipse on RAD 7.0)

XML U.S.E.R. 101

Create workspace Enter C:\Grokking_XML\RAD_workspace and leave Use this as the default….unchecked!

XML U.S.E.R. 101

Open a Perspective Window>Open Perspective>Other>Resource

XML U.S.E.R. 101

Create new project In Navigator pane

Right Click>New>Project>Simple (General on RAD7.0)>Next

XML U.S.E.R. 101

Create new project (continued) Enter Project Name Grokking_XML and click Finish

XML U.S.E.R. 101

Import employee XML document to project Again in Navigator pane

Right click Grokking_XML project>Import>(General on RAD7.0) File System>Next>Browse>Select XML file>Finish

XML U.S.E.R. 101

Create generic schema (XSD) to enable XML Development Again in Navigator pane

Right click employee.XML document>New>Other>Show All Wizards.>XML>XML Schema>Next> Check Always enable capabilities…>OK

XML U.S.E.R. 101

Create generic schema (XSD) (continued) Enter or select the parent folder Grokking_XML>Leave file name NewXMLSchema.xsd>Finish

XML U.S.E.R. 101

Our new generic schema in project The Navigator pane is redisplayed showing the new XSD

XML U.S.E.R. 101

Delete generic schema (XSD) The generic XML Schema generated to enable XML can now be deleted

In the Navigator pane

Right click the NewXMLSchema.xsd document>Delete>Click Yes…

XML U.S.E.R. 101

Exit RAD Click File>Exit and if any resources need to be saved prior to exit, Click OK

XML U.S.E.R. 101

Using IBM RAD for XML recap

What we accomplished:

Created a project to Grok XML

Imported our sample XML document

What we could still accomplish:

Assign a XSD & DTD to our XML document for validation

Edit our XML, XSD & DTD documents in RAD to verify that the XSD & DTD generated are accurate in regards to

Repeating elements are correctly defined

Validations required for elements/attributes

Required elements are defined

Save our document(s) to a mapped IFS folder on our System i

XML U.S.E.R. 101

Using Excel for XML

XML U.S.E.R. 101

Using Excel for XML

New tools in Microsoft Office Excel 2003 and above make it easier to work

with Extensible Markup Language (XML).

We will learn how to use these tools to create XML lists and XML maps.

XML lists have the same look and feel as regular Excel lists, but they take

their data from XML files.

XML maps are sets of links between the cells in a worksheet and the elements

in an XML schema. You can import data into the map, modify the data, and

then save the changed data or export it for others to use.

XML U.S.E.R. 101

Create an Excel document In our Grokking_XML\RAD_workspace\Grokking_XML folder

Right click>New>Microsoft Excel Worksheet

XML U.S.E.R. 101

Create an Excel document (continued)

Now name it employee.xls

XML U.S.E.R. 101

Create an Excel document (continued) Now double click to open it

XML U.S.E.R. 101

Importing XML into Excel is easy There are two ways we can import our XML data into our spreadsheet: with or without an

existing Schema (XSD).

In this example, we will simply import the XML data and let Excel generate a schema template

for us.

Click Data>Import External Data> (Developer in 2007) Import Data

XML U.S.E.R. 101

Importing XML w/o a schema Now select the source of our import, our employee.xml document and click Open

XML U.S.E.R. 101 L

Importing XML w/o a schema (continued) Click OK when the Error in XML window is display to instruct Excel to create a schema for us

and apply to our worksheet

XML U.S.E.R. 101

Importing XML w/o a schema (continued) Except the defaults on the Import Data window and click OK

XML U.S.E.R. 101

Importing XML w/o a schema (continued) We have now successfully imported an XML document into an Excel worksheet

XML U.S.E.R. 101

Importing XML w/o a schema (continued) Notice the bold column names. This indicates that the schema mappings apply to the columns

so that an XML document could be generated from this worksheet. Let’s display the

underlying XML of this worksheet.

Click Data (Developer in 2007) >XML>XML Source

XML U.S.E.R. 101

Importing XML w/o a schema (continued) In the XML Source window displayed to the right of our worksheet, the Excel generated

Employees_Map schema(XSD) is displayed

XML U.S.E.R. 101

Importing XML w/o a schema (continued) If you click the Tips for mapping XML link, a help panel is displayed defining more about the

workings of Excel & XML

XML U.S.E.R. 101

Adding an XML Map using a schema (XSD) In this example, we will use an existing schema (XSD) as the template.

Click Data (Developer in 2007) >XML>XML Source…

XML U.S.E.R. 101

Adding an XML Map using a schema (XSD) (continued) Click XML Maps>Add…>employee.xsd>Open

XML U.S.E.R. 101

Adding an XML Map using a schema (XSD) (continued) Click Employees>OK in the Multiple Roots windows since Employees, not Employee is our root

XML U.S.E.R. 101

Adding an XML Map using a schema (XSD) (continued) Click OK in the XML Maps windows

XML U.S.E.R. 101

Adding an XML Map using a schema (XSD) (continued) In XML Source window, right click the Employees root>Map element…>Click OK in Map XML

Element window

XML U.S.E.R. 101

Our worksheet with the Employee Map Notice the markings on the Employee_Map showing that each element is now mapped to a

worksheet column and that the columns are now in bold and the XML containment box

XML U.S.E.R. 101

Importing the Employee XML document Click Data (Developer in 2007) >XML>Import>employee.xml>Import

XML U.S.E.R. 101

Our worksheet with the Employee XML data Notice the markings on the worksheet that surrounds the column data. The blue box

designates the containment of the XML Source template.

XML U.S.E.R. 101

Adding data to our worksheet Right click a row>Insert>Row>key data to the row to populate each column as you would in any

worksheet

XML U.S.E.R. 101

Saving/Exporting our worksheet To save the worksheet as XML data you can either:

Click File>Save As…>Save as type: XML Data(*.xml)>name file>Save

Click Data>XML>Export…>name file>Export

XML U.S.E.R. 101

Using Excel for XML recap


Imported our employee XML document and generated an XML Map

Applied an existing XSD schema as an XML Map to our worksheet

Imported our employee XML data

Added a row to our worksheet

Saved/Exported our XML data

What we could still accomplish:

Perform editing of column(s) in Excel to validate input

Save our document(s) to a mapped IFS folder on our System i

XML U.S.E.R. 101

Using SQL for XML

XML U.S.E.R. 101

Using SQL for XML composition

Using the System i SQL engine we can create an SQL Procedure that will

generate the Employee XML document used in our lab so that we can

understand the simplicity of XML composition without any special coding

requirements or tools on our existing System i.

XML U.S.E.R. 101

Coding the SQL stored procedure

XML U.S.E.R. 101

Coding the SQL stored procedure (continued)

XML U.S.E.R. 101

Creating the SQL stored procedure

XML U.S.E.R. 101

Executing the SQL stored procedure

XML U.S.E.R. 101

Some considerations for SQL Composition

SQL Stored Procedures contain two parts: a catalog entry and a program object. These two items can exist separate of each other but only work when both parts are there at run time and are not as easy to implement as using a SAVOBJ & RSTOBJ.

Work around for this limitation:

- Run the SQL Create Procedure statement on each box they are to be used on to create the catalog entry and program object

XML U.S.E.R. 101

Using SQL for XML decomposition

Using the System i SQL engine we can create an SQL UDTF (User Defined

Table Function) that will call an RPGLE program to decompose the Employee

Department XML document used in our lab so that we can use basic SQL

processing to drill into our XML document on our System i.

XML U.S.E.R. 101

SQL’s User Defined Table Functions (UDTF)

V5R2 DB2 for i5/OS was enhanced to support user-defined table functions (UDTF).

A UDTF can be used from SQL interfaces to access non-relational data sources such as S/36 files and stream files residing in the IFS.

The UDTF enhancement complemented the user-defined scalar function (UDF) capability added way back in V4R4 of OS/400.

A user-defined function is classified as scalar meaning that the function can only return a single value (or output parameter). That’s where UDTFs come into play with their ability to return multiple values in the form of a table (or result set).

XML U.S.E.R. 101

RPG’s XML-INTO Built in Function (BIF)

V5R4 ILE RPG for i5/OS was enhanced to facilitate decomposition of an XML document into data structures defined in the program using a non-validating parser.

The new operation codes and built-in functions makes the decomposition of the XML data and population of data structure fields automatic and efficient without the developer having to manipulate storage to get the XML data in to workable form. The complexity has been removed from the XML decomposition process and the developer now can just concentrate on business logic required to implement the XML data.

XML U.S.E.R. 101

Coding the SQL UDTF(continued)

XML U.S.E.R. 101

Coding the SQL UDTF (continued)

XML U.S.E.R. 101

Coding the RPGLE program

XML U.S.E.R. 101

Coding the RPGLE program (continued)

XML U.S.E.R. 101

Creating the RPGLE program (continued)

XML U.S.E.R. 101

Coding the RPGLE program (continued)

XML U.S.E.R. 101

Executing the code

XML U.S.E.R. 101

Executing the code (continued)

XML U.S.E.R. 101

Some considerations for SQL Decomposition

This example is a simple demonstration and does not take in to account any error checking or large XML documents. The simple XML document and DS built to

parse it can only process up to 100 records.

Use the XML-HANDLER OpCode instead of the XML-INTO to be able to handle unknown sizes of XML documents.

While this example shows XML parsing, a UDTF program can also do any DB processing, calculations, specific business logic, etc. which makes the UDTF a very versatile addition to the SQL toolbox when a result set is desired.

XML U.S.E.R. 101

Using SQL for XML recap


Created an SQL procedure to generate an XML document from a DB table

Created the SQL UDTF and RPGLE program to parse an XML document and return the result set to any SQL interface as table data

XML U.S.E.R. 101

Using RPG for XML

XML U.S.E.R. 101

Using RPG for XML Composition The ILE RPG Compiler as of the V5R4 release of i5/OS has no native operations codes and built in functions that facilitate composition of an XML document like the decomposition we cover in this presentation. So can RPG compose an XML document from a table? Using the R.P.G. design

principle; Read, Prepare and Generate we can accomplish the same results

for any table on our system.

Choosing to use the old school approach of a Command/Panel Group/CLLE program front ending a SQL RPGLE program, we can generate an XML document for any table on our system.

XML U.S.E.R. 101

Coding a CMD interface

XML U.S.E.R. 101

Coding the UIM help panel

XML U.S.E.R. 101

Coding the UIM help panel (continued)

XML U.S.E.R. 101

Coding the CL validation program

XML U.S.E.R. 101

Coding the CL validation program (continued)

XML U.S.E.R. 101

Coding the SQLRPGLE module for composition

XML U.S.E.R. 101

Coding the SQLRPGLE module for composition (continued) Using the IBM table SYSCOLUMNS in the QSYS2 library, we can retrieve the column names

for the supplied file and store them in the Columns data structure array. We then use the

column names to retrieve the column values for the table and store them in the Values data

structure array

XML U.S.E.R. 101

Coding the SQLRPGLE module for composition (continued)

XML U.S.E.R. 101

Coding the SQLRPGLE module for composition

XML U.S.E.R. 101


XML U.S.E.R. 101

Coding the SQLRPGLE module for composition (continued) We check each Value data retrieved from our Values data structure array for the existence of

predefined XML Character Entities that will cause problems for our XML document

XML U.S.E.R. 101

Coding the SQLRPGLE module for composition (continued) With our data cleansed, we can begin the process of wrapping each of the data values stored

for our columns in the Values data structure array with our column names from the Columns

data structure array

XML U.S.E.R. 101


XML U.S.E.R. 101

Executing the code

XML U.S.E.R. 101

Some considerations for RPG Composition

This program example has a limitation of 100 columns and 100 values, each of which is 128 characters, so that will have to be taken into consideration if the any table being processed exceeds these values.

While this example shows XML parsing, a UDTF program can also do any DB processing, calculations, specific business logic, etc. which makes the UDTF a very versatile addition to the SQL toolbox when a result set is desired.

XML U.S.E.R. 101

Using RPG for XML Composition recap


Created a CMD/PNLGRP/CLLE/SQLRPGLE utility to generate an XML document for any passed DB table

Used RPG and the power of embedded SQL, we were able to quickly reformat any provided DB2 table and create a self-described exchange document to share with an internal or external source

XML U.S.E.R. 101

Using RPG for XML Decomposition The ILE RPG Compiler as of the V5R4 release of i5/OS has new native operations codes and built in functions that facilitate decomposition of an XML document into data structures defined in the program using a non-validating parser. For more information on the XML parser used by ILE RPG, please refer to Chapter 11 in the WebSphere Development Studio ILE RPG Programmer’s Guide, SC09-2507.

The advantage of using the new operation codes and built-in functions added to RPG is that once the hierarchy of the XML document is replicated using data structures in our RPG program code, the decomposition of the XML data and population of data structure fields is done automatically and efficiently without the developer having to manipulate storage to get the XML data in to workable form. The complexity has been removed from the XML decomposition process and the developer now can just concentrate on business logic required to implement the XML data. No pointers, API interfaces, buffer manipulation, just straight forward RPG coding.

XML U.S.E.R. 101

Coding a CMD interface

XML U.S.E.R. 101

Coding the UIM help panel

XML U.S.E.R. 101

Coding the CL validation program

XML U.S.E.R. 101

Coding a CL validation program(continued)

XML U.S.E.R. 101

Coding the SQLRPGLE module for decomposition

XML U.S.E.R. 101

Coding the SQLRPGLE module for decomposition (continued)We code our WriteTbl internal Prototype to write to disk and our data structures for the target

table. The Employees data structure array is used to store our parsed XML data

XML U.S.E.R. 101

Coding the SQLRPGLE module for decomposition (continued)We now can use the RPG BIF XML-INTO to parse our XML data into our Employees data

structure array for processing

XML U.S.E.R. 101

Coding the SQLRPGLE module for decomposition (continued)With our XML data now parsed and stored in the Employees data structure array we can loop

through the array and begin writing to our target table

XML U.S.E.R. 101

Coding the SQLRPGLE module for decomposition (continued)Data structure array loop and writing to target table continued…

XML U.S.E.R. 101

Coding the SQLRPGLE module for decomposition (continued)

XML U.S.E.R. 101

Executing the code

XML U.S.E.R. 101

Some considerations for RPG Decomposition

Use of the Java Virtual Machine (JVM) on a System i requires some simple tuning to optimize performance. Since we use the JVM to run our Transform class it is important to note the memory pool that your JVM executes in be set appropriately to improve performance.

XML-INTO operation cannot handle empty elements or attributes that contain zero length numeric, date, time or timestamp fields during decomposition.


- Use a stylesheet (XSL) as we did to remove the empty limitations

noted above

XML U.S.E.R. 101

Some considerations for RPG Decomposition (continued)

The RPG compiler limits character variables to 65535 characters in length. Data Structures, being considered character fields, are subject to this length limitation as well. You must be aware when replicating the XML tree using data structures, especially data structures that are dimensional, that the total size of the data structure tree does not exceed this limit as well.


- break apart the data structures without nesting them in side each other and

use multiple XML-INTO operations to decompose specifying the specific

path of the XML document that matches the data structure names.

XML U.S.E.R. 101

Questions?

xml u.s.e.r. 101 u.s.e.r. 101.pdf · xml u.s.e.r. 101 what you need to get started: xml editing...

Documents