19 – 21 MARCH 2006 Riyadh, Saudi Arabia19 – 21 MARCH 2006 Riyadh, Saudi Arabia

XML and Web Services support in SQL Server 2005XML and Web Services support in SQL Server 2005

Michael StoreyXpertise Training Ltd (UK)

OutlineOutline

The XML Data Type

Schema validated and untyped xml

XML type indexes

FOR XML and OpenXML enhancements

Using XQuery with the XML data type

Updating the XML data type with XQuery DML

SQL Server 2005 Web Service Support

Accessing HTTP endpoints with .NET clients

XML in a relational databaseXML in a relational database

XML can be stored as text

loses much of value of XML representation

XML can decomposed into multiple relational tables

allows use of relational technologies

XML can be stored as an xml data type

allows use of XML technologies

Using the XML data typeUsing the XML data typeCREATE TABLE xml_tab ( the_id INTEGER, xml_col XML)

CREATE PROCEDURE transform ( @x XML, @y XML OUTPUT)AS...

CREATE FUNCTION simple ( @x NVARCHAR(max))RETURNS XMLASDECLARE @a XMLSET @a = @x...RETURN @a

XML columnXML column

XML column can store well-formed XML

XML 1.0 recommendation

documents or fragments

XML schema in databaseXML schema in database

XML schemas used by XML data types must be in database

Create XML SCHEMA COLLECTION

collection name associated with XML instance

Create XML SCHEMA COLLECTION requires literal schemas

CREATE XML SCHEMA COLLECTION geocollAS '<xs:schema ...

targetNamespace= urn:geo>

... </xs:schema>'

literal schematargetNamespace

CREATE TABLE Locations( location xml(geocoll) ...)

reference toschema collection

Using strongly typed XMLUsing strongly typed XML

CREATE TABLE point_tab( id int IDENTITY primary key, -- geocoll include schema for 'urn:geo' namespace thepoint xml(CONTENT geocoll)GO

-- this works, schema-valid PointINSERT INTO point_tab VALUES( '<Point xmlns="urn:geo"><X>10</X><Y>20</Y></Point>')

-- this insert fails, value foo is not a dim (integer)INSERT INTO point_tab VALUES( '<Point xmlns="urn:geo"><X>10</X><Y>foo</Y></Point>')

XML indexesXML indexes

You can create XML INDEXes on an XML column

optimizes XML Queries on the column

primary XML index must be created first (node table)

three specialized index types also available

VALUE – optimizes content queries

PATH – optimizes structure queries

PROPERTY – optimizes name/value pair predicates

CREATE TABLE xml_tab ( id integer primary key, doc xml)GOCREATE PRIMARY XML INDEX xml_idx on xml_tab (doc)GO

Enhancements to SELECT...FOR XMLEnhancements to SELECT...FOR XML

SELECT FOR XML is an extension to Transact-SQL

usually produces a stream

in SQL Server 2005, it can also produce an XML data type

use TYPE keyword after FOR XML

can be used with RAW, AUTO formats

can be used to query a "collection" of documents

DECLARE @x xmlSET @x = SELECT * FROM authors FOR XML AUTO, TYPE

FOR XML PATHFOR XML PATH

SELECT FOR XML PATH allows shaping of output

FOR XML AUTO, RAW allow little shaping

FOR XML EXPLICIT is complex to write

FOR XML PATH uses path-syntax in column aliases

allows "creation" of elements

simpler than XML EXPLICIT

can specify namespaces with XMLNAMESPACES function

XML PATH exampleXML PATH example

WITH XMLNAMESPACES('urn:authors' AS au)SELECT au_id as [@au:authorid], au_fname as [name/firstname], au_lname as [name/lastname]FROM authors FOR XML PATH

<!– one row per selected row --><row xmlns:au="urn:authors" au:authorid="111-11-1111"> <name> <firstname>Bob</firstname> <lastname>Smith</lastname> </name></row>...

Producing an XML Schema with FOR XMLProducing an XML Schema with FOR XML

XMLSCHEMA keyword prepends XML Schema to FOR XML

can be used with other keywords

schema namespace can be specified

DECLARE @x xmlSET @x = (SELECT * FROM authors FOR XML AUTO, TYPE, XMLSCHEMA('urn:authors')

More FOR XML EnhancementsMore FOR XML Enhancements

Many refinements and extensions to FOR XML

element-centric XML using FOR XML RAW

generate xsi:nil for NULL database values

FOR XML AUTO/RAW, ELEMENTS XSINIL

nested FOR XML queries

can specify ROOT element

can name <row> element in FOR XML RAW

changes to nesting algorithm in XML AUTO

supports new data types

varchar(max), nvarchar(max), varbinary(max)

UDTs (must be cast/converted to XML in query)

Using XML Data Type with OpenXmlUsing XML Data Type with OpenXml

OpenXml function can use XML data type

originally used varchar or TEXT input

still must be parsed with sp_xml_preparedocument

permits user-defined fill into existing tables

overflow column may be xml-- this assumes that the document-- looks exactly like the tableCREATE PROCEDURE fillTablesWithMyDoc(@doc xml)ASdeclare @idoc intexec sp_xml_preparedocument @idoc OUTPUT, @docSELECT * FROM OpenXML(@idoc, '/ROOT',2) WITH (mytable) AS AINTO mytableexec sp_xml_removedocument @idoc

Inserting XML Through Bulk LoadInserting XML Through Bulk Load

New BULK Rowset Provider can insert XML

inserts from file to XML column

using SINGLE_BLOB option inserts only XML column

SINGLE_CLOB, SINGLE_NCLOB also work

SINGLE_BLOB avoids encoding problems

format file required if inserting more than one rowCREATE TABLE invoices ( rowid int primary key identity, invoice xml )INSERT invoices SELECT * FROM OPENROWSET (BULK 'c:\invoice.txt', SINGLE_BLOB) as X

XML data type and FOR XML enhancements

SQL Server 2005 Support of XQuerySQL Server 2005 Support of XQuery

XQuery is supported through methods on the XML type

xml.exist - returns bool

xml.value - returns scalar

xml.query - returns XML data type instance

xml.nodes - returns one column rowset w/XML column

xml.modify - modifies an instance

These methods can return

XML data type (query) or SQL Scalar (exist, value)

columns in rowsets - when used with SQL SELECT

variables

XQuery ExpressionsXQuery Expressions

XQuery is a superset of XPath

can use XPath or FLWOR expressions

almost all valid XPath statements are also XQuery statements

XPath used to select values in FLWOR expressions

for $p in /people/personwhere $p/age > 30order by $p/age[1]return $p/name

FLWOR ExpressionsFLWOR Expressions

XML Queries are FLWOR expressions

made up of five types of clause

FOR

LET (not supported by SQL Server 2005)

WHERE

ORDER BY

RETURN

ConstructorsConstructors

Constructors are used to construct nodes

constructors can construct static content; XML 1.0 + Namespaces serialized form

constructors can construct dynamic content

dynamic content can be content of elements and/or attributes

dynamic part of constructor enclosed in curly braces

or use literal constructor

<person> <name>Martin</name></person><person> <name>Simon</name></person>

Dynamic constructors: Text

return <person><name> {$p/name[1]/givenName[1]/text()} </name></person>

Literal constructorsLiteral constructors

<person name = "Martin"><person name = "Simon">

return element person { attribute name {data($p/name[1]/givenName[1]/text()[1])} }

value and exist XML data type methodsvalue and exist XML data type methods

value() data type method accesses scalar value in xml column

XQuery or simple XPath can be used

exist() data type method tests conditions in xml column

XQuery or simple XPath can be used

used with column name separated by "."

select invoice.value('*[1]/@ID','int')from Invoices

<Invoice ID="12"> ...</Invoice>

Invoices table

invoice column

123426

XPath expression getsvalue of @ID attribute

xml.existxml.exist

xml.exist uses XML type and XQuery expression

returns false if query returns NULL sequence

returns true otherwise

Usable in XML type check constraints

xml functions must be encapsulated as UDF

xml.valuexml.value

xml.value return a SQL Server scalar type

SQL type, xml data type instance, XQuery as input

returns scalar type or NULL

cannot return XML type instance

Usable within SQL Server

in predicates

as result values

Using xml.valueUsing xml.value-- insert some rowsINSERT xml_tab VALUES('<people><person name="curly"/></people>')INSERT xml_tab VALUES('<people><person name="larry"/></people>')INSERT xml_tab VALUES('<people><person name="moe"/></people>')

-- this query SELECT id, xml_col.value('/people/person/@name','varchar(50)') AS nameFROM xml_tab

-- yields this resultset id name--------------------------1 curly2 larry3 moe

xml.queryxml.query

xml.query returns an XML data type instance

XML type instance and XQuery are inputs

can return scalar type as XML fragment

can use constructors to compose new data

<Invoice> <LineItem>Soda</LineItem> <LineItem>Ice</LineItem></Invoice>

from invoice column in Invoices table

SELECT invoice.query('Invoice/LineItem') FROM Invoices

<LineItem>Soda</LineItem><LineItem>Ice</LineItem>

selects LineItem'sfrom XML data

-- yields this resultset --------------------------<LineItem>Soda</LineItem><LineItem>Ice</LineItem>

Using Relational Data in XQueryUsing Relational Data in XQuery

SQL Server XQuery can combine relational and XML

sql:variable - uses TSQL variables

sql:column - uses column value

same row as XML type column

Using Relational Data With XMLUsing Relational Data With XML

-- returns <li>moe in record number x</li> -- where x is the ID column, or blank columnSELECT xml_col.query(' for $b in //person where $b/@name="moe" return <li>{ data($b/@name) } in record number {sql:column("xml_tab.id")}</li> ') FROM xml_tab

-- returns <li>moe is a stooge</li> DECLARE @occupation VARCHAR(50)SET @occupation = ' is a stooge'SELECT xml_col.query(' for $b in //person where $b/@name="moe" return <li>{ data($b/@name) } {sql:variable("@occupation") }</li> ') FROM xml_tab

nodes methodnodes method

nodes can decompose an XML data type

similar to value(), but produces

references to XML nodes

many nodes rather than single scalar value

result is one-column rowset

T-SQL CROSS APPLY can be use to produce rowset

similar to OpenXML

better performance, XML column is already parsed

-- xmlinvoices table contains custid, xmlinv columns-- return one row for each lineitemSELECT custid, tab.col.query('itemno') --get itemno subelement FROM xmlinvoices CROSS APPLY xmlinv.nodes('/invoice/lineitem') as tab(col) WHERE custid = 'ALFKI'

XQuery DML - Using xml.modifyXQuery DML - Using xml.modify

-- use modify to insert a subelementSET @x.modify( 'insert <InvoiceDate>2002-06-15</InvoiceDate> into /Invoice[1] ')

-- or insert an attributeSET @x.modify('insert attribute status{"backorder"} into /Invoice[1] ')

-- this deletes all LineItem elementsSET @x.modify('delete /Invoice/LineItems/LineItem')

-- change the value of the CustomerName elementSET @x.modify('replace value of /Invoice[1]/CustomerName[1]/text()[1] with "John Smith" ')

XQuery

Why SQL Server and Web Services?Why SQL Server and Web Services?

Databases can deal with Web service clients directly

Using TDS limits potential clients

Web service check box requirement

OLE DB, ODBC, .NET data provider required

JDBC driver required

free TDS - reduced functionality

Web services from SQL Server

HTTP, all clients have it

XML, all clients support it

SQL Server Web Services descriptionsSQL Server Web Services descriptions

SQL Server Web services configured in database

just like other database features

SQL Server Web service operations part of database

assemblies, sprocs in database

just like other database functionality

SQL Server Web service security part of data base

just like other database security

SQL Server Web services fully described inside database

just as database objects should be

SQL Server directly leverages HTTP SQL Server directly leverages HTTP

Uses HTTP kernel protocol stack

less layers, better scalability and thru-put

requires WS2K3 or XP SP2

Can use SSL

Inetinfo.exe not used

Web Service configurationWeb Service configuration

Web service endpoint must be configured

location, access, and security

Web service operations must be configured

specific operation names and/or ad hoc sql batches

CREATE ENDPOINT for HTTP used to configure

names endpoint

before FOR SOAP configures endpoint

after FOR SOAP configures soap operations

CREATE ENDPOINT genled ... AS HTTP(...)FOR SOAP(...)

endpoint name

endpoint configuration

operations configuration

Endpoint authenticationEndpoint authentication

Basic authentication supported (w/Windows login)

WS-Security supported w/SQL login (WS-I standard)

but must specify an endpoint authentication type too

Digest and integrated authentication supported

integrated requires client in same or trusted domain

Multiple authentication types supported

Client must also be granted CONNECT ENDPOINT privilege

Clear_port and ssl_port may be specified

CREATE ENDPOINT genLed... AS HTTP(Authentication = (INTEGRATED), PORTS = (CLEAR), CLEAR_PORT = 8082...)FOR SOAP

CREATE ENDPOINT genLed... AS HTTP(Authentication = (BASIC), PORTS = (SSL)... )FOR SOAP

Endpoint startupEndpoint startup

Endpoint must be in started state to accept request

initial state can be started, stopped, disabled

disabled state requires SQL Server restart to enable

Endpoint state changed by using ALTER HTTP ENDPOINT

CREATE ENDPOINT generalLedgerSTATE = STARTEDAS HTTP ( ... )FOR SOAP ( ... )

endpoint willaccept requests

CREATE ENDPOINT generalLedgerSTATE = STOPPEDAS HTTP ( ... )FOR SOAP ( ... )

endpoint will notaccept requests

Operation configurationOperation configuration

Operations supported must be configured

operation name, function, schema

defined after FOR SOAP

Context for Web methods must be configured

namespace, database used, etc.

CREATE ENDPOINT genLedSTATE = STARTED AS HTTP ( ... )FOR SOAP (WEBMETHOD a (...),WEBMETHOD b (...), ... )

operations supportedby endpoint

context for operations

Endpoint operationEndpoint operation

Webmethod is used to define an endpoint operation

namespace and name of operation

name of stored proc or udf that implements it

standard schema, none or default

Multiple operations may be defined

FOR SOAPWEBMETHOD'http://tempUri.org/'.'CustomerInfo'(name='Pubs.dbo.GetCInfo', schema=STANDARD ),WEBMETHOD ... ,

client sees aCustomerInfooperation

operation is implementedby the GetCInfo sprocin the pubs database

schema definition isavailable for thisoperation

Endpoint contextEndpoint context

Context operations execute in must be defined

automatic WSDL generation

ad hoc queries, disabled by default

database used, default for login if not specified

namespace for endpoint

FOR SOAP (WEBMETHOD ...,

WSDL = DEFAULT,BATCHES = ENABLED,DATABASE = 'pubs',NAMESPACE = 'urn:Accounting' )

Typical Web Service configurationTypical Web Service configurationCREATE ENDPOINT AccountingSTATE = STARTEDAS HTTP ( SITE = 'www.account.com', PATH = '/nwind', AUTHENTICATION = (INTEGRATED), PORTS = (CLEAR)FOR SOAP ( WEBMETHOD 'http://tempUri.org/'.'GetCustomerInfo' (name='Northwind.dbo.GetCustInfo', schema=STANDARD ),

WSDL = DEFAULT, BATCHES = ENABLED, DATABASE = 'northwind', NAMESPACE = 'http://Northwind/Customers')GO

Creating and calling a SQL Server 2005 web service

static void Main(string[] args){ // instantiate the proxy class myprog.localhost.soap s = new myprog.localhost.soap(); // use security of current client principal s.Credentials = System.Net.CredentialCache.DefaultCredentials; object[] oa; try { oa = s.AuthorsByRoyaltyAsDataSets(50); processResults(oa); } catch (Exception e) { Console.WriteLine("{0} {1}", e.GetType().ToString(), e.Message); if (oa) Console.WriteLine("error, {0} results", oa.Length); }}

.NET Client to Retrieve Results.NET Client to Retrieve Results

void processResults(object oa[]) { for (int i=0;i<oa.Length;i++) { switch (oa[i].GetType().ToString()) { case "System.Data.DataSet": DataSet ds = (DataSet)oa[i]; Console.WriteLine(ds.GetXml()); break; case "myprog.localhost.SqlMessage": SqlMessage sqlm = (SqlMessage)oa[i]; Console.WriteLine("Error: {0} in proc {1}", sqlm.Message, sqlm.Procedure); break; case "System.Xml.XmlElement": XmlElement elem = (XmlElement)oa[i]; Console.WriteLine(elem.OuterXml); break; // more cases... default: Console.WriteLine("Unexpected type {0}:{1}", oa[i].GetType().ToString(), oa[i].ToString()); break; } }}

Processing ResultsProcessing Results