xml and sql server
DESCRIPTION
XML and SQL ServerTRANSCRIPT
1
xml and sql server
2
sql server and xml
• The way to present data in a platform independent manner is through XML. The client-cursor, SQLOLEDB provider, and SQL Server internally all provide XML support.
3
why databases and xml?
• There are many reasons for integrating XML format and relational databases– Most data is stored in a database for administrative and
concurrency reasons– XML can be used as a method of data exchange– XML has become a popular data representation method– XML can represent hierarchical composition of multiple
relational tables– XML data can be decomposed into multiple relational tables
4
sql server and xml choices
• Many facets of SQL Server and XML integration– SQL Server 2000 has built-in XML functionality– SQLOLEDB 2.6 provider enables XML query dialects and
streamed input and output– ISAPI application enables HTTP access– XML for SQL Server Web Release 1 adds updategrams and
XML bulk load– SqlClient data provider supports extended XML function– XML for SQL Web Release 2 adds client-side XML, XSD
support and .NET support of SQLOLEDB 2.6 functionality
5
xml support in sql server 2000
• SQL Server 2000 includes internal XML support– SELECT…FOR XML queries return the resultset as a
stream of XML– sp_xml_preparedocument extended stored procedure
parses a VARCHAR variable containing XML, returns a document handle
– OpenXML function uses the document handle to produce a rectangular resultset
6
select … for xml
• SQL Server 2000 has some XML query capability built in– "FOR XML" on the SELECT statement is an extension to
Transact-SQL– Three different variations of XML output (RAW, AUTO and
EXPLICIT)– Produces streamed single-column XML as output– Element-normal form and XDR schema/DTD inclusion is
supported
7
for xml variations
• Different "FOR XML" variations server different purposes– RAW mode produces one set of <row> elements, even when
using JOIN– AUTO mode produces hierarchies with separate nested
elements for each JOINed table– EXPLICIT mode produces arbitrary shapes using UNION
queries and special columns and column names
8
elements, attributes and data values
• SQL column values can be stored in elements or attributes– All data in elements/attributes known as element/attribute
normal form XML– RAW always produces attribute-normal form– AUTO can produce element-normal or attribute-normal form– EXPLICIT can produce mixtures of elements and attributes
9
-- this query:SELECT Customers.CustomerID, Orders.OrderIDFROM Customers, OrdersWHERE Customers.CustomerID = Orders.CustomerIDORDER BY Customers.CustomerIDFOR XML RAW
-- produces this XML output document fragment<row CustomerID="ALFKI" OrderID="10643" /><row CustomerID="ALFKI" OrderID="10692" /><row CustomerID="ALFKI" OrderID="10703" /><row CustomerID="ALFKI" OrderID="10835" /><row CustomerID="ANATR" OrderID="10308" />
XML RAW query and output
10
-- this query:SELECT Customers.CustomerID, Orders.OrderIDFROM Customers, OrdersWHERE Customers.CustomerID = Orders.CustomerIDORDER BY Customers.CustomerIDFOR XML AUTO
-- produces the following XML document fragment<Customers CustomerID="ALFKI"><Orders OrderID="10643" /><Orders OrderID="10692" /><Orders OrderID="10702" /><Orders OrderID="10835" />
</Customers><Customers CustomerID="ANATR"><Orders OrderID="10308" />
</Customers>
XML AUTO query and output
11
-- this query:SELECT 1 as Tag, NULL as Parent,
Customers.CustomerID as [Customer!1!CustomerID],NULL as [Order!2!OrderID]
FROM CustomersUNION ALLSELECT 2, 1,
Customers.CustomerID,Orders.OrderID
FROM Customers, OrdersWHERE Customers.CustomerID = Orders.CustomerIDORDER BY [Customer!1!CustomerID]FOR XML EXPLICIT
-- produces this output document fragment<Customer CustomerID="ALFKI">
<Order OrderID="10643"/><Order OrderID="10692"/><Order OrderID="10702"/>
</Customer>
FOR XML Explicit format
12
storing xml in sql server
• Stored procedures and UDF can assist decomposition of XML documents– System stored procedure parse an XML document into a
"DOM handle"– OpenXML function produces a rectangular resultset from
XML– Nodesets of the XML document are mapped to columns with
XPath mappings– Can use INSERT...SELECT or UPDATE/DELETE to insert
document nodesets
13
declare @idoc intdeclare @doc varchar(1000)set @doc ='<ROOT><Customers CustomerID="VINET" ContactName="Paul Henriot">
<Orders CustomerID="VINET" EmployeeID="5"><OrderDetails OrderID="10248" ProductID="11" /><OrderDetails OrderID="10248" ProductID="42" />
</Orders></Customers><Customers CustomerID="LILAS" ContactName="Carlos Gonzlez">
<Orders CustomerID="LILAS" EmployeeID="3"><OrderDetails OrderID="10283" ProductID="72" Quantity="3"/>
</Orders></Customers></ROOT>'
exec sp_xml_preparedocument @idoc OUTPUT, @docSELECT *FROM OpenXML (@idoc, '/ROOT/Customers/Orders/OrderDetails',2)WITH (CustomerID varchar(10) '../@CustomerID',
ProdID int '@ProductID',Qty int '@Quantity')
exec sp_xml_removedocument @idoc
Using OpenXML and column patterns
14
DECLARE @h intDECLARE @xmldoc varchar(1000)
set @xmldoc ='<root><stores stor_id="8888" stor_name="Bob''s Books"
stor_address="111 Somewhere" city="Portland" state="OR"zip="97225">
<discounts discounttype="A Discount" stor_id="8888" discount="50.00"/></stores> </root>'
EXEC sp_xml_preparedocument @h OUTPUT, @xmldoc
INSERT INTO storesSELECT * FROM OpenXML(@h,'/root/stores')WITH stores
INSERT INTO discountsSELECT * FROM OpenXML(@h,'/root/stores/discounts')WITH discounts
EXEC sp_xml_removedocument @h
Decomposition using OpenXML
15
sqloledb and xml
• SQLOLEDB provider in OLE DB 2.6 enables XML features– XML functionality separated into SQLXMLX.DLL– Command coclass supports streamed input and output– Extra properties on Command enable stylesheets and
database/XML mappings• Two new query dialects enable XML format queries
– MSSQLXML wraps SQL or XPath queries in XML (requires streamed input)
– XPath queries are permitted as strings
16
<ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"><sql:query>SELECT Customers.CustomerID, Orders.OrderIDFROM Customers, OrdersWHERE Customers.CustomerID = Orders.CustomerIDORDER BY Customers.CustomerIDFOR XML RAW</sql:query></ROOT>
<ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"><sql:xpath-query mapping-schema="rtest.xdr">/Customers[@CustomerID="ALFKI"]
</sql:xpath-query></ROOT>
MSSQLXML input documents
17
using xpath queries
• Support of XML XPath query language requires two pieces– Mapping schema maps tables, columns, etc. to a "virtual
XML document"– SQLOLEDB 2.6 mapping schemas are XDR; WebRelease 2
adds support for XSD– XPath query is applied to virtual document to produce output– Templated and string based XPath queries supported
• Internally, XPath + Mapping Schema is translated to "SELECT FOR XML EXPLICIT" SQL query by SQLOLEDB
18
<Schema xmlns="urn:schemas-microsoft-com:xml-data"xmlns:sql="urn:schemas-microsoft-com:xml-sql">
<ElementType name="discounts" sql:relation="discounts" ><AttributeType name="discounttype" /><AttributeType name="stor_id" /><AttributeType name="discount" /><attribute type="discounttype" sql:field="discounttype" /><attribute type="stor_id" sql:field="stor_id" /><attribute type="discount" sql:field="discount" /></ElementType>
<ElementType name="stores" sql:relation="stores" ><AttributeType name="stor_id" /><AttributeType name="stor_name" /><attribute type="stor_id" sql:field="stor_id" /><attribute type="stor_name" sql:field="stor_name" /><element type="discounts" ><sql:relationshipkey-relation="stores“ key="stor_id"foreign-relation="discounts“ foreign-key="stor_id" />
</element></ElementType></Schema>
An XDR mapping schema
19
Sub DoXPath(xmlDoc as MSXML.DOMDocument)Dim conn as New ADODB.ConnectionDim cmd as New ADODB.CommandDim stmOut as New ADODB.Stream
conn.Open "provider=sqloledb;uid=sa;initial catalog=pubs"Set cmd.ActiveConnection = connstmOut.Open
' xmlDoc contains XPath query from next slideSet cmd.CommandStream = xmlDoc
' Set XPath dialect, output stream and mappingscmd.Dialect = "{ec2a4293-e898-11d2-b1b7-00c04f680c56}"cmd.Properties("Output Stream") = stmOutcmd.Properties("Base path") = "c:\schemas"cmd.Properties("Mapping schema") = "mappings.xdr"
' Return XML result in output streamcmd.Execute , , adExecuteStreamEnd Sub
Using an XML mapping schema (ADO)
20
-- query input<ROOT xmlns:sql="urn:schemas-microsoft-com:xml-sql"><sql:xpath-query mapping-schema="rtest.xdr">/stores</sql:xpath-query></ROOT>
-- returned document<root><stores stor_id="6380" stor_name="Eric the Read Books" /><stores stor_id="7066" stor_name="Barnum's" /><stores stor_id="7067" stor_name="News & Brews" /><stores stor_id="7131” stor_name="Doc-U-Mat: Quality" /><stores stor_id="7896" stor_name="Fricative Bookshop" /><stores stor_id="8042" stor_name="Bookbeat"><discounts discounttype=“Usual" stor_id="8042"
discount="5.00" /></stores>
</root>
Using an XML mapping schema - XPath query and results
21
sqlxmloledb provider
• Web Release 2 includes new OLE DB provider– SQL Server 2000 handles "FOR XML" inside SQL Server;
SQLXMLOLEDB provider exposes analogous functionality on client
– Preprocesses "FOR XML" queries so only raw SQL is submitted to SQL Server
– Post-processes SQL resultset in XML format on client– SQLXMLOLEDB is an OLE DB service provider - currently
only supports SQLOLEDB as underlying provider– Enables SQL Server 2000-like support for SQL Server 7
22
iis support for database driven xml
• Any version of SQL Server can directly export XML through IIS– SQL Server 2000 uses native "FOR XML" clause and
SQLOLEDB provider– HTTP GET and POST supported– Query templates enable fixed and parameterized queries
XPath and SQL supported
– Transactional XML Updategrams in WR1– Client-side XML is supported in WR2 through
SQLXMLOLEDB provider– Client-side XML enables using "FOR XML" queries with SQL
Server 7 or 2000
23
Using “FOR XML” and SQL Server 2000
Client SQLISAPI.DLL SQLServerHTTP
Submit through SQLOLEDB
Add stylesheet
TDS
XML
24
Using SQLXMLOLEDB provider
Client SQLISAPI.DLL SQLServerHTTP
Format SQL from XMLSubmit by SQLXMLOLEDB
Format output XMLAdd stylesheet, DTD, schema
TDS
25
updategrams
• Updategrams use XML format to update SQL Server– Supported in Web Release 1 through ISAPI program– Insert/Update/Delete consist of XML-format before and after
images– SQLOLEDB converts these XML format to SQL– Batching of updates and transactions are supported– Web Release 2 supports .NET Diffgrams
26
<ROOT xmlns:updg="urn:schemas-microsoft-com:xml-updategram"><updg:sync >
<updg:before></updg:before><updg:after>
<Employees><FirstName>Nancy</FirstName>
<LastName>Davolio</LastName></Employees>
</updg:after></updg:sync></ROOT>
Updategram in element-centric mode
27
<ROOT xmlns:updg="urn:schemas-microsoft-com:xml-updategram"><updg:sync >
<updg:before></updg:before><updg:after>
<Employees FirstName="Nancy" ><LastName>Davoliio</LastName>
</Employees></updg:after>
</updg:sync></ROOT>
Updategram in mixed mode
28
xml bulk loader
• Web Release 1 includes XML bulk load facility– OpenXML requires too much memory for bulk load of large
XML documents– Bulk load exposed through separate COM object– Bulk load analogous to import through DTS or BCP
29
set objBL = CreateObject("SQLXMLBulkLoad.SQLXMLBulkLoad")objBL.ConnectionString =
"provider=SQLOLEDB;uid=sa;pwd=;database=pubs"
objBL.ErrorLogFile = "c:\error.log"objBL.Execute "c:\SampleSchema.xml", "c:\SampleData.xml"set objBL=Nothing
XML Bulk Load through the bulk load class
30
sqlclient, datasets and diffgrams
• Three integration points in native .NET libraries– SQLISAPI program can be used directly as a web endpoint– SqlClient data provider exposes "FOR XML" results using
SqlCommand.ExecuteXmlReader– Resulting XmlReader can be used for DataSet input
producing multiple DataTables– DataSets can export two variations of Diffgram format
Entire DataSet and changes
Only changed rows
31
SqlCommand cmd = new SqlCommand("select * from authors for xml auto",new SqlConnection("server=localhost;uid=sa;database=pubs"));
cmd.Connection.Open();
XmlTextReader rdr;rdr = (XmlTextReader)cmd.ExecuteXmlReader();
DataSet ds = new DataSet();ds.ReadXml(rdr, XmlReadMode.Fragment);
Using SQLXML through a ExecuteXmlReader
32
DataSet ds = new DataSet();XmlTextReader rdr = new XmlTextReader(
"http://localhost/northwind/template/modeauto1.xml");
ds.ReadXml(rdr);
Using SQL Server 2000 ISAPI application
33
SqlDataAdapter da = new SqlDataAdapter("select * from sample","server=localhost;uid=sa;database=pubs");
DataSet dsAll = new DataSet("all");DataSet dsChanges = new DataSet("changes");
da.Fill(dsAll, "sample");DataTable t = dsAll.Tables["sample"];
DataRow r = t.NewRow();r[0] = 4;r[1] = "Shemp";t.Rows.Add(r);
t.Rows[1]["name"] = "CurleyJoe";t.Rows[2].Delete();
dsAll.WriteXml("c:\\all.xml", XmlWriteMode.DiffGram);dsChanges = dsAll.GetChanges();dsChanges.WriteXml("c:\\changes.xml", XmlWriteMode.DiffGram);
Producing DiffGrams
34
Microsoft.Data.SqlXml
• Web Release 2 includes SQLOLEDB-XML functionality– SqlXml data provider exposes Command, Parameter and
DataAdapter– All SQLOLEDB 2.6 and above functionality supported
MSSQLXML and XPath queries
XSL transforms
– DataAdapter enables integration with DataSet and DiffGrams
35
Using SqlXmlCommand and XSD mapping schema
void doXPath(){Stream strm;SqlXmlCommand cmd = new SqlXmlCommand("provider=sqloledb;uid=sa;database=northwind");cmd.CommandText = "x:Employee[@EID='1']";cmd.CommandType = SqlXmlCommandType.XPath;cmd.RootTag = "ROOT";cmd.Namespaces = "xmlns:x='urn:myschema:Employees'";cmd.SchemaPath = "c:\\xml_mappings\\MySchemaNS.xml";strm = cmd.ExecuteStream();StreamReader sw = new StreamReader(strm);Console.WriteLine(sw.ReadToEnd());}
36
Using SqlXmlCommand and XSD mapping schema
<!-- this mapping schema and previous code --><schema xmlns="http://www.w3.org/2001/XMLSchema"
xmlns:msdata="urn:schemas-microsoft-com:mapping-schema"xmlns:emp="urn:myschema:Employees"targetNamespace="urn:myschema:Employees">
<complexType name="EmployeeType"><attribute name="EID" msdata:field="EmployeeID" type="ID"/><attribute name="FName" msdata:field="FirstName" type="string"/><attribute name="LName" msdata:field="LastName"/>
</complexType><element name="Employee" type="emp:EmployeeType"
msdata:relation="Employees"/></schema>
<!-- produces this output --><?xml version="1.0" encoding="utf-8" ?><ROOT><y0:Employee xmlns:y0="urn:myschema:Employees"
LName="Davolio" EID="1" FName="Nancy"/></ROOT>
37
Using XSLT and SqlXmlCommand output
Stream strm;SqlXmlCommand cmd = new SqlXmlCommand(NorthwindConnString);cmd.CommandText ="select FirstName, LastName from Employees For XML Auto";
cmd.RootTag = "root";// get output Streamstrm = cmd.ExecuteStream();
// line up transform and documentXmlTextReader reader = new XmlTextReader(strm);XPathDocument xd = new XPathDocument(reader, XmlSpace.Preserve);XslTransform xslt = new XslTransform();xslt.Load("c:\\xml_mappings\\MyXSL.xsl", null);XmlTextWriter writer = new XmlTextWriter(
"c:\\xml_mappings\\xslt_output.html",System.Text.Encoding.UTF8);
// transform stream outputxslt.Transform(xd, null, writer);
38
summary
• SQL Server 2000 includes native XML support• SQLOLEDB provider contains additional support• ISAPI DLL exposes SQL/XML as HTTP endpoints• Updategrams and Bulk Load added in Web Release 1• Native .NET support in Web Release 2