read, store and create xml and json

Read, Store and Create XML and JSONOUGN Spring Seminar 10-12 March 2016

Kim Berg HansenSenior Consultant

Read, Store and Create XML and JSON2 05/03/2023

About me

• Danish geek• SQL & PL/SQL developer since 2000• Developer at Trivadis AG since 2016

http://www.trivadis.dk• Oracle Certified Expert in SQL• Oracle ACE• Blogger at http://www.kibeha.dk• SQL quizmaster at

http://plsqlchallenge.oracle.com• Likes to cook• Reads sci-fi• Chairman of local chapter of

Danish Beer Enthusiasts

http://www.trivadis.dk/

http://www.kibeha.dk/

http://plsqlchallenge.oracle.com/


About Trivadis

Trivadis is a market leader in IT consulting, system integration, solution engineeringand the provision of IT services focusing on and technologies in Switzerland, Germany, Austria and Denmark.We offer our services in the following strategic business fields:

Trivadis Services takes over the interacting operation of your IT systems.

O P E R A T I O N


COPENHAGEN

MUNICH

LAUSANNEBERN

ZURICHBRUGG

GENEVA

HAMBURG

DÜSSELDORF

FRANKFURT

STUTTGART

FREIBURG

BASEL

VIENNA

With over 600 specialists and IT experts in your region

14 Trivadis branches and more than600 employees

260 Service Level Agreements

Over 4,000 training participants

Research and development budget:EUR 5.0 million

Financially self-supporting and sustainably profitable

Experience from more than 1,900 projects per year at over 800customers

Agenda for XML and JSON


1. Retrieval

SOAP, REST, APEX_WEB_SERVICE, HttpUriType2. Storage

XMLType, CLOB, Object relational3. Query

XMLTABLE, XMLQUERY, JSON_TABLE, JSON_QUERY, JSON dot notation4. Creation

XML* functions, XMLTYPE constructor, DbUriType, APEX_JSON


Requisites

begin dbms_network_acl_admin.create_acl ( acl => 'google_apis_acl.xml' , description => 'Demo ACL for Google apis' , principal => 'SCOTT' , is_grant => true , privilege => 'connect' ); dbms_network_acl_admin.add_privilege( acl => 'google_apis_acl.xml' , principal => 'APEX_050000' , is_grant => true , privilege => 'connect' ); dbms_network_acl_admin.assign_acl ( acl => 'google_apis_acl.xml' , host => '*.googleapis.com' ); dbms_network_acl_admin.assign_acl ( acl => 'google_apis_acl.xml' , host => '*.cdyne.com' ); commit;end;/

As privileged user (i.e. SYSTEM) create ACL and give network privileges


Retrieval


HTTP GET with HttpUriType

select httpuritype( 'maps.googleapis.com/maps/api/directions/xml' || '?'||'origin=' || utl_url.escape(convert( 'Endelavevej 55, DK-5500 Middelfart' , 'UTF8' )) || '&'||'destination=' || utl_url.escape(convert( 'Paul-Dessau-Strasse 6, D-22761 Hamburg' , 'UTF8' )) || '&'||'mode=driving'||'&'||'alternatives=true'||'&'||'units=metric' || '&'||'region=eu'||'&'||'language=en'||'&'||'sensor=false' ).getxml() directions from dual;

Retrieve XML directions from Google Maps API – No SSL (https) supported

<?xml version="1.0" encoding="UTF-8"?><DirectionsResponse> <status>OK</status> <route> <summary>E45 and A7</summary> <leg> <step> <travel_mode>DRIVING</travel_mode> <start_location> <lat>55.4892447</lat> <lng>9.7517085</lng> </start_location> <end_location> <lat>55.4878639</lat> <lng>9.7539612</lng> </end_location> <polyline><points>wvtqIesoz@k@j@k@iCh@i@tBsBVY`B_BpAkA</points> </polyline>

<duration> <value>65</value> <text>1 min</text> </duration>... <geocoded_waypoint> <geocoder_status>OK</geocoder_status> <type>street_address</type> <place_id>ChIJRUp9K2iWTEYRmzjwsDRYoto</place_id> </geocoded_waypoint> <geocoded_waypoint> <geocoder_status>OK</geocoder_status> <type>street_address</type> <partial_match>true</partial_match> <place_id>ChIJZ1x_bZWFsUcRx7t3-L0HwsI</place_id> </geocoded_waypoint></DirectionsResponse>

Retrieve XML directions from Google Maps API – XMLTYPE object output



select httpuritype( 'maps.googleapis.com/maps/api/directions/json' || '?'||'origin=' || utl_url.escape(convert( 'Endelavevej 55, DK-5500 Middelfart' , 'UTF8' )) || '&'||'destination=' || utl_url.escape(convert( 'Paul-Dessau-Strasse 6, D-22761 Hamburg' , 'UTF8' )) || '&'||'mode=driving'||'&'||'alternatives=true'||'&'||'units=metric' || '&'||'region=eu'||'&'||'language=en'||'&'||'sensor=false' ).getclob() directions from dual;

Retrieve JSON directions from Google Maps API – No SSL (https) supported



{ "geocoded_waypoints" : [ { "geocoder_status" : "OK", "place_id" : "ChIJRUp9K2iWTEYRmzjwsDRYoto", "types" : [ "street_address" ] }, { "geocoder_status" : "OK", "partial_match" : true, "place_id" : "ChIJZ1x_bZWFsUcRx7t3-L0HwsI", "types" : [ "street_address" ] } ], "routes" : [ { "bounds" : { "northeast" : { "lat" : 55.5465325,

"lng" : 9.964266 }, "southwest" : { "lat" : 53.5627269, "lng" : 9.3241038 } },... }, "summary" : "E45", "warnings" : [], "waypoint_order" : [] } ], "status" : "OK"}

Retrieve JSON directions from Google Maps API – CLOB output containing JSON


HTTP GET with APEX_WEB_SERVICE

declare directions clob; directions_xml xmltype;begin directions := apex_web_service.make_rest_request( p_url => 'http://maps.googleapis.com/maps/api/directions/xml' , p_http_method => 'GET' , p_parm_name => apex_util.string_to_table('origin:destination:mode:alternatives:units:region:language:sensor' ) , p_parm_value => apex_util.string_to_table('Endelavevej 55, DK-5500 Middelfart:Paul-Dessau-Strasse 6, D-22761 Hamburg:driving:true:metric:eu:en:false' ) ); dbms_output.put_line(substr(directions,1,4000)); directions_xml := xmltype(directions);end;/

Retrieve XML (or JSON) directions from Google Maps API – SSL (https) supported via WalletData always as CLOB whether XML or JSON – If XMLTYPE needed, use constructor


HTTP GET with APEX_WEB_SERVICE

declare parm_name wwv_flow_global.vc_arr2; parm_value wwv_flow_global.vc_arr2; directions clob;begin parm_name(1) := 'origin'; parm_value(1) := 'Endelavevej 55, DK-5500 Middelfart'; parm_name(2) := 'destination'; parm_value(2) := 'Paul-Dessau-Strasse 6, D-22761 Hamburg'; parm_name(3) := 'mode'; parm_value(3) := 'driving'; parm_name(4) := 'alternatives'; parm_value(4) := 'true'; parm_name(5) := 'units'; parm_value(5) := 'metric'; parm_name(6) := 'region'; parm_value(6) := 'eu'; parm_name(7) := 'language'; parm_value(7) := 'en'; parm_name(8) := 'sensor'; parm_value(8) := 'false'; directions := apex_web_service.make_rest_request( p_url => 'http://maps.googleapis.com/maps/api/directions/json' , p_http_method => 'GET' , p_parm_name => parm_name , p_parm_value => parm_value ); dbms_output.put_line(substr(directions,1,4000));end;/

Retrieve JSON (or XML) directions from Google Maps APIAlternative call syntax building parameter name/value pairs manually


HTTP SOAP Webservice with APEX_WEB_SERVICE

declare envelope clob; weather xmltype;begin select xmlroot( xmlelement( "soap:Envelope" , xmlattributes( 'http://www.w3.org/2001/XMLSchema-instance' as "xmlns:xsi" , 'http://www.w3.org/2001/XMLSchema' as "xmlns:xsd" , 'http://schemas.xmlsoap.org/soap/envelope/' as "xmlns:soap" ) , xmlelement( "soap:Body" , xmlelement( "GetCityWeatherByZIP" , xmlattributes('http://ws.cdyne.com/WeatherWS/' as "xmlns") , xmlelement("ZIP", '60611') ) ) ), version '1.0').getclobval() into envelope from dual; weather := apex_web_service.make_request( p_url => 'http://wsf.cdyne.com/WeatherWS/Weather.asmx' , p_action => 'http://ws.cdyne.com/WeatherWS/GetCityWeatherByZIP' , p_version => '1.1' , p_envelope => envelope ); dbms_output.put_line(weather.getclobval());end;/

Retrieve weather in Chicago, USA – Data always as CLOB even though contains XML


Further information on callouts

My presentation from ODTUG KScope14:

– External Data via HTTP, FTP and Web Services

– http://bit.ly/kibeha_http_ws_slides


Storage


XMLTYPE column

create table addresses ( addr_id integer primary key , address varchar2(4000) , geocoded xmltype) xmltype geocoded store as securefile binary xml;

insert into addresses values ( 1 , 'Endelavevej 55, DK-5500 Middelfart' , httpuritype( 'http://maps.googleapis.com/maps/api/geocode/xml?address='|| utl_url.escape('Endelavevej 55, DK-5500 Middelfart') ).getxml());

Column object type XMLTYPE

XMLTYPE column

select geocoded from addresses;

Column object type XMLTYPE

<?xml version="1.0" encoding="UTF-8"?><GeocodeResponse> <status>OK</status> <result> <type>street_address</type> <formatted_address>Endelavevej 55, 5500 Middelfart, Denmark</formatted_address> <address_component> <long_name>55</long_name> <short_name>55</short_name> <type>street_number</type> </address_component>... <place_id>ChIJRUp9K2iWTEYRmzjwsDRYoto</place_id> </result></GeocodeResponse>


CLOB column with JSON

create table addresses2 ( addr_id integer primary key , address varchar2(4000) , geocoded clob check(geocoded is json));

insert into addresses2 (addr_id, address) values (2, 'Endelavevej 55, DK-5500 Middelfart');

update addresses2 set geocoded = httpuritype( 'http://maps.googleapis.com/maps/api/geocode/json?address='|| utl_url.escape(address) ).getclob();

Any CLOB can contain JSON - Use IS JSON check constraint to be certain


CLOB column with JSON

select geocoded from addresses2;

Content like any other CLOB

{ "results" : [ { "address_components" : [ { "long_name" : "55", "short_name" : "55", "types" : [ "street_number" ] },.. "formatted_address" : "Endelavevej 55, 5500 Middelfart, Denmark", "geometry" : { "location" : { "lat" : 55.4892307, "lng" : 9.7519265..


XMLTYPE table

create table geocoded_addresses of xmltype xmltype store as securefile binary xml;

insert into geocoded_addresses values ( httpuritype( 'http://maps.googleapis.com/maps/api/geocode/xml?address='|| utl_url.escape('Endelavevej 55, DK-5500 Middelfart') ).getxml());

Object table of type XMLTYPE

XMLTYPE table

select sys_nc_rowinfo$ from geocoded_addresses;

Table now contains objects, not scalar columns

<?xml version="1.0" encoding="UTF-8"?><GeocodeResponse> <status>OK</status> <result> <type>street_address</type> <formatted_address>Endelavevej 55, 5500 Middelfart, Denmark</formatted_address> <address_component> <long_name>55</long_name> <short_name>55</short_name> <type>street_number</type> </address_component>... <place_id>ChIJRUp9K2iWTEYRmzjwsDRYoto</place_id> </result></GeocodeResponse>

XMLSchema

begin dbms_xmlschema.registerschema( schemaurl=>'/nsroot/demo/note/' , schemadoc=>'<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"><xs:element name="note"> <xs:complexType> <xs:sequence> <xs:element name="to" type="xs:string"/> <xs:element name="from" type="xs:string"/> <xs:element name="heading" type="xs:string"/> <xs:element name="body" type="xs:string"/> </xs:sequence> </xs:complexType></xs:element></xs:schema>' , local=>true , gentypes=>true , genbean=>false , gentables=>false );end;/

Schema defines structure of XML – schema is registered under URL “path” in repository

XMLSchema

create table notes of xmltype xmltype store as object relational xmlschema "/nsroot/demo/note" element "note";

-- Raises error: ORA-30937: No schema definition for 'something' (namespace '') in parent '/'insert into notes values ( xmltype('<something>Not valid schema</something>'));

-- Works, because XML fits schema definitioninsert into notes values ( xmltype('<note><to>Jack</to><from>Jill</from><heading>Be my Valentine?</heading><body>Let us meet at Luigis Restaurant</body></note>') );

XMLTYPE table stored as object relational, not binary XMLXMLSCHEMA keyword may be used for binary XML too and enforce XML structure

XMLSchema stored as object

select sys_nc_rowinfo$ from notes;

XML re-assembled from object relational dataWhitespace between elements no longer as original

<note> <to>Jack</to> <from>Jill</from> <heading>Be my Valentine?</heading> <body>Let us meet at Luigis Restaurant</body></note>

select dbms_metadata.get_ddl('TYPE', 'note928_T') ddl from dual;

-- Output:

CREATE OR REPLACE EDITIONABLE TYPE "SCOTT"."note928_T" AS OBJECT ( "SYS_XDBPD$" "XDB"."XDB$RAW_LIST_T" , "to" VARCHAR2(32767 CHAR) , "from" VARCHAR2(32767 CHAR) , "heading" VARCHAR2(32767 CHAR) , "body" VARCHAR2(32767 CHAR)) FINAL INSTANTIABLE;

CREATE TABLE autocreated object type named <xml name><sequence>_T



select sys_nc00008$, sys_nc00009$, sys_nc00010$, sys_nc00011$ from notes;

SYS_NC00008$ SYS_NC00009$ SYS_NC00010$ SYS_NC00011$--------------- --------------- -------------------- -----------------------------------Jack Jill Be my Valentine? Let us meet at Luigis Restaurant

Object type table contains hidden columns with relational data

XMLSchema with annotations

dbms_xmlschema.registerschema( schemaurl=>'/nsroot/demo/note/' , schemadoc=>'<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xdb="http://xmlns.oracle.com/xdb"> <xs:element name="note" type="NoteType" xdb:defaultTable="NOTES"/> <xs:complexType name="NoteType" xdb:SQLType="NOTE_T"> <xs:sequence> <xs:element name="to" type="HeaderType" xdb:SQLName="TO"/> <xs:element name="from" type="HeaderType" xdb:SQLName="FROM"/> <xs:element name="heading" type="HeaderType" xdb:SQLName="HEADING"/> <xs:element name="body" type="BodyType" xdb:SQLName="BODY"/> </xs:sequence> </xs:complexType> <xs:simpleType name="HeaderType"> <xs:restriction base="xs:string"><xs:minLength value="0"/><xs:maxLength value="30"/></xs:restriction> </xs:simpleType> <xs:simpleType name="BodyType"> <xs:restriction base="xs:string"><xs:minLength value="0"/><xs:maxLength value="4000"/></xs:restriction> </xs:simpleType></xs:schema>' , local=>true , gentypes=>true , genbean=>false , gentables=>true );

Put xdb:*** annotations in XML schema along with length restrictionsObject type and table generating will use the annotated names

insert into notes values ( xmltype('<note><to>Jack</to><from>Jill</from><heading>Be my Valentine?</heading><body>Let us meet at Luigis Restaurant</body></note>') );

select sys_nc_rowinfo$ from notes;

<note> <to>Jack</to> <from>Jill</from> <heading>Be my Valentine?</heading> <body>Let us meet at Luigis Restaurant</body></note>

NOTES table here created by “gentables=>true”Inserting and selecting data same result as before



select dbms_metadata.get_ddl('TYPE', 'NOTE_T') ddl from dual;

-- Output:

CREATE OR REPLACE EDITIONABLE TYPE "SCOTT"."NOTE_T" AS OBJECT ( "SYS_XDBPD$" "XDB"."XDB$RAW_LIST_T" , "TO" VARCHAR2(30 CHAR) , "FROM" VARCHAR2(30 CHAR) , "HEADING" VARCHAR2(30 CHAR) , "BODY" VARCHAR2(4000 CHAR)) NOT FINAL INSTANTIABLE;

Register schema autocreated object type named as specified in annotationsTable still contains invisible SYS named columns, but now with better datatypes


Query

XMLQUERY

select xmlquery( '/GeocodeResponse/result/formatted_address' passing a.geocoded returning content ) formatted from addresses a;

FORMATTED--------------------------------------------------------------------------------<formatted_address>Endelavevej 55, 5500 Middelfart, Denmark</formatted_address>

XMLQUERY returns XML snippet


XMLCAST

select xmlcast( xmlquery( '/GeocodeResponse/result/formatted_address' passing a.geocoded returning content ) as varchar2(50) ) formatted from addresses a;

FORMATTED-----------------------------------------------------Endelavevej 55, 5500 Middelfart, Denmark

XMLCAST is one way to turn the snippet into only the content


XMLTABLE

select geo.* from addresses a , xmltable( '*' passing a.geocoded columns status varchar2(20) path '/GeocodeResponse/status' , lat number path '/GeocodeResponse/result/geometry/location/lat' , lng number path '/GeocodeResponse/result/geometry/location/lng' , formatted_address varchar2(50) path '/GeocodeResponse/result/formatted_address' ) geo;

STATUS LAT LNG FORMATTED_ADDRESS-------------------- ---------- ---------- --------------------------------------------------OK 55.4892307 9.7519265 Endelavevej 55, 5500 Middelfart, Denmark

Pass XMLTABLE function some XML and define columns with XPath expressions


XMLTABLE

select geo.* from addresses a , xmltable( '/GeocodeResponse/result' passing a.geocoded columns lat number path 'geometry/location/lat' , lng number path 'geometry/location/lng' , formatted_address varchar2(50) path 'formatted_address' ) geo;

LAT LNG FORMATTED_ADDRESS---------- ---------- --------------------------------------------------55.4892307 9.7519265 Endelavevej 55, 5500 Middelfart, Denmark

Can tell XMLTABLE to operate only on a sub-node


XMLTABLE

select geo.* from addresses a , xmltable( '/GeocodeResponse/result' passing a.geocoded columns formatted_address varchar2(50) path 'formatted_address' , street_number_name varchar2(10) path 'address_component/long_name' ) geo;

ERROR at line 3:ORA-19279: XPTY0004 - XQuery dynamic type mismatch: expected singleton sequence - got multi-item sequence

Multiple elements “address_component/long_name” in XML => error

XMLTABLE

select geo.* from addresses a , xmltable( '/GeocodeResponse/result' passing a.geocoded columns adr_components xmltype path 'address_component' ) geo;

Column can be XMLTYPE to allow us to inspect why the error

<address_component> <long_name>55</long_name> <short_name>55</short_name> <type>street_number</type></address_component><address_component> <long_name>Endelavevej</long_name> <short_name>Endelavevej</short_name> <type>route</type></address_component><address_component> <long_name>Middelfart</long_name> <short_name>Middelfart</short_name> <type>locality</type> <type>political</type></address_component><address_component> <long_name>Middelfart Municipality</long_name> <short_name>Middelfart Municipality</short_name> <type>administrative_area_level_2</type> <type>political</type>..


XMLTABLE

select geo.formatted_address , com.* from addresses a , xmltable( '/GeocodeResponse/result' passing a.geocoded columns formatted_address varchar2(50) path 'formatted_address' , adr_components xmltype path 'address_component' ) geo , xmltable( '/address_component' passing geo.adr_components columns type varchar2(30) path 'type' , long_name varchar2(30) path 'long_name' ) com;

ORA-19279: XPTY0004 - XQuery dynamic type mismatch: expected singleton sequence - got multi-item sequence

Chain two XMLTABLE calls – except “type” still has too many


XMLTABLE

select geo.formatted_address , com.* from addresses a , xmltable( '/GeocodeResponse/result' passing a.geocoded columns formatted_address varchar2(50) path 'formatted_address' , adr_components xmltype path 'address_component' ) geo , xmltable( '/address_component' passing geo.adr_components columns type varchar2(30) path 'type[1]' , long_name varchar2(30) path 'long_name' ) com;

type[1] allows us to specify we just want the first if there are multiple


XMLTABLE

FORMATTED_ADDRESS TYPE LONG_NAME------------------------------------------ ------------------------------ -------------------------Endelavevej 55, 5500 Middelfart, Denmark street_number 55Endelavevej 55, 5500 Middelfart, Denmark route EndelavevejEndelavevej 55, 5500 Middelfart, Denmark locality MiddelfartEndelavevej 55, 5500 Middelfart, Denmark administrative_area_level_2 Middelfart MunicipalityEndelavevej 55, 5500 Middelfart, Denmark country DenmarkEndelavevej 55, 5500 Middelfart, Denmark postal_code 5500

Output of previous slide


XMLTABLE

select geo.* from addresses a , xmltable( '/GeocodeResponse/result' passing a.geocoded columns street_number_name varchar2(10) path 'address_component[type="street_number"]/long_name' , street_name varchar2(20) path 'address_component[type="route"]/long_name' , city_name varchar2(20) path 'address_component[type="locality"]/long_name' , zip_name varchar2(10) path 'address_component[type="postal_code"]/long_name' , country_name varchar2(20) path 'address_component[type="country"]/long_name' ) geo;

STREET_NUM STREET_NAME CITY_NAME ZIP_NAME COUNTRY_NAME---------- -------------------- -------------------- ---------- --------------------55 Endelavevej Middelfart 5500 Denmark

[ ] in XPath expressions also allows syntax to query for specific values


DIRECTIONS EXAMPLE - 1select route.routename , step.instructions , to_char(to_date(to_char(step.seconds,'TM9'), 'SSSSS'), 'HH24:MI:SS') steptime , step.meters/1000 stepkmfrom xmltable( '/DirectionsResponse' passing httpuritype( 'maps.googleapis.com/maps/api/directions/xml' || '?'||'origin=' || utl_url.escape(convert( 'Endelavevej 55, DK-5500 Middelfart' , 'UTF8' )) || '&'||'destination=' || utl_url.escape(convert( 'Rue Marterey 5, CH-1005 Lausanne' , 'UTF8' )) || '&'||'mode=driving'||'&'||'alternatives=true'||'&'||'units=metric' || '&'||'region=eu'||'&'||'language=en'||'&'||'sensor=false' ).getxml() columns response xmltype path '/') directions,...

DIRECTIONS EXAMPLE - 2...xmltable( 'if (fn:empty(/DirectionsResponse/route)) then <route><leg></leg></route> else /DirectionsResponse/route' passing directions.response columns routenum for ordinality , routename varchar2(100) path 'summary' , routexml xmltype path '/') route, xmltable( 'for $l in /route/leg return $l' passing route.routexml columns legnum for ordinality , seconds number path 'duration/value' , meters number path 'distance/value' , startaddr varchar2(100) path 'start_address' , endaddr varchar2(100) path 'end_address' , legxml xmltype path '/') leg,...


DIRECTIONS EXAMPLE - 3...xmltable( 'for $s in /leg/step return $s' passing leg.legxml columns stepnum for ordinality , seconds number path 'duration/value' , meters number path 'distance/value' , instructions varchar2(400) path 'html_instructions') steporder by route.routenum , leg.legnum , step.stepnum;

DIRECTIONS EXAMPLE - OutputROUTENAME INSTRUCTIONS STEPTIME STEPKM---------- ------------------------------------------------------------ -------- ----------A7 and A5 Head northwest on Endelavevej 00:01:05 .309A7 and A5 Turn right to stay on Endelavevej 00:00:27 .124A7 and A5 Turn left onto Bornholmsvej 00:00:42 .396A7 and A5 Turn left onto Vandværksvej 00:04:36 3.274A7 and A5 At the roundabout, take the 1st exit onto Jyllands 00:00:26 .242 vej

A7 and A5 At the roundabout, take the 1st exit and stay on J 00:00:33 .35 yllandsvej

A7 and A5 At the roundabout, take the 2nd exit onto the E20< 00:00:38 .546 /b> ramp

A7 and A5 Merge onto E20 00:04:10 7.229A7 and A5 Keep left at the fork to stay on E20, follow s 00:08:57 15.359 igns for E45s/Flensborg/Esbjerg/Kold ing

A7 and A5 Keep left at the fork to continue on E45, foll 00:47:31 83.159 ow signs for Kolding<div style="font-size:0.9em">Ente ring Germany</div>

A7 and A5 Continue onto A7/E45 01:35:25 155.339A7 and A5 Keep left at the fork to stay on A7, follow si 00:09:33 15.067 gns for HH. Othmarschen

A7 and A5 Keep left at the fork to stay on A7 00:03:39 6.845...


JSON_TABLE

select geo.* from addresses2 a , json_table( a.geocoded , '$' columns ( status varchar2(10) path '$.status' , nested path '$.results[*]' columns ( lat number path '$.geometry.location.lat' , lng number path '$.geometry.location.lng' , nested path '$.address_components[*]' columns ( long_name varchar2(30) path '$.long_name' , nested path '$.types[*]' columns ( component_type varchar2(30) path '$' ) ) ) ) ) geo;

Similar but slightly different syntax as XMLTABLESupports NESTED PATH as alternative to chaining XMLTABLE


JSON_TABLE

STATUS LAT LNG LONG_NAME COMPONENT_TYPE---------- ---------- ---------- ------------------------- ------------------------------OK 55.4892307 9.7519265 55 street_numberOK 55.4892307 9.7519265 Endelavevej routeOK 55.4892307 9.7519265 Middelfart localityOK 55.4892307 9.7519265 Middelfart politicalOK 55.4892307 9.7519265 Middelfart Municipality administrative_area_level_2OK 55.4892307 9.7519265 Middelfart Municipality politicalOK 55.4892307 9.7519265 Denmark countryOK 55.4892307 9.7519265 Denmark politicalOK 55.4892307 9.7519265 5500 postal_code



JSON short-cut dot-notation

select a.geocoded.status , a.geocoded.results.formatted_address , a.geocoded.results.address_components.long_name from addresses2 a;

STATUS RESULTS RESULTS------ -------------------------------------------------- --------------------------------------------------OK Endelavevej 55, 5500 Middelfart, Denmark ["55","Endelavevej","Middelfart","Middelfart Munic ipality","Denmark","5500"]

Requires table alias – Datatype will just be generic varchar2Can return JSON arrays for multiple elements – Column names quirky


JSON short-cut dot-notation

select a.addr_id , a.geocoded.STATUS , a.geocoded.RESULTS.formatted_address from addresses2 a;

ADDR_ID STATUS RESULTS---------- ------ -------------------------------------------------- 2

Dot-notation is case sensitive – even though looks like regular SQL


JSON short-cut dot-notation with JSON_TABLE

select a.geocoded.results.formatted_address as formatted , geo.* from addresses2 a , json_table( a.geocoded.results.address_components , '$[*]' columns ( long_name varchar2(30) path '$.long_name' , nested path '$.types[*]' columns ( component_type varchar2(30) path '$' ) ) ) geo;

FORMATTED LONG_NAME COMPONENT_TYPE------------------------------------------ ------------------------- ------------------------------Endelavevej 55, 5500 Middelfart, Denmark 55 street_numberEndelavevej 55, 5500 Middelfart, Denmark Endelavevej routeEndelavevej 55, 5500 Middelfart, Denmark Middelfart localityEndelavevej 55, 5500 Middelfart, Denmark Middelfart politicalEndelavevej 55, 5500 Middelfart, Denmark Middelfart Municipality administrative_area_level_2Endelavevej 55, 5500 Middelfart, Denmark Middelfart Municipality politicalEndelavevej 55, 5500 Middelfart, Denmark Denmark countryEndelavevej 55, 5500 Middelfart, Denmark Denmark politicalEndelavevej 55, 5500 Middelfart, Denmark 5500 postal_code

Combine JSON dot-notation and JSON_TABLE


JSON_TABLE vs. XMLTABLE - Comparison of syntax

JSON_TABLE

With underscore

CLOB with IS JSON check

Arg 1: PASSING object

Arg 2: Path to data

Arg 3: Column defs/paths in ( )

Simple JSON dot path with $ = root

Support for NESTED JSON arrays

Cannot query paths dependent on values

XMLTABLE

Without underscore

XMLTYPE object type

Arg 1: Path / XQuery exp to data

Arg 2: PASSING object

Arg 3: Column defs/paths – no ( )

Full XQuery syntax with / = root

Need multiple successive XMLTABLE

XPath expressions allow query like address_component[type="country"]/...


JSON_TABLE

select * from ( select geo.* from addresses2 a, json_table( a.geocoded, '$' columns ( nested path '$.results[*]' columns ( lat number path '$.geometry.location.lat' , lng number path '$.geometry.location.lng' , nested path '$.address_components[*]' columns ( long_name varchar2(15) path '$.long_name' , nested path '$.types[*]' columns ( component_type varchar2(15) path '$' ) ) ) ) ) geo) pivot ( max(long_name) name for component_type in ( 'street_number' as street_number , 'route' as street , 'locality' as city , 'postal_code' as zip , 'country' as country ) );

LAT LNG STREET_NUMBER_N STREET_NAME CITY_NAME ZIP_NAME COUNTRY_NAME---------- ---------- --------------- --------------- --------------- --------------- ---------------55.4892307 9.7519265 55 Endelavevej Middelfart 5500 Denmark

No support for “value lookup” – need SQL to handle that

JSON_TABLE and APEX_JSON

select a.geocoded, apex_json.to_xmltype(a.geocoded) xmlgeo from addresses2 a;GEOCODED XMLGEO------------------------------------------------------------ ------------------------------------------------------------{ <?xml version="1.0" encoding="WINDOWS-1252"?> "results" : [ <json> { <results> "address_components" : [ <row> { <address_components> "long_name" : "55", <row> "short_name" : "55", <long_name>55</long_name> "types" : [ "street_number" ] <short_name>55</short_name> }, <types> { <row>street_number</row> "long_name" : "Endelavevej", </types> "short_name" : "Endelavevej", </row> "types" : [ "route" ] <row> }, <long_name>Endelavevej</long_name> { <short_name>Endelavevej</short_name> "long_name" : "Middelfart", <types> "short_name" : "Middelfart", <row>route</row>... ...

Alternative to PIVOT we can turn JSON into XML with APEX_JSON


JSON_TABLE and APEX_JSON

select geo.* from addresses2 a , xmltable( '/json/results/row' passing apex_json.to_xmltype(a.geocoded) columns street_number_name varchar2(10) path 'address_components/row[types/row="street_number"]/long_name' , street_name varchar2(20) path 'address_components/row[types/row="route"]/long_name' , city_name varchar2(20) path 'address_components/row[types/row="locality"]/long_name' , zip_name varchar2(10) path 'address_components/row[types/row="postal_code"]/long_name' , country_name varchar2(20) path 'address_components/row[types/row="country"]/long_name' ) geo;

STREET_NUM STREET_NAME CITY_NAME ZIP_NAME COUNTRY_NAME---------- -------------------- -------------------- ---------- --------------------55 Endelavevej Middelfart 5500 Denmark

Then we can use XMLTABLE with greater syntax on the converted JSON


Creation

XMLROOT, XMLELEMENT

select xmlroot( xmlelement("DeptName", d.dname) , version '1.0' ) department from scott.dept d where d.deptno = 30;

DEPARTMENT--------------------------------------------------------------------------------<?xml version="1.0"?><DeptName>SALES</DeptName>

XMLROOT creates XML header with version infoXMLELEMENT creates an element with start & end tag and content between

XMLELEMENT

select xmlroot( xmlelement( "Department" , xmlelement("DeptNo", d.deptno) , xmlelement("DeptName", d.dname) ) , version '1.0' ) department from scott.dept d where d.deptno = 30;

Nesting XMLELEMENT – multiple XMLELEMENT within one XMLELEMENT

DEPARTMENT-------------------------------<?xml version="1.0"?><Department> <DeptNo>30</DeptNo> <DeptName>SALES</DeptName></Department>

XMLFOREST

select xmlroot( xmlelement( "Department" , xmlforest( d.deptno as "DeptNo" , d.dname as "DeptName" ) ) , version '1.0' ) department from scott.dept d where d.deptno = 30;

Shortcut for multiple XMLELEMENT

DEPARTMENT---------------------------------------<?xml version="1.0"?><Department> <DeptNo>30</DeptNo> <DeptName>SALES</DeptName></Department>

Multiple rows

select xmlroot( xmlelement( "Department" , xmlforest( d.deptno as "DeptNo" , d.dname as "DeptName" ) ) , version '1.0' ) department from scott.dept d;

Here we get one XML object for each department – one in each row of output

DEPARTMENT-------------------------------------<?xml version="1.0"?><Department> <DeptNo>10</DeptNo> <DeptName>ACCOUNTING</DeptName></Department>

<?xml version="1.0"?><Department> <DeptNo>20</DeptNo> <DeptName>RESEARCH</DeptName></Department>

<?xml version="1.0"?><Department> <DeptNo>30</DeptNo> <DeptName>SALES</DeptName></Department>

<?xml version="1.0"?><Department> <DeptNo>40</DeptNo> <DeptName>OPERATIONS</DeptName></Department>

XMLAGG

select xmlroot( xmlelement( "Departments" , xmlagg( xmlelement( "Department" , xmlforest( d.deptno as "DeptNo" , d.dname as "DeptName" ) ) order by d.deptno ) ) , version '1.0' ) department from scott.dept d;

XMLAGG aggregates “Department” elements within “Departments” element

DEPARTMENT--------------------------------------<?xml version="1.0"?><Departments> <Department> <DeptNo>10</DeptNo> <DeptName>ACCOUNTING</DeptName> </Department> <Department> <DeptNo>20</DeptNo> <DeptName>RESEARCH</DeptName> </Department> <Department> <DeptNo>30</DeptNo> <DeptName>SALES</DeptName> </Department> <Department> <DeptNo>40</DeptNo> <DeptName>OPERATIONS</DeptName> </Department></Departments>

Nested XMLAGG

select xmlroot( xmlelement( "Departments" , xmlagg( xmlelement( "Department" , xmlforest( d.deptno as "DeptNo" , d.dname as "DeptName" , ( select xmlagg( xmlelement( "Employee" , xmlforest( e.empno as "EmpNo“ , e.ename as "EmpName" ) ) order by e.empno ) from scott.emp e where e.deptno = d.deptno ) as "Employees" ) ) order by d.deptno ) ) , version '1.0' ) department from scott.dept d where d.deptno = 10;

XMLAGG nested as subquery within XMLAGG

<?xml version="1.0"?><Departments> <Department> <DeptNo>10</DeptNo> <DeptName>ACCOUNTING</DeptName> <Employees> <Employee> <EmpNo>7782</EmpNo> <EmpName>CLARK</EmpName> </Employee> <Employee> <EmpNo>7839</EmpNo> <EmpName>KING</EmpName> </Employee> <Employee> <EmpNo>7934</EmpNo> <EmpName>MILLER</EmpName> </Employee> </Employees> </Department></Departments>

Nested XMLAGG

select xmlroot( xmlelement( "Departments" , xmlagg( xmlelement( "Department" , xmlforest( d.deptno as "DeptNo" , max(d.dname) as "DeptName" , xmlagg( xmlelement( "Employee" , xmlforest( e.empno as "EmpNo" , e.ename as "EmpName" ) ) order by e.empno ) as "Employees" ) ) order by d.deptno ) ) , version '1.0' ) department from scott.dept d join scott.emp e on e.deptno = d.deptno where d.deptno = 10 group by d.deptno;

Join and GROUP BY – Inner XMLAGG is “group” aggregate – Outer XMLAGG is “total”

<?xml version="1.0"?><Departments> <Department> <DeptNo>10</DeptNo> <DeptName>ACCOUNTING</DeptName> <Employees> <Employee> <EmpNo>7782</EmpNo> <EmpName>CLARK</EmpName> </Employee> <Employee> <EmpNo>7839</EmpNo> <EmpName>KING</EmpName> </Employee> <Employee> <EmpNo>7934</EmpNo> <EmpName>MILLER</EmpName> </Employee> </Employees> </Department></Departments>

XMLATTRIBUTES

select xmlroot( xmlelement( "Departments" , xmlagg( xmlelement( "Department" , xmlattributes(d.deptno as "DeptNo") , xmlforest( max(d.dname) as "DeptName" , xmlagg( xmlelement( "Employee" , xmlattributes(e.empno as "EmpNo") , xmlelement("EmpName", e.ename) ) order by e.empno ) as "Employees" ) ) order by d.deptno ) ) , version '1.0' ) department from scott.dept d join scott.emp e on e.deptno = d.deptno where d.deptno = 10 group by d.deptno;

Typically some elements are turned into attributes – makes more “terse” XML

<?xml version="1.0"?><Departments> <Department DeptNo="10"> <DeptName>ACCOUNTING</DeptName> <Employees> <Employee EmpNo="7782"> <EmpName>CLARK</EmpName> </Employee> <Employee EmpNo="7839"> <EmpName>KING</EmpName> </Employee> <Employee EmpNo="7934"> <EmpName>MILLER</EmpName> </Employee> </Employees> </Department></Departments>


Object types

create type emp_t as object (empno number, empname varchar2(10));

create type emp_tt as table of emp_t;

create type dept_t as object ( deptno number , deptname varchar2(15) , employees emp_tt);

create type dept_tt as table of dept_t;

create type depts_t as object (departments dept_tt);

Created object types can be basis for XML

XMLTYPE Constructor for Object Types

select xmltype( depts_t( cast( collect( dept_t( d.deptno , max(d.dname) , cast( collect( emp_t(e.empno, e.ename) ) as emp_tt ) ) ) as dept_tt ) ) ) department from scott.dept d join scott.emp e on e.deptno = d.deptno where d.deptno = 10 group by d.deptno;

Object types can be passed as argument to XMLTYPE constructor

<DEPTS_T> <DEPARTMENTS> <DEPT_T> <DEPTNO>10</DEPTNO> <DEPTNAME>ACCOUNTING</DEPTNAME> <EMPLOYEES> <EMP_T> <EMPNO>7782</EMPNO> <EMPNAME>CLARK</EMPNAME> </EMP_T> <EMP_T> <EMPNO>7934</EMPNO> <EMPNAME>MILLER</EMPNAME> </EMP_T> <EMP_T> <EMPNO>7839</EMPNO> <EMPNAME>KING</EMPNAME> </EMP_T> </EMPLOYEES> </DEPT_T> </DEPARTMENTS></DEPTS_T>

XMLTYPE Constructor for REF CURSOR

select xmltype( cursor( select d.deptno "DeptNo" , d.dname "DeptName" from scott.dept d where d.deptno = 10 )) department from dual;

Cursor variables can be passed to XMLTYPE constructor

<?xml version="1.0"?><ROWSET> <ROW> <DeptNo>10</DeptNo> <DeptName>ACCOUNTING</DeptName> </ROW></ROWSET>

Nested REF CURSOR

select xmltype( cursor( select d.deptno "DeptNo" , d.dname "DeptName" , cursor( select e.empno "EmpNo" , e.ename "EmpName" from scott.emp e where e.deptno = d.deptno ) "Employees" from scott.dept d where d.deptno = 10 )) department from dual;

Cursor variables can contain nested cursor variables

<?xml version="1.0"?><ROWSET> <ROW> <DeptNo>10</DeptNo> <DeptName>ACCOUNTING</DeptName> <Employees> <Employees_ROW> <EmpNo>7782</EmpNo> <EmpName>CLARK</EmpName> </Employees_ROW> <Employees_ROW> <EmpNo>7839</EmpNo> <EmpName>KING</EmpName> </Employees_ROW> <Employees_ROW> <EmpNo>7934</EmpNo> <EmpName>MILLER</EmpName> </Employees_ROW> </Employees> </ROW></ROWSET>

DbUriType

select dburitype( '/SCOTT/DEPT' ).getxml() departments from dual;

DbUriType accesses schema/table via XMLDB

<?xml version="1.0"?><DEPT> <ROW> <DEPTNO>10</DEPTNO> <DNAME>ACCOUNTING</DNAME> <LOC>NEW YORK</LOC> </ROW> <ROW> <DEPTNO>20</DEPTNO> <DNAME>RESEARCH</DNAME> <LOC>DALLAS</LOC> </ROW> <ROW> <DEPTNO>30</DEPTNO> <DNAME>SALES</DNAME> <LOC>CHICAGO</LOC> </ROW> <ROW> <DEPTNO>40</DEPTNO> <DNAME>OPERATIONS</DNAME> <LOC>BOSTON</LOC> </ROW></DEPT>

DbUriType

select dburitype( '/SCOTT/EMP/ROW[ENAME="KING"]' ).getxml() employees from dual;

Data can be queried using XPath expressions

<?xml version="1.0"?><ROW> <EMPNO>7839</EMPNO> <ENAME>KING</ENAME> <JOB>PRESIDENT</JOB> <HIREDATE>17-NOV-81</HIREDATE> <SAL>5000</SAL> <DEPTNO>10</DEPTNO></ROW>


APEX_JSON

begin apex_json.initialize_clob_output; apex_json.open_array('Departments'); for d in (select deptno, dname from scott.dept where deptno = 10) loop apex_json.open_object('Department'); apex_json.write('DeptNo', d.deptno); apex_json.write('DeptName', d.dname); apex_json.open_array('Employees'); for e in (select empno, ename from scott.emp e where e.deptno = d.deptno) loop apex_json.open_object('Employee'); apex_json.write('EmpNo', e.empno); apex_json.write('EmpName', e.ename); apex_json.close_object; end loop; apex_json.close_array; apex_json.close_object; end loop; apex_json.close_array; dbms_output.put_line(apex_json.get_clob_output); apex_json.free_output;end;/


APEX_JSON

"Departments":["Department":{"DeptNo":10,"DeptName":"ACCOUNTING","Employees":["Employee":{"EmpNo":7782,"EmpName":"CLARK"},"Employee":{"EmpNo":7839,"EmpName":"KING"},"Employee":{"EmpNo":7934,"EmpName":"MILLER"}]}]



APEX_JSON

declare c sys_refcursor;begin open c for select d.deptno "DeptNo" , d.dname "DeptName" , cursor( select e.empno "EmpNo" , e.ename "EmpName" from scott.emp e where e.deptno = d.deptno ) "Employees" from scott.dept d where d.deptno = 10; apex_json.initialize_clob_output; apex_json.open_object; apex_json.write('Departments', c); apex_json.close_object; dbms_output.put_line(apex_json.get_clob_output); apex_json.free_output;end;

Besides building manually, APEX_JSON also supports cursor variable


APEX_JSON

{"Departments":[{"DeptNo":10,"DeptName":"ACCOUNTING","Employees":[{"EmpNo":7782,"EmpName":"CLARK"},{"EmpNo":7839,"EmpName":"KING"},{"EmpNo":7934,"EmpName":"MILLER"}]}]}

This time output of ref cursor in a single line


The end


Links

This presentation PowerPoint http://bit.ly/kibeha_xmljson_pptx

Script with all examples from this presentation http://bit.ly/kibeha_xmljson_sql

Questions & AnswersKim Berg HansenSenior Consultant

[email protected]

05/03/2023 Read, Store and Create XML and JSON76

http://bit.ly/kibeha_xmljson_pptxhttp://bit.ly/kibeha_xmljson_sql

read, store and create xml and json

Data & Analytics