oracle database - get external data via http, ftp and web services

External Data via HTTP, FTPand Web Services

Kim Berg Hansen

T. Hansen Gruppen A/S

Network Access Control List FTP_UTIL_PKG UTL_HTTP HttpUriType UTL_DBWS JPublisher APEX_WEB_SERVICE

Topics

Network ACLbegin dbms_network_acl_admin.create_acl( acl => 'kscope_demo.xml' , description => 'KScope14 demo ACL' , principal => 'SCOTT' , is_grant => true , privilege => 'connect' ); dbms_network_acl_admin.add_privilege( acl => 'kscope_demo.xml' , principal => 'HR' , is_grant => true , privilege => 'connect' ); commit;end;/

begin dbms_network_acl_admin.assign_acl( acl => 'kscope_demo.xml' , host => 'kscope14.com' , lower_port => 80 , upper_port => 80 ); dbms_network_acl_admin.assign_acl( acl => 'kscope_demo.xml' , host => '*.googleapis.com' ); commit;end;/

Create one or more ACLsGrant users connect privilege

Assign hosts / ports to ACLs

Network Access Control List

FTP_UTIL_PKG UTL_HTTP HttpUriType UTL_DBWS JPublisher APEX_WEB_SERVICE

Topics

Tim Hall, ORACLE-BASE

http://www.oracle-base.com/articles/misc/ftp-from-plsql.php

Alexandria PL/SQL Code Library

https://code.google.com/p/plsql-utils/

FTP_UTIL_PKG

Get, put, delete, rename, mkdir, rmdir Ascii, binary

Our use:● We buy data regarding cars on the danish market● Data supplier generates files on FTP server monthly● Scheduled job uses FTP_UTIL_PKG to get files and

imports via external table

FTP_UTIL_PKG

FTP GET file to directorycreate directory ftp_downloads as

'/mnt/vol1/ftp_downloads';

grant read, write on directory ftp_downloads to scott;

create or replace package body car_datais procedure get_carinfo_file is l_conn utl_tcp.connection; begin l_conn := ftp_util_pkg.login( p_host => 'ftp.oursupplier.dk' , p_port => '21' , p_user => 'mycompany' , p_pass => 'myPas$w0rd' ); util_ftp.ascii(p_conn => l_conn); util_ftp.get( p_conn => l_conn , p_from_file => '/carinfo.txt' , p_to_dir => 'FTP_DOWNLOADS' , p_to_file => 'carinfo.txt' ); util_ftp.logout(l_conn); end get_carinfo_file;... continues ...

Server directory and grants to work with files(as SYS or other dba)

FTP protocol calls:login, get, logout

External table and materialized viewcreate table carinfo_ext ( crm_artcode number , approval_id number , variant number , year number , population number) organization external ( type oracle_loader default directory ftp_downloads access parameters ( records delimited by newline territory 'DENMARK' skip 1 fields terminated by '\t' lrtrim missing field values are null ( crm_artcode , approval_id , variant , year , population ) ) location ('carinfo.txt'));

create materialized view carinfo_mviewbuild deferredrefresh complete on demandasselect * from carinfo_ext;

create index carinfo_mview_key on carinfo_mview ( crm_artcode );

External table defined on the file we got by FTP

Materialized view defined on the external table

FTP GET and refresh mview...continued... procedure fresh_carinfo is begin get_carinfo_file; dbms_refresh.refresh('CARINFO_MVIEW'); end fresh_carinfo;end car_data;/

declare jobno binary_integer;begin dbms_job.submit( job => jobno , what => 'car_data.fresh_carinfo' , next_date =>

add_months(trunc(sysdate,'MM'),1) + interval '3' day + interval '22' hour

, interval => q'[add_months(trunc(sysdate,'MM'),1) + interval '3' day + interval '22' hour]'

); commit; dbms_output.put_line(jobno);end;/

When we need fresh data, we first call procedure that FTP downloads the file, then refresh the mat.view from the file Fresh data every 3rd day of month

Use Scheduler rather than jobs if more complex rules are needed

Not quite as easy to do SFTP protocol AskTom discussion:

● https://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:2556405800346433753

Load SFTP java libraries – examples:● http://michailgalvidis.blogspot.com/2010/03/sftp-from-

oracle.html● http://dbashavefeelingstoo.wordpress.com/

2011/07/22/sftp-with-plsql/

SFTP

Network Access Control List FTP_UTIL_PKG

UTL_HTTP HttpUriType UTL_DBWS JPublisher APEX_WEB_SERVICE

Topics

Fairly low-level Can do SOAP, REST, practically whatever

- manual code whatever protocol on top of http GET, POST, … Cookies, redirects, SSL (https), authentication Set parameters for connections, charsets, … In many ways a ”browser emulator” Our use: http POST to SMS service provider

UTL_HTTP

Eric van Roonhttp://www.evrocs.nl/evenementen/kscope14-seattle/

HTTP GET of KScope14 HTML session list UTL_HTTP read HTML a line at a time Pipelined function String functions search for content Pipe sessions allow SELECT from webpage

UTL_HTTP scraping HTML webpage

function pagesource(p_url in st_maxchar, p_proxy in st_maxchar default null) return ta_ero_sourcelines pipelinedis t_http_req utl_http.req; t_http_resp utl_http.resp; t_line st_maxchar; t_lineno integer := 0; r_return to_ero_sourcelines := to_ero_sourcelines();begin if p_proxy is not null then utl_http.set_proxy(p_proxy); end if; t_http_req := utl_http.begin_request (p_url); utl_http.set_header (t_http_req,'User-Agent','Mozilla/4.0'); t_http_resp := utl_http.get_response (t_http_req); loop utl_http.read_line (t_http_resp, t_line, true); t_lineno := t_lineno + 1; r_return.lineno := t_lineno; r_return.text := t_line; pipe row (r_return); end loop;exception when utl_http.end_of_body then utl_http.end_response(t_http_resp); when no_data_needed then utl_http.end_response(t_http_resp);end pagesource;

HTTP request, header and get response

Read response line by line

Close when everything read or "client cursor" stops reading the pipe

procedure retrieve_field_values(p_itemtext in st_maxchar, p_fieldvalues out to_ero_kscope_sessions)is cn_start_indicator_title constant st_maxchar := gc_start_of_new_item||gc_eol; cn_end_indicator_title constant st_maxchar := '<a name="'; cn_start_indicator_presenterid constant st_maxchar := 'presenterlist?last_id='; cn_end_indicator_presenterid constant st_maxchar := '">'; cn_start_indicator_presenter constant st_maxchar := '">'; cn_end_indicator_presenter constant st_maxchar := '</a>'; cn_start_indicator_company constant st_maxchar := ''||

gc_eol; cn_end_indicator_company constant st_maxchar := ''; cn_start_indicator_copresenter constant st_maxchar := 'Co-presenter(s):'; cn_end_indicator_copresenter constant st_maxchar := '</div>'; cn_start_indicator_date constant st_maxchar := 'When: '||gc_eol; cn_end_indicator_date constant st_maxchar := ','; ...begin get_stripped_value (p_itemtext => t_itemtext ,p_start_indicator => cn_start_indicator_title ,p_end_indicator => cn_end_indicator_title ,p_value => r_sessions.title ); get_stripped_value (p_itemtext => t_itemtext ,p_start_indicator => cn_start_indicator_presenterid ,p_end_indicator => cn_end_indicator_presenterid ,p_value => t_presenterid ); ... p_fieldvalues := r_sessions;end retrieve_field_values;

Look for certain html elements

Parse one by one

select * from table(ero$kscope_sessions.kscope_sessions('http://kscope14.com/seminarlist')) where lower(presenter) like '%hansen%' order by starttime;

TOPIC SUBTOPIC STARTTIME ENDTIME SESSION_ID PRESENTER COMPANY COPRES------------------- ----------- ---------------- ---------------- ---------- ---------- ----------------- ------TITLE----------------------------------------------------------------------------------------------------------------DESCRIPTION----------------------------------------------------------------------------------------------------------------Developer's Toolkit Languages 2014-06-24 11:15 2014-06-24 12:15 Session 7 Kim Hansen T. Hansen GruppenAnalytic Functions: Advanced CasesAnalytic functions can often help you, using data across rows in ways to allow you to solve tasks that otherwise might have called for slowly using the data one row at a time procedurally. This presentation will show several such cases of using analytic functions to avoid slow procedural data handling. All cases are from actual production system and will hopefully serve as inspiration.

Developer's Toolkit No Subtopic 2014-06-25 14:00 2014-06-25 15:00 Session 14 Kim Hansen T. Hansen GruppenExternal Data via HTTP, FTP, and Web ServicesThere are many ways you can access external data from within the Oracle database. This presentation will show different methods using HTTP or FTP protocol getting raw data, files, or XML via proprietary calls or web-servicecalls (either REST or SOAP). Topics include HttpUriType, UTL_HTTP, UTL_DBWS, APEX_WEB_SERVICE API, and more.

Just two presentations by a presenter named Hansen

select presenter, title from table(ero$kscope_sessions.kscope_sessions('http://kscope14.com/seminarlist')) where session_id = 'Session 14' order by presenter, title;

PRESENTER TITLE--------------- ------------------------------------------------------------------------------------------------Ashley Chen Hands-On Training: Developing RESTful APIs with Oracle REST Data Services for On-Premises or theCharles Beyer Bulletproofing Excel and Smart ViewDonald Clarke How to Integrate EBS 11i with APEX 4.2Eric Erikson Why Aren't You Using Calc Manager with HFM Yet?Heli Helskyaho Nine Use Cases for Oracle SQL Developer Data ModelerJackie Womble Production-Level Control of APEX Troubleshooting and Performance TracingJake Turrell Planning Experts PanelJason Jones How to Integrate Essbase with Oracle and Third-Party ApplicationsJeremy Harms Sorry, McGinley: Even Advanced OBIEE Modeling and Front-End Features Show Big Ten Football < SECJohn Flack Magic Managed Beans: A Primer for Jack and Jill ADF DeveloperJohn Kozar Common APEX Responsive Design Issues and How to Correct ThemJonathan Lewis Developers' Guide to Cost-Based OptimizationKenny Vernon Numbers Don't Always Tell the Whole Story: Allowing Users to Enter Comments on OBIEE DashboardsKim Hansen External Data via HTTP, FTP, and Web ServicesKyle Hailey Agile Data Platform: Revolutionizing Database CloningMichael Casey Using the Open Interface Adapter in FDM Enterprise Edition 11.1.2.3Paul Hoch Driver-Based Planning: When More Isn't BetterScott LeBeau Implementing Advanced Financial Reporting Concepts for Rolling Trend ReportsTiffany Briseno Hyperion Financial Reporting: Tips, Tricks, and Best Practices at TeleTech

19 rows selected.

Uh oh… Jonathan Lewis in same timeslot…

• Save site certificate using browser• Create wallet on db server and add certificate:

mkdir /home/oracle/admin/wallet

orapki wallet create -wallet /home/oracle/admin/wallet -pwd myW4lletPasswd -auto_login

orapki wallet add -wallet /home/oracle/admin/wallet -trusted_cert -cert "/mnt/share/savedcertificate.crt" -pwd myW4lletPasswd

• Set wallet for utl_http:

utl_http.set_wallet('file:/home/oracle/admin/wallet', 'myW4lletPasswd');

• Wallet will now be used for https:// urls

• http://www.oracle-base.com/articles/misc/utl_http-and-ssl.php

SSL (https) with UTL_HTTP

Network Access Control List FTP_UTIL_PKG UTL_HTTP

HttpUriType UTL_DBWS JPublisher APEX_WEB_SERVICE

Topics

Very simple for HTTP GET of content No SSL, No POST, No Cookies, No… Object type with member methods

● GetBlob()● GetClob()● GetXml()

Not browser simulator But easy to get a resource from the web

HttpUriType

Our use: HttpUriType('url').GetBlob()

● Retrieve employee picture from intranet website

HttpUriType('url').GetClob()● Get simple up/down status from payment provider

HttpUriType('url').GetXml()● Google Maps routes to find time needed for delivery

routes from shop to shop in fireworks season

HttpUriType

declare directions xmltype;begin directions := httpuritype( 'maps.googleapis.com/maps/api/directions/xml' || '?origin=' || utl_url.escape(convert( '100 Oracle Pkwy, Redwood City, CA' , 'UTF8' )) || '&destination=' || utl_url.escape(convert( '1400 6th Ave, Seattle, Washington' , 'UTF8' )) || '&mode=driving&alternatives=true&units=metric' || '&region=us&language=en&sensor=false' ).getxml();end;/

REST-like Google Maps

Simply pass complete URL to HttpUriType

In SQL*Plus use SET DEFINE OFF to avoid substitution variable promptingOr use || '&' || 'destination... as single '&' will not be treated as variable

<?xml version="1.0" encoding="UTF-8"?><DirectionsResponse> <status>OK</status> <route> <summary>I-5 N</summary> <leg> <step> <travel_mode>DRIVING</travel_mode> <start_location> <lat>37.5288827</lat> <lng>-122.2666866</lng> </start_location> <end_location> <lat>37.5288322</lat> <lng>-122.2601636</lng> </end_location> <polyline><points>oz`dFxegiVCBADCHKTQX_@ZSHO@SDU?QAOCMGMIOMQSISKWKSKOSOKGKEOCiCMUCU@]?Q?

OAMCOEMGQIQKKMU_@MWK]G_@C[AS?U?O?QBk@Dg@Jg@^wAZ}@L]BODQDYBW@]@O?m@A}A@q@@Q?MBIBMFUDIJMRQHEBCJELCLA`@?T@F?x@@PAn@OZOLKXWLMFEd@Ob@IB?LCPCPGRMPQN?BABAFCJGFAFCZE</points>

</polyline> <duration> <value>170</value> <text>3 mins</text> </duration> <html_instructions>Head northwest on Oracle Pkwy toward

Marine Pkwy</html_instructions> <distance> <value>1247</value>...

XML results of call

select r.routenum, l.legnum , numtodsinterval(l.seconds, 'second') time, l.meters / 1000 distance_km , extractvalue(x.x,'/DirectionsResponse/status') status , convert(r.routename,'WE8ISO8859P1','UTF8') routename , convert(l.startaddr,'WE8ISO8859P1','UTF8') startaddr , convert(l.endaddr,'WE8ISO8859P1','UTF8') endaddr from xmltable( '/DirectionsResponse' passing httpuritype( 'maps.googleapis.com/maps/api/directions/xml' || '?origin=' || utl_url.escape(convert( '100 Oracle Pkwy, Redwood City, CA' , 'UTF8' )) || '&destination=' || utl_url.escape(convert( '1400 6th Ave, Seattle, Washington' , 'UTF8' )) || '&mode=driving&alternatives=true&units=metric' || '&region=us&language=en&sensor=false' ).getxml() columns x xmltype path '/' ) x,...

Process with XMLTABLE …

httpuritype().getxml() in passing clause for xmltable for easy parsing of the result

... xmltable( 'if (fn:empty(/DirectionsResponse/route)) then <route><leg></leg></route> else /DirectionsResponse/route' passing x.x columns routenum for ordinality , routename varchar2(100) path 'summary' , r xmltype path '/' ) r, xmltable( 'for $l in /route/leg return $l' passing r.r columns legnum for ordinality , seconds number path 'duration/value' , meters number path 'distance/value' , startaddr varchar2(100) path 'start_address' , endaddr varchar2(100) path 'end_address' ) l order by r.routenum, l.legnum;

… and some Xquery …

Xquery here used to make dummy empty route if Google returned an error status

ROUTENUM LEGNUM TIME DISTANCE_KM STATUS ROUTENAME-------- ------ ------------------------------ ----------- ------ --------------------STARTADDR ENDADDR------------------------------------------------

--------------------------------------- 1 1 +000000000 12:37:44.000000000 1334.2 OK I-5 N100 Oracle Parkway, Redwood City, CA 94065, USA 1400 6th Avenue, Seattle, WA 98101,

USA

2 1 +000000000 15:19:09.000000000 1521.3 OK I-5 N100 Oracle Parkway, Redwood City, CA 94065, USA 1400 6th Avenue, Seattle, WA 98101,

USA

… to get route data

Google shows two alternative routes- one 1334 km in a bit more than 12½ hours- one 1521 km in a bit less than 15½ hours

select r.routenum, l.legnum, s.stepnum , numtodsinterval(s.seconds, 'second') time, s.meters / 1000 distance_km , convert(s.html_instructions,'WE8ISO8859P1','UTF8') html_instructions from xmltable( '/DirectionsResponse'

passing httpuritype(

'maps.googleapis.com/maps/api/directions/xml'

|| '?origin=' || utl_url.escape(convert(

'100 Oracle Pkwy, Redwood City, CA'

, 'UTF8'

))

|| '&destination=' || utl_url.escape(convert(

'1400 6th Ave, Seattle, Washington'

, 'UTF8'

))

|| '&mode=driving&alternatives=true&units=metric'

|| '&region=us&language=en&sensor=false'

).getxml()

columns x xmltype path '/'

) x, xmltable(

'if (fn:empty(/DirectionsResponse/route)) then <route><leg></leg></route>

else /DirectionsResponse/route'

passing x.x

columns routenum for ordinality

, routename varchar2(100) path 'summary'

, r xmltype path '/'

) r, xmltable(

'for $l in /route/leg return $l'

passing r.r

columns legnum for ordinality

, seconds number path 'duration/value'

, meters number path 'distance/value'

, startaddr varchar2(100) path 'start_address'

, endaddr varchar2(100) path 'end_address'

, l xmltype path '/' ) l, xmltable( '/leg/step' passing l.l columns stepnum for ordinality , seconds number path 'duration/value' , meters number path 'distance/value' , html_instructions varchar2(1000) path 'html_instructions' ) s order by r.routenum, l.legnum, s.stepnum;

More XMLTABLE for route steps

Put on yet another XMLTABLE to get individual steps

RO LE STE TIME DISTANC HTML_INSTRUCTIONS-- -- --- ------------------------------ ------- -------------------------------------------------------------------------------- 1 1 1 +000000000 00:02:50.000000000 1.2 Head northwest on Oracle Pkwy toward Marine Pkwy 1 1 2 +000000000 00:00:58.000000000 .7 Turn right onto Marine Pkwy 1 1 3 +000000000 00:21:28.000000000 33.6 Merge onto US-101 N via the ramp to San Francisco 1 1 4 +000000000 00:00:33.000000000 .6 Take the Interstate 80 exit toward Bay Bridge 1 1 5 +000000000 00:14:30.000000000 19.0 Merge onto I-80 E 1 1 6 +000000000 00:39:50.000000000 69.2 Keep left to stay on I-80 E<div style="font-size:0.9em">Partial to ll road</div> 1 1 7 +000000000 00:28:11.000000000 55.0 Take the exit onto I-505 N toward Winters/Redding 1 1 8 +000000000 10:48:17.000000000 1154.3 Merge onto I-5 N<div style="font-size:0.9em">Passing through Oregon</div> <div style="font-size:0.9em">Entering Washington</div> 1 1 9 +000000000 00:00:24.000000000 .4 Take exit 165 on the left for Seneca St 1 1 10 +000000000 00:00:43.000000000 .3 Slight right onto 6th Ave<div style="font-size:0.9em">Destination will be on the right</div> 2 1 1 +000000000 00:02:50.000000000 1.2 Head northwest on Oracle Pkwy toward Marine Pkwy 2 1 2 +000000000 00:00:58.000000000 .7 Turn right onto Marine Pkwy 2 1 3 +000000000 00:21:28.000000000 33.6 Merge onto US-101 N via the ramp to San Francisco 2 1 4 +000000000 00:00:33.000000000 .6 Take the Interstate 80 exit toward Bay Bridge 2 1 5 +000000000 00:14:30.000000000 19.0 Merge onto I-80 E 2 1 6 +000000000 00:52:11.000000000 91.9 Keep left to stay on I-80 E<div style="font-size:0.9em">Partial to ll road</div> 2 1 7 +000000000 00:10:46.000000000 19.5 Keep left to stay on I-80 E 2 1 8 +000000000 02:04:42.000000000 220.0 Keep right to stay on I-80 E, follow signs for Reno<div sty le="font-size:0.9em">Entering Nevada</div> 2 1 9 +000000000 01:13:38.000000000 129.5 Take exit 15 to merge onto US-395 N toward Susanville<div s tyle="font-size:0.9em">Entering California</div> 2 1 10 +000000000 00:00:06.000000000 .1 Slight left to stay on US-395 N 2 1 11 +000000000 00:03:12.000000000 4.6 Continue onto CA-36 W 2 1 12 +000000000 00:00:20.000000000 .2 Turn right onto Riverside Dr 2 1 13 +000000000 00:00:47.000000000 .6 Continue straight onto Johnstonville Rd 2 1 14 +000000000 00:02:52.000000000 3.3 Turn left onto Skyline Rd 2 1 15 +000000000 01:31:15.000000000 140.0 Turn right onto CA-139 N 2 1 16 +000000000 00:51:31.000000000 88.5 Turn left to stay on CA-139 N<div style="font-size:0.9em">Entering Oregon</div> 2 1 17 +000000000 00:19:24.000000000 27.6 Continue onto OR-39 N 2 1 18 +000000000 00:06:36.000000000 9.4 Turn left onto OR-140 W 2 1 19 +000000000 00:04:52.000000000 7.1 Turn left to merge onto US-97 N 2 1 20 +000000000 01:19:32.000000000 125.1 Take the ramp onto US-97 N 2 1 21 +000000000 01:31:17.000000000 138.3 Exit on the left onto OR-58 W toward Oakridge/Eugene 2 1 22 +000000000 04:24:42.000000000 459.7 Slight right to merge onto I-5 N/OR-99 N toward Eugene<div style="font-size:0.9em">Continue to follow I-5 N</div><div style="font-size:0.9e m">Entering Washington</div> 2 1 23 +000000000 00:00:24.000000000 .4 Take exit 165 on the left for Seneca St 2 1 24 +000000000 00:00:43.000000000 .3 Slight right onto 6th Ave<div style="font-size:0.9em">Destination will be on the right</div>

34 rows selected.

"Proper" REST webservice exposed like Oracle Rest Data Services with documentcentric URLs● http://myserver.net/myprefix/hr/employees

GET data with HttpUriType('url').GetXml()or

GET data with HttpUriType('url').GetClob()and use 12.2 JsonTable() to parse result

Not POST, PUT, DELETE etc.

RESTful webservicecalls

Network Access Control List FTP_UTIL_PKG UTL_HTTP HttpUriType

UTL_DBWS JPublisher APEX_WEB_SERVICE

Topics

Helps setup SOAP webservice calls● Handles SOAP envelope, service, port, operation

Can do SSL (https) with wallets Extra installation (not in default install) Uses java classes Our use:

● Payment authorization via provider’s SOAP webservice that uses SSL

UTL_DBWS

Starting point:http://www.oracle.com/technetwork/database/database-083829.html

Download:http://download.oracle.com/technology/sample_code/tech/java/jsp/dbws-callout-utility-10131.zip

MOS (metalink) note 838892.1:FAQ on UTL_DBWS / Jpublisher callout

UTL_DBWS

UTL_DBWS privilegesgrant javauserpriv to scott;

BEGIN SYS.DBMS_JAVA.GRANT_PERMISSION( grantee => 'SCOTT' ,permission_type =>

'SYS:java.net.SocketPermission' ,permission_name => '*' ,permission_action => 'connect, resolve' ); SYS.DBMS_JAVA.GRANT_PERMISSION( grantee => 'SCOTT' ,permission_type =>

'SYS:java.util.PropertyPermission' ,permission_name => '*' ,permission_action => 'read,write' ); SYS.DBMS_JAVA.GRANT_PERMISSION( grantee => 'SCOTT' ,permission_type =>

'SYS:java.lang.RuntimePermission' ,permission_name => 'getClassLoader' ,permission_action => '' );

SYS.DBMS_JAVA.GRANT_PERMISSION( grantee => 'SCOTT' ,permission_type =>

'SYS:java.lang.RuntimePermission' ,permission_name => 'createClassLoader' ,permission_action => '' ); SYS.DBMS_JAVA.GRANT_PERMISSION( grantee => 'SCOTT' ,permission_type =>

'SYS:java.util.logging.LoggingPermission' ,permission_name => 'control' ,permission_action => '' ); SYS.DBMS_JAVA.GRANT_PERMISSION( grantee => 'SCOTT' ,permission_type =>

'SYS:java.lang.RuntimePermission' ,permission_name => 'setFactory' ,permission_action => '' ); commit;END;/ Not ACL, but java privileges

unzip dbws-callout-utility-10131.zip

cp sqlj/lib/* $ORACLE_HOME/sqlj/lib

cd $ORACLE_HOME/sqlj/lib

loadjava -user scott/tiger -resolve -verbose -force -genmissing dbwsclientws.jar dbwsclientdb11.jar

sqlplus scott/tiger

@utl_dbws_decl.sql

@utl_dbws_body.sql

Alternative for installing for public use:loadjava -user tools/tooluserpsw -resolve -verbose -force -genmissing -synonym -grant

public dbwsclientws.jar dbwsclientdb11.jarCreate packages in tools schema and grant privileges as necessary

UTL_DBWS install

declare l_namespace varchar2(1000) := 'http://ws.cdyne.com/WeatherWS/'; l_wsdlurl varchar2(1000) := 'http://wsf.cdyne.com/WeatherWS/Weather.asmx?WSDL'; l_endpointurl varchar2(1000) := 'http://wsf.cdyne.com/WeatherWS/Weather.asmx'; l_zipcode varchar2(10) := '98101'; -- Seattle l_service utl_dbws.service; l_call utl_dbws.call; l_service_qname utl_dbws.qname; l_port_qname utl_dbws.qname; l_operation_qname utl_dbws.qname; l_request xmltype; l_response xmltype;begin l_service_qname := utl_dbws.to_qname(l_namespace, 'Weather'); l_service := utl_dbws.create_service(httpuritype(l_wsdlurl), l_service_qname); l_port_qname := utl_dbws.to_qname(l_namespace, 'WeatherSoap12'); l_operation_qname := utl_dbws.to_qname(l_namespace, 'GetCityWeatherByZIP'); l_call := utl_dbws.create_call( l_service , l_port_qname , l_operation_qname );

utl_dbws.set_property(l_call, 'SESSION_MAINTAIN', 'FALSE'); utl_dbws.set_property(l_call, 'SOAPACTION_USE', 'TRUE'); utl_dbws.set_property(l_call, 'SOAPACTION_URI', l_endpointurl); utl_dbws.set_property(l_call, 'ENCODINGSTYLE_URI', 'http://schemas.xmlsoap.org/soap/encoding/'); utl_dbws.set_property(l_call, 'OPERATION_STYLE', 'document');...

... utl_dbws.set_target_endpoint_address(l_call, l_endpointurl);

select xmlroot( xmlelement( "GetCityWeatherByZIP" , xmlattributes('http://ws.cdyne.com/WeatherWS/' as "xmlns") , xmlelement( "ZIP" , l_zipcode ) ) , version '1.0' ) into l_request from dual;

l_response := utl_dbws.invoke(call_Handle => l_call, request => l_request);

utl_dbws.release_call(l_call); utl_dbws.release_service(l_service); utl_dbws.release_all_services(); dbms_output.put_line(l_response.getclobval());end;/

Select from DUAL to avoid error:PLS-00122: AS as separator is allowed only with specific built-in functions

-- l_request –-

<?xml version="1.0"?><GetCityWeatherByZIP xmlns="http://ws.cdyne.com/WeatherWS/"> <ZIP>98101</ZIP></GetCityWeatherByZIP>

-- l_response --

<GetCityWeatherByZIPResponse xmlns="http://ws.cdyne.com/WeatherWS/"><GetCityWeatherByZIPResult><Success>true</Success><ResponseText>City Found</ResponseText><State>WA</State><City>Seattle</City><WeatherStationCity>Renton</WeatherStationCity><WeatherID>3</WeatherID><Description>Mostly Cloudy</Description><Temperature>72</Temperature><RelativeHumidity>37</RelativeHumidity><Wind>VRB6</Wind><Pressure>29.98R</Pressure><Visibility/><WindChill/><Remarks/></GetCityWeatherByZIPResult></GetCityWeatherByZIPResponse>

• Save site certificate using browser• Add certificate to wallet in path:

$ORACLE_HOME/javavm/lib/security/cacerts

• Java based UTL_DBWS can now use wallet for https:// urls

• MOS (metalink) note 443438.1

SSL (https) with UTL_DBWS

Network Access Control List FTP_UTIL_PKG UTL_HTTP HttpUriType UTL_DBWS

JPublisher APEX_WEB_SERVICE

Topics

Install SQLJ from client installationSee JPublisher user guide for details● http://docs.oracle.com/cd/E11882_01/java.112/e105

87/toc.htm

Serverside load java and UTL_DBWSSee previous slides

Can be clientside only on development PC if so desired – jpub really is just code generator

JPublisher

Might use local copy of wsdl file rather than URL to original WSDL file● For example to annotate WSDL file with Oracle

datatypes or fix WSDL file xml schema errors

Our use:● Other cardata from dataprovider we FTP from● Parcel registering with parcel service● Parcel track & trace with parcel service

JPublisher for SOAP callout

SET JAVA_HOME=%ORACLE_HOME%\jdkSET TNS_ADMIN=%ORACLE_HOME%\network\adminSET PATH=%JAVA_HOME%\bin;%JAVA_HOME%\jre\bin;%PATH%

SET CLASSPATH=%ORACLE_HOME%\sqlj\lib\dbwsa.jar;%ORACLE_HOME%\jdk\lib\dt.jar;^%ORACLE_HOME%\jdk\lib\tools.jar;^%ORACLE_HOME%\jlib\jssl-1_1.jar;%ORACLE_HOME%\jdbc\lib\ojdbc5.jar;^%ORACLE_HOME%\sqlj\lib\runtime12.jar;%ORACLE_HOME%\jlib\orai18n.jar;^%ORACLE_HOME%\sqlj\lib\translator.jar;%ORACLE_HOME%\javavm\lib\aurora.zip;^%ORACLE_HOME%\rdbms\jlib\xdb.jar;%ORACLE_HOME%\lib\xsu12.jar:^%ORACLE_HOME%\jlib\jndi.jar;%ORACLE_HOME%\rdbms\jlib\aqapi.jar;^%ORACLE_HOME%\rdbms\jlib\jmscommon.jar;%ORACLE_HOME%\lib\xmlparserv2.jar

SET LOCAL=MYORCL

jpub -user=scott/tiger -sysuser=sys/syspwd -dir=weather -package=weather -plsqlpackage=weather -proxywsdl=http://wsf.cdyne.com/WeatherWS/Weather.asmx?WSDL

For testing without loading into the database:jpub -user=scott/tiger -sysuser=sys/syspwd -proxyopts=noload -dir=weather -

package=weather -plsqlpackage=weather -proxywsdl=http://wsf.cdyne.com/WeatherWS/Weather.asmx?WSDL

On windows client with jpub

weatherobject := weather.GetCityWeatherByZIP('98101');dbms_output.put_line(weatherobject.Description);

select w.weatherobject.Success , w.weatherobject.ResponseText , w.weatherobject.State , w.weatherobject.City , w.weatherobject.WeatherStationCity , w.weatherobject.WeatherID , w.weatherobject.Description , w.weatherobject.Temperature , w.weatherobject.RelativeHumidity , w.weatherobject.Wind , w.weatherobject.Pressure , w.weatherobject.Visibility , w.weatherobject.WindChill , w.weatherobject.Remarks from ( select weather.GetCityWeatherByZIP('98101') weatherobject from dual ) w/

Use of generated objects PL/SQL or SQLPL/SQL use object type

Object in SQL as wellBeware this may call the webservice 14 times as SQL engine may rewrite to call package function once for every column

Network Access Control List FTP_UTIL_PKG UTL_HTTP HttpUriType UTL_DBWS JPublisher

APEX_WEB_SERVICE

Topics

Useful even if not using APEX SOAP

● Manual SOAP envelope handling

REST● GET, POST, PUT, DELETE, etc.

SSL made simple with wallet parameters Uses network ACL, not java privileges Possibly more "supported" in future

APEX_WEB_SERVICE

Add a bit to network ACLbegin dbms_network_acl_admin.add_privilege( acl => 'kscope_demo.xml' , principal => 'APEX_040100' , is_grant => true , privilege => 'connect' ); commit;end;/

begin dbms_network_acl_admin.assign_acl( acl => 'kscope_demo.xml' , host => '*.cdyne.com' ); commit;end;/

APEX user granted connect privilege to our demo ACLUse APEX user for your APEX version

Weather URL added to ACLWas not needed for UTL_DBWS java based, but is needed for APEX_WEB_SERVICE

declare directions clob;begin directions := apex_web_service.make_rest_request( p_url => 'http://maps.googleapis.com/maps/api/directions/xml' , p_http_method => 'GET' , p_parm_name => apex_util.string_to_table('origin:destination:mode:alternatives:units:region:language:sensor' ) , p_parm_value => apex_util.string_to_table('100 Oracle Pkwy, Redwood City, CA:1400 6th Ave, Seattle,

Washington:driving:true:metric:us:en:false' ) ); dbms_output.put_line(substr(directions,1,4000));end;/

APEX_WEB_SERVICE Google Maps

APEX_WEB_SERVICE REST callExample of our google maps call

Colon separated name/value pairs

declare envelope clob; weather xmltype;begin select xmlroot( xmlelement( "soap:Envelope" , xmlattributes( 'http://www.w3.org/2001/XMLSchema-instance' as "xmlns:xsi" , 'http://www.w3.org/2001/XMLSchema' as "xmlns:xsd" , 'http://schemas.xmlsoap.org/soap/envelope/' as "xmlns:soap" ) , xmlelement( "soap:Body" , xmlelement( "GetCityWeatherByZIP" , xmlattributes('http://ws.cdyne.com/WeatherWS/' as "xmlns") , xmlelement("ZIP", '98101') ) ) ) , version '1.0' ).getclobval() into envelope from dual; weather := apex_web_service.make_request( p_url => 'http://wsf.cdyne.com/WeatherWS/Weather.asmx' , p_action => 'http://ws.cdyne.com/WeatherWS/GetCityWeatherByZIP' , p_version => '1.1' , p_envelope => envelope ); dbms_output.put_line(weather.getclobval());end;/

APEX_WEB_SERVICE SOAP callExample of our weather call

Unlike UTL_DBWS SOAP envelope is manually created here and response will be wrapped in SOAP envelope

• Save site certificate using browser• Create wallet on db server and add certificate:

mkdir /home/oracle/admin/wallet

orapki wallet create -wallet /home/oracle/admin/wallet -pwd myW4lletPasswd -auto_login

orapki wallet add -wallet /home/oracle/admin/wallet -trusted_cert -cert "/mnt/share/savedcertificate.crt" -pwd myW4lletPasswd

• Use wallet in calls like:

directions := apex_web_service.make_rest_request( p_url => 'https://maps.googleapis.com/maps/api/directions/xml' , ... , p_wallet_path => '/home/oracle/admin/wallet' , p_wallet_pwd => 'myW4lletPasswd');

SSL (https) with APEX_WEB_SERVICE

Several methods available Each has different pros and cons No single "best" way Pick easiest in each circumstance Maybe APEX_WEB_SERVICE is where future

development might be from Oracle?

Summary

Danish geek Oracle SQL Evangelist Oracle PL/SQL Developer Likes to cook Reads sci-fi Member of

Danish Beer Enthusiasts

Questions ?

http://dspsd.blogspot.com

http://dk.linkedin.com/in/kibeha/

@kibeha

Kim Berg Hansen

http://goo.gl/QuHQ1f

for this presentation

oracle database - get external data via http, ftp and web services

Software

meters number path

day interval

language en

browser create wallet

seconds number path

mode driving

begin dbms network acl admin

alternatives true