data on the web exchange, formats and state advanced web-based systems | misbhauddin
TRANSCRIPT
Advanced Web-based Systems | Misbhauddin
Data on the WebExchange, Formats and State
Components of Data ExchangeTwo applications want to share data over the Internet• Initiating Action: Things don’t happen without any reason. A triggering
mechanism is needed.• Automatic: started at a fixed time (chron job, feed exchange)• User Initiated: form submission
• Data Format: Data needs to be stored and transferred in a format understood by both applications.
• Data Transfer Mechanism: Process to transfer the data over the internet
Advanced Web-based Systems | Misbhauddin
Initiating Action
Advanced Web-based Systems | Misbhauddin
• Human Initiated• Import or export a file• Running a command
• Scheduler• Relying on a fixed schedule• Checking information everyday (weather), every second (traffic update) or
any other period of time. Custom programmed or OS based (chron)
• End User Action• Form Submission
Advanced Web-based Systems | Misbhauddin
Data Format
• Common format that is understood by the communicating applications
• Many formats have been seen over the years• CSV• XML• JSON• HTML5 Data
Advanced Web-based Systems | Misbhauddin
Data Formats
• Why not use existing DBMS• Very popular and provide excellent data storage and access
mechanisms• Main reason for not using them - Incompatibility• Formats• Internal representations• Data types• Encoding
CSV – Comma Separated Values
Advanced Web-based Systems | Misbhauddin
• Fixed-text format• Values separated by a delimiter – usually a comma (,)• Easier to extract data• Simplest of all the data formats• Easy for the customer to import – can use excel as an alternative
to read the contents• Commonly used to export and import data for variety of web
applications
CSV – Comma Separated Values
Advanced Web-based Systems | Misbhauddin
• Program receiving the data is told about the length and datatype of each field.
• Challenges• What if the data has the delimiter inside the text itself• Data is fixed• Needs additional programming
• CSV is standard for data export.
“Susan”,”f”,25 “Mark”,”m”,37
CSV – Language Support
Advanced Web-based Systems | Misbhauddin
• Client-Side• No direct JavaScript Functions• JQuery has many functions that can read and write CSV files
easily• https://code.google.com/p/jquery-csv/• http://papaparse.com/ [CSV ↔JSON]
• Server-Side• Many server-side PLs provide functions to handle CSV files• Example – PHP• fputscsv(file-handle, data) is used to store data into a CSV file• str_getcsv(data) is used to parse CSV data
XML – eXtensible Markup Language
Advanced Web-based Systems | Misbhauddin
• Popularly used to exchange data over the Internet• Create data structures that can be exchanged between
incompatible systems• Tag / Element based language• Derived from SGML• Resembles HTML but more stricter
• Customizable Tags• No SET Element Names – “Self-Describing”• Interoperable• Machine-readable
XML – eXtensible Markup Language
Advanced Web-based Systems | Misbhauddin
USES
Data Presentation (xhtml)<contact> <name>Mohammed Misbhauddin</name> <office>CCSIT 2078</office> <city>Al-Ahsa</city> <province>Eastern</province> <zip>32168</zip> <phone type="office">03-589-9253</phone> <email>[email protected]</email></contact>
<table> <tr> <th>Name</th> <td>Mohammed Misbhauddin</td> </tr>…………………………………..………………..</table>
Name Mohammed Misbhauddin
Office CCSIT 2078
Phone 03-589-9253
Email [email protected]
XML for Structuring Data
Advanced Web-based Systems | Misbhauddin
Data<contact> <name>Mohammed Misbhauddin</name> <office>CCSIT 2078</office> <city>Al-Ahsa</city> <province>Eastern</province> <zip>32168</zip> <phone type="office">03-589-9253</phone> <email>[email protected]</email></contact>
<phone type="office">03-589-9253</phone>
Meta DataThat describes the data
Data
Attributes
XML – Applications
Advanced Web-based Systems | Misbhauddin
• Used as a source for many application communication formats• Universal Business Language (UBL)• Universal Plug ‘n’ Play (UPnP)• Word Processing• Open Data Format (ODF)• Open Office XML (OOXML)
• Scalable Vector Graphics (SVG)• RSS Feeds
XML – Applications
Advanced Web-based Systems | Misbhauddin
Weather Data Communication
<!-- The Weather Channel Interactive, Inc. All Rights Reserved. --><weather ver="2.0"> <loc id="SAXX0031">
<dnam>Al Ahsa, Saudi Arabia</dnam> <tm>01:00 PM</tm> <lat>42.38</lat> <lon>-71.13</lon>
</loc> <cc> <temp> <value>88</value> <deg>F</deg> </temp>……….</weather>
Posts a weather feed
XML – Applications
Advanced Web-based Systems | Misbhauddin
Maps: Google Maps & Google Earth
<kml xmlns="http://schemas.opengis.net/kml/2.2.0/ogckml22.xsd"> <Style id="style1"> <IconStyle> <Icon> <href>http://maps.google.com/mapfiles/ms/micons/green-arrow.png</href> </Icon> </IconStyle> </Style> <Placemark> <name>CCSIT</name> <styleUrl>#style1</styleUrl> <Point> <coordinates>25.334693,49.600021</coordinates> </Point> </Placemark>……….</kml>
Location Data
Keyhole Markup Language (KML) is an XML notation for expressing geographic annotation and visualization within Internet-based, two-dimensional maps
XML – Applications
Advanced Web-based Systems | Misbhauddin
Syndication Feeds: RSS
<rss xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns# version="2.0"> <channel> <title>Saudi Gazette - Kingdom</title> <link>http://www.saudigazette.com.sa</link> <description>Saudi Gazette Newspaper</description> <pubDate>Sat, 02 Mar 2013 22:05:34 GMT</pubDate> <item> <title>Crackdown to flush out infiltrators continues</title> <link>http://www.saudigazette.com.sa/index.cfm</link> <description> <img src='http://www.saudigazette.com.sa/ki09-big.jpg' Falih Al-Dhibyani and Abdullah Gharman Okaz/Saudi Gazette RIYADH/ABHA &mdash; No murders, thefts, or attacks against </description> <pubDate>Sat, 02 Mar 2013 21:00:00 GMT</pubDate> </item>……………</rss>
News Feeds
XML Meta Data
Advanced Web-based Systems | Misbhauddin
• XML allows users to define their own tags• Tags describe the data that it contains (meta-data)• Links to document that describes the data is stored on a server
and linked in the XML document• Documents are called XML Schema (.xsd)• Included in the xmlns attribute of the root element• xmlns: XML Namespace• Example - KML
<kml xmlns=" http://schemas.opengis.net/kml/2.2.0/ogckml22.xsd ">
XML Rules
Advanced Web-based Systems | Misbhauddin
• Well-formed XML Documents• Must have one and only one root element• Elements must be properly nested• Attributes must be “quoted”• XML elements are case sensitive• End tags are required• Empty elements can contain the end marker
• Attributes cannot be repeated
XML DOM Model
Advanced Web-based Systems | Misbhauddin
DOM Tree (Document Object Model)
XML DOM Model
Advanced Web-based Systems | Misbhauddin
• In a node tree, the top node is called the root• Every node, except the root, has exactly one parent node• A node can have any number of children• A leaf is a node with no children• Siblings are nodes with the same parent
Professor
name
id
firstChild
rank
deptlastChild
parentNode
nextSibling
previousSibling
child
Nod
es
XML Processing
Advanced Web-based Systems | Misbhauddin
• Step 1: Load the file into the DOM• All major browsers have a built-in XML parser to read and
manipulate XML• XML parser converts XML into an XML DOM object• How• Create an XMLHttpRequest object• Use the open() and send() methods of the XMLHttpRequest object to
send a request to a server• Get the response data as XML data
https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest
XML DOM Properties
Advanced Web-based Systems | Misbhauddin
• These are some typical DOM properties:• x.nodeName - the name of x• x.nodeValue - the value of x• x.parentNode - the parent node of x• x.childNodes - the child nodes of x• x.attributes - the attributes nodes of x
Note: In the list above, x is a node object.
XML DOM Methods
Advanced Web-based Systems | Misbhauddin
• Methods are what you can do• x.getElementsByTagName(name) - get all elements with a
specified tag name• x.appendChild(node) - insert a child node to x• x.removeChild(node) - remove a child node from x
Note: In the list above, x is a node object.
XML Processing
Advanced Web-based Systems | Misbhauddin
• Step 2: Access elements of the DOM Tree• You can access a node in three ways:
1. By using the getElementsByTagName() method2. By looping through (traversing) the nodes tree.3. By navigating the node tree, using the node relationships.
XML Node Types
Advanced Web-based Systems | Misbhauddin
nodeType Name nodeName Description nodeValue
1 Element element Name Represents an element Null
2 Attribute attribute name Represents an attribute attribute value
3 Text #textRepresents textual content in an element or attribute
content of node
name
type
full
#text
Mohammed Misbhauddin
• nodeName• nodeValue• nodeType
XML Node Lists
Advanced Web-based Systems | Misbhauddin
• Child Nodes• x.childNodes;• Returns an array with the nodes of the parent x
• Attribute Nodes• y = x.attributes;• Returns a Node Value Map• Accessed using • y.getNamedItem(“type")
XML White Spaces
Advanced Web-based Systems | Misbhauddin
• XML often contains new line, or white space characters, between nodes.
• This is often the case when the document is edited by a simple editor like Notepad.
• Always check the type of the node as the total nodes can be deceptive.
XML Technologies
Advanced Web-based Systems | Misbhauddin
• Specification• Data Type Definition (DTD)• Namespaces
• Styling XML• XSLT – Transform XML to HTML and other formats• xPath – Target XML content• XSL-FO – Format XML to different media
JSON – JavaScript Object Notation
Advanced Web-based Systems | Misbhauddin
• Similar to XML, designed to be human-readable• Uses the JavaScript Notation• But is truly platform-independent
• Text-based Code to create Objects
JSON – JavaScript Object Notation
Advanced Web-based Systems | Misbhauddin
• Used for serializing & transmitting structured data over network connection
• Primarily used to transmit data between server and web application
• Web Services and API.s use JSON format to provide public data• Supported by most modern programming languages• Positives
• Easy to read and write JSON• Lightweight text based interchange format• Language independent
JSON – Syntax
Advanced Web-based Systems | Misbhauddin
• Data is in name/value pairs• Separated by Comma (,)• Curly brackets are for objects• Square brackets are for arrays• Names are always strings• Values = strings, numbers, Boolean, arrays, JSON objects or null
JSON – Sample Object
Advanced Web-based Systems | Misbhauddin
{ "book": [ { "id":"01", "language": "Java", "edition": "third", "author": "Herbert Schildt" }, { "id":"07", "language": "C++", "edition": "second" "author": "E.Balagurusamy" } ]}
JSON – Creation in JS
Advanced Web-based Systems | Misbhauddin
var books = { "Pascal" : [ { "Name" : "Pascal Made Simple", "price" : 700 }, { "Name" : "Guide to Pascal", "price" : 400 } ], "Scala" : [ { "Name" : "Scala for the Impatient", "price" : 1000 }, { "Name" : "Scala in Depth", "price" : 1300 } ] }
Advanced Web-based Systems | Misbhauddin
JSON Parsing
• Use eval() to parse and convert JSON to JS Object• Another operation which is more convenient• JSON.parse()
• Convert JS Object to JSON• JSON.stringify();
Advanced Web-based Systems | Misbhauddin
JSON vs. XML
Factor JSON XML
Size JSON is light-weight and easy to read
XML is verbose. Lots of information
Array Usage JSON allows arrays XML is purely structured data. So no arrays.
Parsing Simple. Use eval() to convert to a JS Object
A number of steps to parse and retrieve the data
Advanced Web-based Systems | Misbhauddin
JSON Tools• JSON Lint• Validator for JSON Object• http://jsonlint.com/
• JSON Online Editor• Every browser provides the user the ability to define his own set of styles.• http://jsoneditoronline.org/
• XML to JSON Converter• Converts XML Data to JSON• http://www.utilities-online.info/xmltojson
Advanced Web-based Systems | Misbhauddin
HTML5• Custom Data Attributes using data-*• Application Data• Can be read and written from JavaScript• Using setAttribute and getAttribute
<div data-something=“new” id=“test”>
<script> var a = document.getElementById(“test”); var attr = a.getAttribute(“data-something”); //do Something with the value</script>
Advanced Web-based Systems | Misbhauddin
HTML5• Using DataList• Datalist holds values for autocomplete options• Not supported by iOS
<datalist id="browsers"> <option value="Internet Explorer"> <option value="Firefox"> <option value="Chrome"> <option value="Opera"> <option value="Safari"></datalist>
<input list=“browsers”/>
Advanced Web-based Systems | Misbhauddin
Data for State• The HTTP protocol is stateless• Means that the state of a Request-Response transaction is not stored
Gmail Server
Username=“Mohammed” && Password= “*********”
success
Client
getMain()
error
Advanced Web-based Systems | Misbhauddin
Data for State• State is maintained externally• Needs server-side coding to create variables
State Recognition
Cookies
Returned and stored in the user's browser
Sessions
Session data is stored on your web server (often stored in a public temporary
directory on the server)
Life span of a cookie can be set to almost any duration
Sessions have a predetermined short life
(depends on config)
Advanced Web-based Systems | Misbhauddin
CookiesGmail Server
Username=“Mohammed” && Password= “*********”
Success , Cookie Data
ClientgetMail(), cookie
Store Cookie Data
PHP Function for set and getting a cookie value
setcookie (name, value, expire) $_COOKIE[name]
Set Cookie Get Cookie Value
setcookie (name, “”, currenttime-1)
Delete Cookie Value
Advanced Web-based Systems | Misbhauddin
Sessions
Gmail Server
Username=“Mohammed” && Password= “*********”
Success , Session ID
ClientgetMail(), Session ID
Store Session ID
PHP Function for set and getting a Session value
$_SESSION[name] = value; $_SESSION[name]
Set Session Get Session
unset($_SESSION[name]);
Delete Session
Store Session Data
session_destroy();
session_start();
Advanced Web-based Systems | Misbhauddin
Data for State• HTML5 provides mechanisms known as Local Storage• Store and retrieve data locally• Add state to web without server-side coding• Uses two functions only
SET ITEM: localStorage.setItem(“ItemName”,”ItemValue”);
GET ITEM: localStorage.getItem(“ItemName”);
Advanced Web-based Systems | Misbhauddin
Data Transfer Mechanism
Simple Object Access Protocol (SOAP)
Web Services Description Language or WSDL
Representational State Transfer (REST)
COMING SOON
Advanced Web-based Systems | Misbhauddin
Why not change the DB
If the Data exchanged over the internet is not Relational, then why Relational DBMS
This is why we have schema-less databases
COMING SOONRelational vs. Schema-less Databases