w5 - sql queries and aggregation › pluginfile.php › 1230613 › course › sectio… ·...

Databases :: Lecture 7Databases :: Lecture 7

SQL Queries and Aggregation

CS-A1153: Databases

Prof. Nitin Sawhney


Acknowledgements

These slides are based in part on presentation materials created by Kerttu Pollari-Malmi and Juha Puustjärvi in previous years and on the course text book: A First Course in Database Systems, Third Edition. Pearson by Jeffery D. Ullman and Jennifer Widom.

Thanks to Etna Lindy & Ville Vuorenmaa for translating prior lecture slides for this course.


► Creating SQL queries that use results from another query (subqueries).

► Using different SQL join expressions.

► Computing sums and means with tuples that satisfy a given condition for the values of their attributes (aggregation and grouping).

Learning Goals


► Structured Query Language (SQL) used for:

► Querying a Database – searching for tuples

► Data Manipulation – modifying (e.g. inserting/deleting tuples in database)

► Data Definition – declaring a database schema

Role of SQL in Databases


Example Database► Examples in this lecture use the database from prior

lectures consisting of the following relations/tables:Customers(custNo, name, born, bonus, address, email) Products(number, prodName, description, price, manufID) Manufacturers(ID, manufName, phone)Orders(orderNo, deliver, status, custNo) BelongsTo(orderNo, productNo, count)

Product

number PKprodNamedescriptionprice

Manufacturer

ID PKname phone

Order

orderNo PKdeliver status

Customer

custNo PKname born bonus address email

0..1

*

* * * 1..1

Made-by

Belongs-to

Ordered-by


Using SQLiteStudio


► Most simple SQL Queries use a 3 keyword form:

► SELECT - which attributes of tuples are produced.► FROM - relation(s) to which the query refers.► WHERE - condition tuples must satisfy to match query.

SELECT *FROM ProductsWHERE price > 200

Using SQL for Database Queries


Specifying the WHERE condition using:► Comparison operators like =, >, <, =>, <=, <>

combined with► Logical operators AND, OR and NOT

Disambiguating attributes of relations in tuples.

SELECT *FROM Manufacturers, ProductsWHERE price > 200 AND Manufacturers.ID = Products.manufID;

Comparison and Logical Operators in SQL


► Projecting the relation produced by a SQL query onto some of its attributes (instead of using SELECT *).

► Using the keyword AS to rename attributes.► Using an expression to compute an arithmetic function.

SELECT ID, manufName AS Company, prodName AS Device, price*0.89 AS Euros

FROM Manufacturers, ProductsWHERE price > 200 AND Manufacturers.ID = manufID;

Projection in SQL


Subqueries in SQL► In SQL, one query can be used in various ways to

help in the evaluation of another query.

► Queries that are part of another are called subqueries.

► Subqueries can be written inside the WHERE or FROM clause and can have their own subqueries.

► The result of the subquery can be a single value (subquery that produces a scalar value) or a relation.


Query that Produces a Scalar Value

► Example: Find the name of the manufacturer and thename of the product for model number R-55336.

► The SQL can be formulated as follows:SELECT manufName, prodName, number FROM Manufacturers, ProductsWHERE ID = manufID AND number = 'R-55336’;

This SQL query works fine for solving the problem, but we can also use another approach using a subquery.


Alternative SQL using a Subquery

► First search for the ID of the manufacturer of R-55336 from the table Products. Then look for information corresponding to this ID in the Manufacturers table. (This is not necessarily the real order of execution)

► The first query is now inserted inside the WHERE part of the other query, giving the same result:

SELECT manufName, prodName, number FROM Manufacturers WHERE ID =

(SELECT manufIDFROM ProductsWHERE number = 'R-55336’);


Subqueries that Produce Relations

► We can use the earlier approach only if the result of the subquery is a single value instead of a relation.

► If the result of the subquery can be a relation, SQL offers a set of different operators that can be used to compare values from the subquery and the main query.

► For instance, we can investigate whether or not the resulting relation is empty or if some given tuple belongs to the resulting relation.


► EXISTS RTrue if R is not empty.

► s IN RTrue if and only if s is same as some tuple that belongs to R. (In SQLite, there can be only one column in R)

► s > ALL RTrue if and only if s is greater than all the values in R. R must be unary, i.e. it can only have one column. We can use other comparison operators <, <=, >=, = or <>.

► s > ANY RTrue if and only if s is greater than some value in R. We can also use other comparison operators.

► We can negate the condition by adding NOT in front of the operator.

Note: SQLite doesn’t support the operators ALL and ANY.

Operators for Subqueries that Produce Relations


Example

► Query for the numbers of those orders that include a product with a description “cellphone”.

SELECT DISTINCT orderNoFROM BelongsToWHERE productNo IN

(SELECT number FROM ProductsWHERE description = ’cellphone’

);


► Query for the names of the manufacturers, that have products with price greater than 100 euros.

SELECT DISTINCT manufNameFROM ManufacturersWHERE ID IN

(SELECT manufIDFROM Products WHERE price > 100

);

Example 2


► Query the names and numbers of customers that have ordered the product T-77445 (model number).

SELECT name, custNoFROM Customers WHERE custNo IN

(SELECT custNoFROM Orders AS O WHERE O.orderNo IN

(SELECT B.orderNoFROM BelongsTo AS BWHERE productNo = ’T−77445’)

);

Example 3: Nested Subqueries


Referring to a Tuple from the Outer Query► In this subquery, refer to a tuple used in the outer query.► Example: Query for the cheapest products of each product

type (same description). If there is more than one product with the cheapest price, the query includes them all.

SELECT DISTINCT number, prodName, description, price FROM Products AS CheapWHERE price <= ALL

(SELECT price FROM ProductsWHERE description = Cheap.description

);

► Subquery has to be executed separately for each tuple in the outer query. Note: This query doesn’t work in SQLite, since it does not support the ALL and ANY operators.


Referring to a Tuple from the Outer Query, Part 2

► Alternative query that works in SQLite:SELECT DISTINCT number, prodName, description, price FROM Products AS CheapWHERE NOT EXISTS

(SELECT ∗FROM ProductsWHERE description = Cheap.descriptionAND price < Cheap.price

);

► The condition in the WHERE clause of the outer query is true if there are no tuples in the subquery.


Referring to a Tuple from the Outer Query

► With subqueries we need to pay close attention to which attributes refer to which tables.

► Rule: The attribute in question refers to some table in the FROM clause, if there exists a table with an an attribute with this name. If there isn’t one, we look for the attribute from the FROM clause of the outer query. Using a tuple variable we can specify that we mean a table in the outer query.

► Thus the tuple variables O and B in the nested subqueries earlier were not needed, as shown inthe example next.


► Query the names and numbers of customers that have ordered the product T-77445 (model number).

SELECT name, custNoFROM Customers WHERE custNo IN

(SELECT custNoFROM Orders AS O WHERE O.orderNo IN

(SELECT B.orderNoFROM BelongsTo AS BWHERE productNo = ’T−77445’)

);

Example


Subquery in the FROM clause► We can write subqueries in the FROM clause. Outer

query executes the query on the relation that results from the subquery.

► If we want to refer to the resulting relation, we need to use a tuple variable.

► Example: another way to search for names & numbers of customers, that have ordered product T-77445.SELECT name, Customers.custNoFROM Customers, (SELECT custNo

FROM Orders AS O, BelongsTo AS B WHERE O.orderNo = B.orderNoAND productNo = ’T−77445’

) AS C1WHERE Customers.custNo = C1.custNo;


Writing Join Operations in SQL

► We can construct relations by a number of variations to the join operator applied to two relations.

► SQL provides several join operators: JOIN, NATURALJOIN, OUTER JOIN and CROSS JOIN.

► In a sense, using the join operators doesn’t offer anything new but an alternative way to write the same query. OUTER JOIN is an exeption, as an equivalent query is a lot more laborious to write without thisoperator.

► CROSS JOIN produces the cartesian product between the two tables, which is rarely useful as it is. In the following slides we introduce the other join operators mentioned here.


JOIN Operator

► The JOIN operator is equivalent to a theta join obtained with the keyword ON.

► We can put join between two relation names and follow them by ON and a condition.

► We can for instance connect products with their manufacturer. SELECT prodName, manufNameFROM Products JOIN Manufacturers

ON manufID = ID;

► This would result in all attributes from both tables that match this condition.


Natural Join Operator

► Natural join is done with the operator NATURAL JOIN. The resulting relation consists of all the combinations of tuples of the tables, where we have the same values on all the attributes with the same names.

► Attributes with the same name only appear once in the resulting relation (in SQLite they appear multiple times).

► Example: let’s query for the information of the customers together with the orders they have made:

SELECT ∗FROM Customers NATURAL JOIN Orders;

This produces a relation whose schema includes attributes are pertinent to both customers and orders and any additional attributes of the other two relations.


Outer Joins

► The outer join operator is a way to augment the result of a join by the dangling tuples, padded with NULL values.

► When using a join operation, it’s common that there are tuples with no corresponding tuple in the other relation.

► Normally, these tuples are not visible in the resulting relation. In some cases we might want to see these tuples, too. We can do this by using outer join.

► If a tuple doesn’t have a counterpart in the other relation, the attributes of the other relation have value NULL in the resulting relation.


Outer Joins: ExampleJoin the information of customers and their orders so the result contains all customers and orders, including customers that have not yet ordered anything, and the orders with no corresponding customer in the database. If we want a full outer join of customers and orders:

SELECT ∗FROM Customers NATURAL FULL OUTER JOIN Orders;

► If we want only customers without a counterpart, but no orders of unknown customers, we can do left outer join:SELECT ∗FROM Customers NATURAL LEFT OUTER JOIN Orders;

► If we want only orders without a counterpart, but no customers, we can do right outer join:SELECT ∗FROM Customers NATURAL RIGHT OUTER JOIN Orders;

Note: These queries don’t work in SQLite.


► In outer join, it is also possible to use some conditions, for example:

SELECT ∗FROM Products FULL OUTER JOIN Manufacturers

ON manufID = ID;

Note: These queries don’t work in SQLite.

Outer Joins


► Outer join with condition can also be done as a left or right sided version:SELECT ∗FROM Products LEFT OUTER JOIN Manufacturers

ON manufID = ID;includes also products without a manufacturer

SELECT ∗FROM Products RIGHT OUTER JOIN Manufacturers

ON manufID = ID;includes also those manufacturers whose products cannot be found in the database.Note: This query does not work in SQLite.

Outer Joins


Outer Joins in SQLite

► In SQLite, only left-sided outer join LEFT OUTER JOIN is implemented.

► FULL OUTER JOIN and RIGHT OUTER JOIN and all versions of NATURAL OUTER JOIN are missing.

► Notice: LEFT JOIN is the same as LEFT OUTER JOIN (also in systems other than SQLite).


Example of Outer Join in SQLite

► List the product number and name of all products found in the database together with the order numbers of the orders to which the products belong to. Also list those products that have not been ordered at all.SELECT number, prodName, orderNoFROM Products LEFT OUTER JOIN BelongsTo

ON number = productNo;Can also be written as:

SELECT number, prodName, orderNo FROM Products LEFT JOIN BelongsTo

ON number = productNo;


Aggregation Operators

► Problem: we want to calculate statistics from some attribute, for example the sum or the average of all products, or the count of those customers, who are born after 1990.

► These can be calculated with aggregation operators.► SQL has aggregation operators SUM, AVG, MIN,

MAX and COUNT.► The operand of the operators can usually be some

attribute that has a numerical value.► Except COUNT, which is simply used to count the

amount of the tuples of the query result.


Examples of Aggregation Operators► Example: let’s count the average price of all products in

the relation Products:

SELECT AVG(price) FROM Products;

► Let’s count the average price of all cameras (description ’camera’):

SELECT AVG(price) FROM ProductsWHERE description = ’camera’;

► Let’s count the number of customers who are born after the year 1990:

SELECT COUNT(∗)FROM Customers WHERE born > 1990;


► Let’s count the number of different customers in the relation Orders. One customer is counted only once, even they have several orders.

SELECT COUNT(DISTINCT custNo)FROM Orders;

► The functionality of the operation COUNT is easy to understand as follows:► COUNT(∗) will count all the rows in the query

result.► COUNT(attribute) will count those rows in the query

result, which have different values than NULL.

Examples of Aggregation Operators


Grouping

► When using the aggregation operations, it is possible to group rows with the operator GROUP BY. The average price by manufacturer can be calculated with the query:

SELECT manufID, AVG(price) FROM ProductsGROUP BY manufID;

► If we want only cameras in the query:SELECT manufID, AVG(price) FROM ProductsWHERE description = ’camera’ GROUP BY manufID;


Example of Query Execution

► Let’s execute the following query for the table below: SELECT manufID, AVG(price)FROM Products GROUP BY manufID;

Relation Productsnumber prodName description price manufIDT-33441 Samsung Galaxy A5 cellphone 250.0 S123R-55336 IPad Air 2 tablet 495.0 M554T-77445 Superstar Track M jacket 30.0 A432S-65221 Brasserie 24 pan 33.50 F542T-33442 NX 300 Smart Camera camera 399 S123T-33455 Sony Cyber-shot camera 463.0 L711P-48221 Teema 26 plate 16.00 F542T-29783 40 Smart LED-TV TV 519.0 L711T-27511 55 Smart LED-TV TV 799.90 S123



► Let’s divide the rows of the groups by the attribute given in GROUP BY:

Relation Productsnumber prodName description price manufIDT-33441 Samsung Galaxy A5 cellphone 250.0 S123T-33442 NX 300 Smart Camera camera 399 S123T-27511 55 Smart LED-TV TV 799.90 S123R-55336 IPad Air 2 tablet 495.0 M554T-77445 Superstar Track M jacket 30.0 A432S-65221 Brasserie 24 pan 33.50 F542P-48221 Teema 26 plate 16.00 F542T-33455 Sony Cyber-shot camera 463.0 L711T-29783 40 Smart LED-TV TV 519.0 L711


► Let’s calculate the desired average separately for all the groups

Relation Productsnumber prodName description price manufID AVGT-33441 Samsung Galaxy A5 cellphone 250.0 S123T-33442 NX 300 Smart Camera camera 399 S123T-27511 55 Smart LED-TV TV 799.90 S123 482.96R-55336 IPad Air 2 tablet 495.0 M554 495.0T-77445 Superstar Track M jacket 30.0 A432 30.0S-65221 Brasserie 24 pan 33.50 F542

24.75P-48221 Teema 26 plate 16.00 F542T-33455 Sony Cyber-shot camera 463.0 L711

491.0T-29783 40 Smart LED-TV TV 519.0 L711



► Let’s list the selected attributes and values of aggregation operations.

ResultmanufID AVG(price)S123 482.96M554 495.0A432 30.0F542 24.75L711 491.0

► In reality, the exact execution can be different thanwhat’s been presented, but the result is the same.



Grouping, continued

► Grouping can also be used when there are several tables in the query.

► Example: we want the total price of each order:

SELECT orderNo, SUM(price ∗ count) FROM Products, BelongsToWHERE productNo = number GROUP BY orderNo;


NULL Values with Aggregation Operations

► The value NULL is ignored in any aggregation. It does not contribute to a sum, average or count of an attribute nor can it be the minimum or maximum in its column.

► If the aggregation operator is assigned to a group with notuples, it will result in a value of NULL. The exception is the operation COUNT, which then returns 0.

► See the example in the textbook (page 281).


HAVING Conditions

► Since aggregation operators cannot be used in the WHERE clause, we can use a Having operator instead.

► Example: We want to calculate the average price of products from different manufacturers. However, we want to include only manufacturers that have at least one product that costs over 100 euros. We want to get the manufacturer ID, the average price and the name of the manufacturer.

► Example:

SELECT manufID, manufName, AVG(price) FROM Products, ManufacturersWHERE id = manufID GROUP BY manufID HAVING MAX(price) > 100;


Note:► The value of the aggregate operation in the

HAVING clause is calculated separately for each group (in the example for each manufacturer’s id).

► The HAVING section can contain any of the aggregation operators provided in the FROMclause, but also the GROUP BY attributes given in clause without aggregation operations.

Next Lecture: Defining SQL Tables and Integrity Constraints

HAVING Conditions

w5 - sql queries and aggregation › pluginfile.php › 1230613 › course › sectio… ·...

Documents