testing document - cs.bgu.ac.ilseproj/2006-2007/shaevich/testing... · web viewin case of...
TRANSCRIPT
SmartSQLTesting Document
Testing Document
Faculty of Engineering ScienceSoftware Engineering Program
Fourth Year Engineering Project
SmartSQL
SmartSQL1
SmartSQLTesting Document
Contents
1. Testing Functional Requirements..........................................................4 1.1.....................................................................Generating SQL query from NL query.
4 1.2......In case of misunderstanding SmartSQL will ask the user for clarification
with alternative word suggestions.........................................................................6 1.3. Generation of glossaries for NL analyzing process:.....................................6 1.3.1. DB schema............................................................................................6 1.3.2. DB connection......................................................................................7 1.3.3. Table conversion....................................................................................7 1.3.4. Field conversion.....................................................................................8 1.3.5. DB fields values......................................................................................8 1.3.6. Keywords db...........................................................................................9 1.3.7. Query type db.......................................................................................10 1.4. Integration of glossaries for the NL analyzing process................................10 1.5............................................... “SQLBuilder” - Building SQL queries using GUI.
11 1.6............ Extracting and Presentation of Information in case of “Select” query.
11 1.7............................................................................................ Login and access control.
12 1.8.................................................... Adding/Removing database to/from the system.
12 1.9...................................................................................... Setting of working database.
12 1.10. Add/Remove users to the system...................................................................13 1.11...................................................................................Saving NL (executed) queries.
13 1.12..............................................................................................Open saved NL queries.
132. Testing Non-functional Requirements...................................................................14 2.1. System speed......................................................................................................14
2.2. Safety & Security..............................................................................................142.3. Usability.............................................................................................................142.4. Availability.........................................................................................................142.5. Portability..........................................................................................................14
3. Test-Driven Development..........................................................................144. Random & Automatically-Generated Tests.....................................14
SmartSQL2
SmartSQLTesting Document
5. Testing The User Interface.....................................................................................156. Testing Build, Integration & Deployment.............................................................15
SmartSQL3
SmartSQLTesting Document
1.Testing Functional Requirements.
1.1.Generating SQL query from NL query.No
Test case Test Description
Expected Result Test Status
ARD Ref.
1 Testing of “simple Select” Queries.
Input a NL query that doesn’t require “join” and “where” operations and doesn’t specify any explicit fields to view.
A “Select * From (Table-name)” query is generated.
Passed 1,3,4,6,7,15b
2 Input a NL query that doesn’t require “join” and “where” operations but there are explicit fields to be shown.
Select (field-name1 , field-name2,…) FROM (Table-name)
Passed 1,3,4,6,7,15b
3 Input a NL query that requires a simple join between two tables.
Select * FROM (Table-name1) INNER JOIN (Table-name2) ON ( Table-name1.field-name = Table-name2.field-name )
Passed 1,3,4,6,7,15b
4 Input a NL query that requires join between two tables through a transaction table.
Select * FROM ( Table-name1 INNER JOIN Transaction-table ON Table-name1.field-name = Transaction-table.field-name ) INNER JOIN Table-name2 ON ( Transaction-table.field-name = Table-name2.field-name )
Passed 1,3,4,6,7,15b
5 Testing of “where” Queries.
Input a simple query that requires the “where”
Select * FROM Table-name1 WHERE Table-
Passed 1,3,4,6,7,15b
SmartSQL4
SmartSQLTesting Document
operation. name1.field-name < num
6 Input a NL query that requires AND in the “where” operation. The “where” operation consists of the same fields.
Select * FROM Table-nameWHERE Table-name.field-name > num1 AND Table-name.field-name < num2
Passed 1,3,4,6,7,15b
7 Input a NL query that requires AND in the “where” operation. The “where” operation consists of different fields.
Select * FROM Table-nameWHERE Table-name.field-name1 < num1 ANDTable-name.field-name2 > num2
Passed 1,3,4,6,7,15b
8 Input a NL query with “where” operation that includes a value
Select * FROM Table-name WHERE Table-name.field-name = 'field-name-value'
Passed 1,3,4,6,7,15b
9 Testing of “order by” Queries.
Input a simple NL query with sort operation.
Select * FROM Table-nameORDER BY Table-name.field-name
Passed 1,3,4,6,7,15b
10 Input a NL query with sort operation and Desc request.
Select * FROM Table-name ORDER BY Table-name.field-name DESC
Passed 1,3,4,6,7,15b
11 Input NL query with sort and “where” operations that refer to the same field.
Select * FROM Table-nameWHERE Table-name.field-name > num ORDER BY Table-name.field-name
Passed 1,3,4,6,7,15b
12 Input NL query with sort and “where” operations that refer to different fields.
Select * FROM Table-nameWHERE Table-name.field-name1 > numORDER BY Table-
Passed 1,3,4,6,7,15b
SmartSQL5
SmartSQLTesting Document
name.field-name2 DESC
13 Testing of “count” Queries.
Input a “count” NL query.
Select Table-name1.field-name1,COUNT(Table-name1.field-name1) FROM Table-name2 INNER JOIN Table-name1 ON (Table-name2.field-name2 = Table-name1.field-name2 ) GROUP BY Table-name1.field-name1
Passed 1,3,4,6,7,15b
14 Input a “count” all NL query.
Select Count(*) FROM Table-name
Passed 1,3,4,6,7,15b
15 Input a “count” NL query with “group by” operation.
Select Table-name.field-name, COUNT(Table-name.field-name) FROM Table-name GROUP BY Table-name.field-name
Passed 1,3,4,6,7,15b
16 Testing of “average” Queries
Input a simple “average” NL query.
Select AVG(Table-name.field-name) FROM Table-name
Passed 1,3,4,6,7,15b
17 Input an “average” NL query which includes “join” between two tables, and “goup by” operations.
Select Table-name1.field-name1,AVG(Table-name2.field-name2) FROM Table-name2 INNER JOIN Table-name1 ON ( Table-name2. field-name2= Table-name1. field-name1) GROUP BY Table-name1. field-name1
Passed 1,3,4,6,7,15b
18 Testing of “max/min” Queries
Input a simple NL query with “max” operation.
Select MAX(Table-name.field-name) FROM Table-name
Passed 1,3,4,6,7,15b
19 Input NL query Select Table- Passed 1,3,4,6,7,
SmartSQL6
SmartSQLTesting Document
with “min”, “join” between two tables and “group by” operations.
name1.field-name1,MIN(Table-name2.field-name2) FROM Table-name2INNER JOIN Table-name1 ON (Table-name2.field-name3 = Table-name1.field-name4 ) GROUP BY Table-name1.field-name1
15b
20 Testing of “sum”
Queries
Input a simple NL query with “sum” operation.
Select SUM(Table-name.field-name) FROM Table-name
Passed 1,3,4,6,7,15b
21 Input NL query with “sum”, “join” between two tables and “group by” operations.
Select Table-name1.field-name1,SUM(Table-name2.field-name2) FROM Table-name2 INNER JOIN Table-name1ON (Table-name2.field-name3 = Table-name1.field-name4 ) GROUP BY Table-name1.field-name1
Passed 1,3,4,6,7,15b
22 Testing of “Insert” Queries
Input NL query of “Insert” type. Should be of the following format:“Insert into table-name field-name1,field-name2 values val1,val2”.
Insert INTO Table-name(field-name1, field-name2) VALUES (val1,’val2’)
Passed 1,3,4,6,7,15b
23 Testing of “Update” Queries
Input NL query of “Update” type. Should be of the following format:Update Table-name set field-name1 = val1,field-name2 = val2 when field-name1 equal val3.
Update Table-name1 Set field-name1=val1, field-name2=val2 WHERE field-name1 =val3
Passed 1,3,4,6,7,15b
SmartSQL7
SmartSQLTesting Document
24 Testing of “Delete” Queries
Input NL query of “Delete” type. Should be of the following format:Erase from Table-name when field-name equal val.
Delete FROM Table-nameWHERE field-name = val
Passed 1,3,4,6,7,15b
1.2.In case of misunderstanding SmartSQL will ask the user for clarification with alternative word suggestions.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Input query with multiple meanings of a certain word.
Input query:“Give me names.”
The system will return list of fields which corresponds to the word “names” and the user can choose one of them.
Passed 2,15b
2 Input “bad” NL query
Input query:“Give me mames.”
The system will return message, that given NL query is not valid.
Passed 2,15b
1.3.Generation of glossaries for NL analyzing process:1.3.1. DB schema- fields and tables of the
database.No Test case Test
DescriptionExpected
ResultTest
StatusARD Ref.
1 Testing of creation DB scheme
Choose DB, update XML of database structure.
The system will create (or update) XML document of table and field names.
Passed 16a
2 Testing of Disconnect DB from The system will Passed 16a
SmartSQL8
SmartSQLTesting Document
creation DB scheme: connection failed
the server, choose DB and update XML of database structure.
return message, that it couldn’t connect to the DB.
SmartSQL9
SmartSQLTesting Document
1.3.2. DB connection-connections between tables.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Testing of creation DB connection
Choose DB, update XML of database connection.
The system will create (or update) XML document of table connection.
Passed 16b
2 Testing of creation DB connection: connection failed.
Disconnect DB from the server, choose DB and update XML of database connection.
The system will return message, that can’t connect to the DB.
Passed 16b
1.3.3. Table conversion- list of words that user will use for the given table of the database.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Testing addition of alternative to table name
Add alternative to the specified table, save changes, reread alternatives for this table.
The added alternative will be in the returned list
Passed 16c
2 Testing removal of alternative from table name.
Remove existing alternative from the table’s alternatives list, save changes, reread table’s alternatives.
Deleted alternative shouldn’t be in the returned list of alternatives.
Passed 16c
SmartSQL10
SmartSQLTesting Document
1.3.4. Field conversion- list of words that user will use for the given field of the database.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Testing addition of alternative to field name
Add alternative to the specified field, save changes, reread alternatives for this field.
The added alternative will be in the returned list.
Passed 16d
2 Testing removal of alternative from field name.
Remove existing alternative from the field’s alternatives list, save changes, reread field’s alternatives.
Deleted alternative shouldn’t be in the returned list of alternatives.
Passed 16d
1.3.5. DB fields values - will hold possible values for the given field in order to recognize field by the given value during NL analyzing process, only relevant for fields with close range of values.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Testing addition of new value to the field.
Add new value to the field’s values, save changes, reread field’s values.
The returned list should contain added value.
Passed 16e
2 Testing removal of existing field’s value.
Remove existing value from the field’s values list, save changes, reread field’s values
The returned list shouldn’t contain removed value.
Passed 16e
SmartSQL11
SmartSQLTesting Document
3 Testing getting list of unique values for specified field.
For specific field with known list of unique values, get the list from DB.
The returned list should be equal to the known list.
Passed 16e
SmartSQL12
SmartSQLTesting Document
1.3.6. Keywords db - holds SQL query constants that common to all databases, for example: or, and, count, sum. Will be build manually by administrator of the system using GUI.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Testing addition of new keyword.
Add new keyword to the list of keywords, reread keyword’s list
The returned list should contain added value.
Passed 16f
2 Testing removal of existing keyword
Remove existing keyword from keyword’s list, reread the list.
The returned list shouldn’t contain removed value.
Passed 16f
3 Testing addition of new value to the keyword.
Add new value to the given keyword; reread the list of values for this keyword.
The returned list should contain added value.
Passed 16f
4 Testing removal of existing value from keyword.
Remove existing value from given keyword’s values, reread the list of values for this keyword.
The returned list shouldn’t contain removed value.
Passed 16f
SmartSQL13
SmartSQLTesting Document
1.3.7. Query type db- holds list of words which represents different query types, for example: Select, Insert, etc. Will be build manually by administrator of the system using GUI.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Testing addition of new value to specified query type
Add new value to the given query type; reread the list of values for this query type.
The returned list should contain added value.
Passed 16g
2 Testing removal of existing value from given query type’s values.
Remove existing value from the list of query type’s values; reread the list of values for this query type.
The returned list shouldn’t contain removed value.
Passed 16g
1.4.Integration of glossaries for the NL analyzing process.
No Test case Test Description
Expected Result Test Status
ARD Ref.
1 Test of “Table conversion” glossary.
Input a sentence with word which associated with a certain table.
The system will return a table or a list of tables which corresponds to the word which is a part of the “Table conversion” glossary and the user can choose one of them.
Passed 3,15a
2 Test of “Field conversion” glossary.
Input a sentence with word which associated with a certain field.
The system will return a field or a list of fields which corresponds to the word which is a part of the “Field conversion” glossary and the user can choose one of them.
Passed 3,15a
3 Test of “DB Values” glossary.
Input a sentence with word which is a value of a certain field.
The system will find and assign the value to its corresponding field.
Passed 3,15a
Note: Rest of the glossaries is checked during “section 1.1” tests.
SmartSQL14
SmartSQLTesting Document
1.5.“SQLBuilder” - Building SQL queries using GUI.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Testing “Select” queries.
Testing all GUI and query variations
Correct SQL query
Passed 12,15c
2 Testing “Insert” queries.
Testing all GUI and query variations
Correct SQL query
Passed 12,15c
3 Testing “Update” queries.
Testing all GUI and query variations
Correct SQL query
Passed 12,15c
4 Testing “Delete” queries.
Testing all GUI and query variations
Correct SQL query
Passed 12,15c
5 Testing “Create” queries.
Testing all GUI and query variations
Correct SQL query
Passed 12,15c
6 Testing “Alter” queries.
Testing all GUI and query variations
Correct SQL query
Passed 12,15c
7 Testing “Drop” queries.
Testing all GUI and query variations
Correct SQL query
Passed 12,15c
1.6.Extracting and Presentation of Information in case of “Select” query.
No Test case Test Descriptio
n
Expected Result
Test Statu
s
ARD Ref.
1 Test in NL mode.
Input correct SQL query.
Correct SQL query generated and corresponding data is presented to the user.
Passed 12,15c
2 Test in “SQLBuilder”
Input correct SQL query.
Correct SQL query generated and
Passed 12,15c
SmartSQL15
SmartSQLTesting Document
1.7.Login and access control.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Testing login with correct data.
Insert correct Login and Password.
The system will permit to user work with the system and show main window.
Passed 15d
2 Testing login with incorrect data.
Insert incorrect Login and Password.
The system will return message, that user can’t login and that the login data was incorrect.
Passed 15d
1.8.Adding/Removing database to/from the system.
No Test case Test Description Expected Result
Test Status
ARD Ref.
1 Testing of Adding Database: “Access”.
Choose Database to be added (“Access” type). Add this DB to the system.
Database will be added to the current DB’s list.
Passed 9,15a
2 Testing of Adding Database: “SQL Server”.
Choose Database to be added (“SQL Server” type). Add this DB to the system.
Database will be added to the current DB’s list.
Passed 9,15a
1.9.Setting of working database.No Test case Test
DescriptionExpected
ResultTest
StatusARD Ref.
1 Testing of changing working DB.
Choose database from current DB’s list (combo box).
The system will change working database.
Passed 10,15a
SmartSQL17
SmartSQLTesting Document
1.10. Add/Remove users to the system.No Test case Test
DescriptionExpected
ResultTest Statu
s
ARD Ref.
1 Testing of Adding new user to the system.
Insert login, password, choose user role and confirm the password.
The system will add a new user to user list and add his data.
Passed 11,15a
2 Testing of Adding new administrator to the system.
Insert login, password, choose administrator role and confirm the password.
The system will add a new administrator to user list and add his data.
Passed 11,15a
3 Testing of Removing user from the system.
Choose a user to removing and remove this user.
The system will remove the user from user list and delete his data.
Passed 11,15a
1.11. Saving NL (executed) queries.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Test after queryExecution.
Saving NL query after its execution.
success Passed 13,15b
2 Test before queryExecution.
Saving NL before execution.
failed Passed 13,15b
1.12. Open saved NL queries.
No Test case Test Description
Expected Result
Test Status
ARD Ref.
1 Open query Open saved NL query.
Corresponding SQL query is opened.
Passed 14,15b
SmartSQL18
SmartSQLTesting Document
2.Testing Non-functional Requirements
2.1. System speed
Checking manually that the system responds within 2 seconds to NL query input.
2.2. Safety & Security
The system does not allow to access to unauthorized users. The test is performed manually and checks that the unauthorized user was unable to login. Only the “administrator” defined users can access the “Administrator” module.
2.3. Usability
Will be checked by default and naïve users that the system is simple to operate, and that can be learned within 30 minutes.
2.4. Availability
The system will run for one week without closing it and users during this time will use our system and we will see if the system is not crashing.
2.5. Portability
The system must work on Windows, thus the application was tested on Windows XP. The system is written in C# on .NET framework 2.0 platform therefore .NET framework 2.0 must be installed on the computer before running the application.
3. Test-Driven Development
We did not make use of the TDD development strategy because the requirements changed all the time.
SmartSQL19
SmartSQLTesting Document
4. Random & Automatically-Generated Tests
We did not perform random & automatically generated tests because in the NL module the system is interactive and users intervene needed during the analyzing process.
5. Testing The User Interface
User interface will be manually tested. The main goals of such tests are:
a. Check that all the GUI components are visible and exist.b. Check that all the GUI components are functioning well.c. Check that GUI is stable.
* In the GUI the input fields (textboxes) will be checked for valid input which is: all the digits and all letters small and capital.
There is no online help to the system but user manual will be provided.
6. Testing Build, Integration & Deployment
1. Installation process is made automatically and consists of the following steps:a. If .Net Framework 2.0 is not installed-installing .Net Framework 2.0b. If WordNet is not installed-installing WordNet.c. If Microsoft Speech SDK is not installed-installing Microsoft Speech SDK.d. Opening folder for the application and copying there all the needed files and
folders including the executable of the application.
Verification of the installation is made automatically which checks that all relevant software is installed and manually by trying to run the executable.
2. Uninstall process is made automatically and consist from:a. Uninstalling all the support software, this is done after user authorization
before each uninstall.b. Deletion of the application folder.
Verification of the uninstallation is made automatically which checks that all relevant software is uninstalled all the relevant folders deleted.
SmartSQL20