DB Development work for Association of General

Contractors

5394 Engineering Database Design Fall 2011, Dr. Weihang Zhu

Project team: Vaibhav Chauhan, Fatemeh Hosseinzadehdastak, Ashish

Patel

Presentation Outline1. Project Description

a) What is the objective of your database? b) Who is your customer?c) What is the requirement from your customer?d) Has your application been reviewed and approved by your

customer?e) What have you done for the project overall?

2. Overview description of the system’s operation and workflow3. E-R Diagram4. Relational Schema5. Lists of the detailed Database Structure

a) Tablesb) Queries

6. Technical problems7. Conclusions and future work8. Acknowledgment9. References

Project Description - Objective

O Provide an application that can be used by the customer to meet their business needs efficiently, and in a robust manner.

O To challenge ourselves further, we parsed the customer’s existing, unstructured data (found to be in first-normal form), into SQL commands.

Project Description - Customer

O For this project, we reached out to a local membership-based trade organization that has national presence.

O The Association of General Contractors founded in 1918, and currently boasts approximately 33,000 member firms.

O Our customer, AGC of Southwest Texas, was charted in 1940 and currently has 278 member companies. The AGC’s Director, Mike White, is a believer in adopting technology in the construction industry.

Project Description - Requirements

The member companies of the AGC pay an annual membership fee, resulting the AGC’s need to track payment dates, and use this information to generate

a) An accurate list of current members (for the dissemination of information to members only),

b) A list of pending membership expirations based on last-payment date,

c) A member’s payment activity, and lastly d) A list of inactive memberships.

Additionally, the AGC wanted the capability to search their database by a) Individual nameb) Telephone number c) Fax numberd) UCI type: Each company in the database is associated with

one or many UCI codes that detail the specialties of that company. For example, HB Nield & Sons features UCI codes of 22222, 22234, 23423, 23425, meaning they specialize in various electrical skills.

Project Description - Customer

O Mr. White commissioned similar work in the recent past. O Because of the low quality of this previous work and a lack of

documentation, the database was never used by the AGC’s employees. They returned to their Excel spreadsheets.

BeforeSELECT Contacts.Company, Contacts.CategoriesFROM Contacts;

Only one Contacts Query and one Table!

Cost AGC Paid to Private Contractor in 2009: $1,500.00

Membership Management

O 90% Customer Relationship Management (CRM with membership and transaction logging).

O 10% Other Contacts (non-member):O Mayor of BeaumontO President of Chamber of CommerceO Public Figure 3O Beaumont Hotel 1

Project Description – What we did!

1. After analyzing the raw database, we provided the ER diagram

2. Relational Schema3. Normalization4. We provided the Access database5. We developed a GUI build in Visual

Studio using VB.NET

Project Description - How we did it!

O We began the process by reviewing the 52 columns of the member data provided initially by Mr. White, in first-normal form database.

O Next, the exclusion of some unnecessary and often unused fields such as HomeTel#3 and CarPhone#, resulting in 26 columns exported to a CSV.

O Then we imported the raw data into Python 3.2.2 to perform a parsing operation that resulted in over 2600 SQL INSERT statements.

O Access cannot execute multiple SQL statements in a batch, and this was unknown to us at the time. Fortunately, Dr. Zhu, suggested using MySQL as a temporary storage space for the data, and exporting the structured data for import into Access. This process was ultimately successful, thanks to Dr. Zhu, and provides the AGC with a significant time savings.

O We designed intuitive Windows forms that met the data entry and query needs of the AGC. We faced many challenges at this step.

System Operation – How it Works!!

O Primarily, the solution is a “transaction-oriented” database application, however it must act as a “display-oriented” application to deliver customer value.

O When a member company pays their annual dues, and analyst enters a transaction record with the current date.

SELECT MembershipID, OrganizationID, Expiration_date, [Last Payment Date]FROM tblTransactionWHERE (DateValue([Last Payment Date]) + 365 > NOW())

O On a monthly basis, the AGC must send out letters reminding their members to remit membership dues. Using the “Pending expiration” query, they can quickly generate a list of companies which need to be called, mailed, and emailed a reminder.

SELECT DISTINCT tblOrganization.Organization_Name, DateValue([Last Payment Date])+365 AS Expr1FROM tblOrganization INNER JOIN (tblMembership INNER JOIN tblTransaction ON tblMembership.MembershipID = tblTransaction.MembershipID) ON tblOrganization.ORGANIZATION_ID = tblTransaction.OrganizationIDWHERE (((DateValue([Last Payment Date])+365)<DateValue(Now())-30 And (DateValue([Last Payment Date])+365)>DateValue(Now())-90));

O If a non-local builder is searching for a local underwater welder, they will contact the AGC in search of a list of members that specifically mention the skill in their profile. Our application permits the AGC to record those skills by adding them to the member organization’s profile through the UCI code form and later return a list of active memberships.

E-R Diagram

Relational Schema

Normalized Relation Tables

Access Table Relationships

Project Description – Review Process

O We regularly met or communicated by email with AGC.

O Each of the forms delivered in the resulting application have been approved by him.

O He wants our team to continue development on the application to meet the needs of the AGC in the future.

O Mr. White foresees changes in local legislation that will require the AGC to retain additional data about their members, create filters on this information, and invite members based on their possession or fulfillment of certain criteria.

TablesO The resulting database features 14 tables, 10 of

which are potentially static, and 4 of which will be populated with new or edited records as the AGC utilizes the application in their operations.

O Category*, Fax_Numbers, Individuals, IndividualSkillsRelation, JobTitle, Membership, Organization, Organization_Addresses, Organization_Individual_Relation, Organization_UCI_Relation, Skills, StatesList, Telephone_Numbers, Transaction, UCIDescription

O *Category table was added by the customer very late in the process, and is not reflected in the original ER and Relational Table Diagrams.

Query 1All memberships that will expire in 30-90 days (TodayDate-(LastPaid+365 ) < 30)

SELECT DISTINCT tblOrganization.Organization_Name, DateValue([Last Payment Date])+365 AS Expr1

FROM tblOrganization INNER JOIN (tblMembership INNER JOIN tblTransaction ON tblMembership.MembershipID = tblTransaction.MembershipID) ON tblOrganization.ORGANIZATION_ID = tblTransaction.OrganizationID

WHERE (((DateValue([Last Payment Date])+365)<DateValue(Now())-30 And (DateValue([Last Payment Date])+365)>DateValue(Now())-90));

Query 2What are the email addresses for all active members

SELECT DISTINCT tblOrganization.[E-mail], DateValue([last payment Date])+365 AS [Activate Status]

FROM tblOrganization INNER JOIN (tblMembership INNER JOIN tblTransaction ON tblMembership.MembershipID = tblTransaction.MembershipID) ON tblOrganization.ORGANIZATION_ID = tblTransaction.OrganizationID

WHERE (((DateValue([last payment Date])+365)>Now()));

Query 3Inactive Members (where TodayDate-(LastPaid+365 ) < 0)

SELECT DISTINCT tblOrganization.Organization_Name, DateValue([Last Payment Date])+365 AS Expr1

FROM tblOrganization INNER JOIN (tblMembership INNER JOIN tblTransaction ON tblMembership.MembershipID = tblTransaction.MembershipID) ON tblOrganization.ORGANIZATION_ID = tblTransaction.OrganizationID

WHERE (((DateValue([Last Payment Date])+365)>Now()));

Query 4Query by UCI Code and Active

SELECT DISTINCT tbltelephone_numbers.[Phone#], tblOrganization.Organization_Name, tblOrganization.[E-mail], tblOrganization_UCI_Relation.[UIC code], DateValue([Last Payment Date])+365 AS [activate status]

FROM ((tblOrganization INNER JOIN tblOrganization_UCI_Relation ON tblOrganization.ORGANIZATION_ID = tblOrganization_UCI_Relation.[Organization_ID]) INNER JOIN tbltelephone_numbers ON tblOrganization.ORGANIZATION_ID = tbltelephone_numbers.Organization_ID) INNER JOIN tblTransaction ON tblOrganization.ORGANIZATION_ID = tblTransaction.OrganizationID

WHERE (((tblOrganization_UCI_Relation.[UIC code])=[?]) AND ((DateValue([Last Payment Date])+365)>Now()));

Query 5Query by Company Name, Telephone, FAX, or Address

SELECT tblOrganization.Organization_Name, tblOrganization.[E-mail], tbltelephone_numbers.[Phone#], tblFAX_Numbers.[FAX#]

FROM (((tblFAX_Numbers INNER JOIN tblOrganization ON tblFAX_Numbers.Organization_ID = tblOrganization.ORGANIZATION_ID) INNER JOIN tblOrganization_Addresses ON tblOrganization.ORGANIZATION_ID = tblOrganization_Addresses.Organization_ID) INNER JOIN tbltelephone_numbers ON tblOrganization.ORGANIZATION_ID = tbltelephone_numbers.Organization_ID)

WHERE (tblOrganization_Addresses.ZIP_Code = ?) OR (tbltelephone_numbers.[Phone#] = ?) OR (tblFAX_Numbers.[FAX#] = ?)

Query 6Query by Individual Name

SELECT tblOrganization.Organization_Name, tblOrganization.WEBSITE, tblOrganization.[E-mail], tblOrganization.Category_ID, tblMembership.JoiningDate, tblMembership.Billing_Information, tblTransaction.Expiration_date, tblTransaction.[Last Payment Date]

FROM ((((tblIndividual INNER JOIN tblOrganization_Individual_Relation ON tblIndividual.Individual_ID = tblOrganization_Individual_Relation.Individual_ID) INNER JOIN tblOrganization ON tblOrganization_Individual_Relation.Organization_ID = tblOrganization.ORGANIZATION_ID) INNER JOIN tblTransaction ON tblOrganization.ORGANIZATION_ID = tblTransaction.OrganizationID) INNER JOIN tblMembership ON tblTransaction.MembershipID = tblMembership.MembershipID)

WHERE (tblIndividual.First_Name = ?) OR (tblIndividual.Last_name = ?)

Technical Difficulties!!!!The storage of new information

OFirst, we attempted to programmatically access the database, yet often received an error (“passing null values”) when executing the UPDATE command. OAs an alternative, we created a newRow object and when calling the datatable.Rows.Add(NewRow) function, we experienced a different error. OSome of the errors we received when attempting to programmatically add and edit data where similar to these:

O “The record cannot be deleted or changed because table 'tblFAX_Numbers' includes related records.”

O “An INSERT INTO query cannot contain a multi-valued field.”

O “Reference to a non-shared member requires an object reference.”

OInstead, we used Visual Studio TableAdapters to maintain the Connection objects. The simplest successful implementation used the TableAdapterManager that was automatically created when dragging a table node from the Data Sources onto a form in Design View.

Technical Difficulties - Python

O We took a “crash course” in Python 3.2.2 using resources such as docs.python.org and free training videos readily available on YouTube.com.

O We abandoned sub-projects such as parsing the UCI-Organization relation.

O cursor positioning, O reading lines versus charactersO character substitutions

O However, we did successfully generate approximately 2600 lines of SQL INSERT statements using the Python environment.

O Access has no method for batch processing these statements, and thanks to Dr. Zhu, we were able to import the large volume of data into our application

The Number of Tables and Forms

18 Tables 19 Forms 115 Button

Screenshot 1 - Welcome

Screenshot 2 – AUD Org

Screenshot 3 – Individual

Screenshot 4 - Membership

Screenshot 5 – Query

Screenshot 5 – Query

ReferencesO MSDNO Python.orgO Text Processing with PythonO Developing Web-Enabled Decision

Support SystemsO Sourcecodester.comO Stackoverflow.comO YouTube.com – Python Basics, Ep1-4, by

ZJunaideenO http://www.penzilla.net/tutorials/python/

fileio/

Conclusions & Future Work

O First, the challenges we faced, and the days we endured trying to answer questions from hundreds of blogs and web-how-to’s showed us how prevalent and version-specific Visual Studio and Access are as development environments.

O How much having an ER diagram and the relational tables can help us. The practice of normalization can help us to resolve clear relationships between tables and distill binary and ternary relationships, resulting in efficient data storage and makes complex queries more stable.

O We fully expect the AGC to demand 100% functionality, and while we feel we are technically able to deliver, we will require outside assistance.

O Future development projects in Objective-C and Java this winter.

Acknowledgment O The teammates for their many 4am nights and dedication to

understanding hierarchical data stability, database connections, parsing, creating SQL statements in-code, and teamwork.

O Like to acknowledge the patience of Mike White, the Director of the AGC for his co-operation and willingness to provide his time and experience as we learned the business processes involved in operating a membership-based organization.

O Dr. Zhu for his lectures and for providing us with the benefit of his experience. We would especially like to thank him for his attention and assistance in solving problems we faced with executing batch commands output by Python in MySQL, and importing the resulting data into Access.

O We would like to thank the rest of the class for their communication and assistance throughout the process. Learning from your challenges and solutions gave us a competitive motivation, but helping each other solve problems made us feel our class has the potential to create great solutions.