webinar: how penton uses mongodb as an analytics platform within their drupal digital platform

38
How Penton Helps Companies Win More Government Work Powered by MongoDB

Upload: mongodb

Post on 25-May-2015

1.433 views

Category:

Technology


1 download

DESCRIPTION

NorthPoint Digital worked with the Penton and MongoDB teams to deliver a MongoDB based solution, Govalytics, to serve city and county governments. We will review the design decisions made and steps taken to implement and integrate into the existing digital platform. In the session, we will review: How Govalytics fits into Penton's entire digital platform? What were the business drivers for choosing MongoDB (with Product Owner testimony) and why it was so successful? How NorthPoint Digital implemented a complete, highly interactive UX solution powered by MongoDB as part of an integrated solution and not just as a database Roadmap for the future – how the solution was designed to be independently scalable

TRANSCRIPT

Page 1: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

How Penton Helps Companies Win More

Government Work

Powered by MongoDB

Page 2: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Penton is a professional information services company. We engage and inform millions of professionals every day, helping them decide how to grow and solve their most critical business challenges. As a leading provider of workflow solutions and insightful content, marketing services and networking via in-person and online events, we help businesses do business.

We focus our industry expertise on five core growth markets: agriculture, transportation, natural products/food, public infrastructure, and industrial manufacturing and design.

Penton has many SaaS business products within its digital platform, and continues to add to their portfolio each year.

Govalytics:Public Infrastructure GroupAtlanta, GA

Who We Are:

Page 3: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Ben WalkerDirector of Brand Management, GovalyticsPenton: Public Infrastructure Group

• Responsible for product development and marketing functions related to new SaaS/DaaS products

• MID, MBA, PMP and Lean Six Sigma Black Belt product developer

• Experience in marketing, with skill sets in market research, analytics, branding, market segmentation and strategy

• Background in the design field, including disciplines such as product design, web design, UX, graphic design and furniture design

Who We Are:

Page 4: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

About NorthPoint Digital:

• Privately held and technology agnostic• Headquartered in New York City with offices in Boston and

Philadelphia• Deliver scalable content and community solutions and services:

Strategy

/ Customer Research/ Digital Strategy/ Mobile Strategy/ Multi-Channel/ Analytics

Experience Design

/ Persona Development/ Information Architecture/ Content Strategy/ User Experience + Design

- Mobile/Tablet Applications

- Responsive Design

Technology

/ Platform Strategy + Implementation/ Architecture + Solution Design/ CMS Implementation/ 3rd Party Integrations/ Analytics/ Search/ Mobile Development

Page 5: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Some of NorthPoint Digital’s Clients

Below are a few of the companies we have helped succeed in the digital arena.

Page 6: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

• 20+ years of experience across a diverse set of technologies

• Led projects for media companies, research foundations, insurance companies, hospitals, and think tanks

• Focus on scalable, cross-platform solutions

• Background in music production, UX and service design

Who We Are:

Ian MarianoProject ManagerNorthPoint Digital

Page 7: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Overview

Page 8: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

“With close ties to local governments and

extensive experience in the public infrastructure space we know that winning government work can be extremely challenging. Government work represents

$3 trillion annually and we wanted to find an easier way to identify opportunities within governments well before the bidding process. Govalytics provides marketers with the critical intelligence and workflow tools needed to navigate the complicated government sector

and win more business.”

- Gregg Herring, Vice President and Public Infrastructure Market Leader

Page 9: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Overview

Origin & Market Pain:• Through extensive experience in the public infrastructure space, know the pain

points for marketers• Have close ties to both local governments and marketers to them• Marketers are constantly looking for more efficient ways to find opportunities

within governments, but struggle to find the right kind of work within the right kind of governments

• Our group has white-labeled a RFP service to help our customers, but quickly came to realize more forward looking analytics was needed

What is Govalytics?• A sales and marketing tool that informs the user as to which governments they

should be establishing relationships with well before the issuance of a RFP• Budget analytics• Capital project data• Government official contact information• An answer to, “Which governments should I be marketing to and why?”

Page 10: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Govalytics Main Areas

1.Budgetary Data Analytics – Tracks the data analytics for each local government

• Total Expenditures, Capital Assets, Long-Term Debt and Budgets

– Current year and 5 years back, down to the department level for accepted and actual budgets providing:• Historical Trending • Projections

2.Top Government Officials– Contact information for city officials – Allows users to track who is in office and changes in

leadership

3.Capital Project Data– Project information for any current or future

project– Projected yearly expenditures for future years

within the projects

Page 11: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

8 Government Department Functions

• Public Safety• Public Works• Utilities• Planning and Development• General Government/Judicial• Education• Health and Human Services• Parks/Recreation/Culture

Government 1:Dept. of SafetyDept. of Parks & Rec

Government 2:Fire DepartmentDept. of Fun Times

Both Normalized:Public SafetyParks/Recreation/Culture

Page 12: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

New York-Newark-Jersey City, NY-NJ-PALos Angeles-Long Beach-Anaheim, CAChicago-Naperville-Elgin, IL-IN-WIDallas-Fort Worth-Arlington, TXHouston-The Woodlands-Sugar Land, TXPhiladelphia-Camden-Wilmington, PA-NJ-DE-MDWashington-Arlington-Alexandria, DC-VA-MD-WVMiami-Fort Lauderdale-West Palm Beach, FLAtlanta-Sandy Springs-Roswell, GABoston-Cambridge-Newton, MA-NHSan Francisco-Oakland-Hayward, CARiverside-San Bernardino-Ontario, CAPhoenix-Mesa-Scottsdale, AZDetroit-Warren-Dearborn, MISeattle-Tacoma-Bellevue, WAMinneapolis-St. Paul-Bloomington, MN-WISan Diego-Carlsbad, CATampa-St. Petersburg-Clearwater, FLSt. Louis, MO-ILBaltimore-Columbia-Towson, MDDenver-Aurora-Lakewood, COCharlotte-Concord-Gastonia, NC-SCPortland-Vancouver-Hillsboro, OR-WASan Antonio-New Braunfels, TXOrlando-Kissimmee-Stanford, FLGovalytics will offer data for the top 50 MSAs in Q1 2014.

Coverage: Top 25 Metropolitan Statistical Areas (MSAs)

Page 13: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Govalytics needed:

• Something scalable• We were launching with the top 25 MSAs, but would quickly be adding the

next 75

• Wanted to be able to add features without breaking the model• We were continuing to add more features and functionality

• Capital Projects• ToDos• Alerts

• We wanted to add more capabilities in the future• State data• More advanced searching and statistical analysis

• Ability to aggregate data

What We Needed: Business Drivers

Page 14: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Wireframes

Page 17: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementing an Integrated Solution

Page 18: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: Challenges

The business needs for Govalytics needed to address the convergence of:

• Building an engaging, interactive and analytics tool for the intended audience

• Delivering accurate analysis in a reliable, timely and secure manner

• Simplicity of maintenance, extensibility and content management

Page 19: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: Key Features

Some of the key features that Govalytics needed to support included:

• A highly interactive UX

• The ability to track the financial data analytics for each local government and offer five years of data providing historical trending and show projections

• Include government contact information from the mayor to the fire chief -- initially nearly 25,000 individuals needed to be in the database

• Include information for any current or future government capital project with projected yearly expenditures for future years within the projects

Page 20: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: Database

Several database technologies were evaluated according to the needs of the application and Penton operations:

• SQL including MS SQL and Oracle

• Oracle not an option (overall cost and corporate shift away from Oracle)

• Traditional SQL not flexible enough for evolving needs

• NoSQL as a solution

• Cassandra (key/value tabular hybrid)

• CouchDB (JSON document based)

• MongoDB (BSON document based)

Page 21: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: MongoDB

MongoDB was chosen as the database for several reasons that “fit” the application solution:

• The “document-like” structure of government data

• An application workload focused mainly on the retrieval of data

• No fixed schema - flexibility and extensibility in data representation

• Operational simplicity for scaling

• Built-in features that support the data analytics need such as map reduce and aggregation

• Product Maturity

Page 22: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: Architecture

Page 23: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: Architecture

• UX• ASP.NET MVC, Angular JS, HighCharts JS• Consumer of the PHP API• Proxy for Solr Searching and Filtering

• API: PHP / Drupal• Abstract away access to MongoDB• Government data import• General Administration

• Solr – Searching and Filtering the Data• DataImportHandler used to index• High performance index of the Government data• Future: Universal Search

• MongoDB• Government and other document-esque storage• Analytics (Forecasting, Aggregation, Map-Reduce)

Page 24: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: MongoDB Flexible Schema

• Governments (Cities, Counties, Towns)• Initial representation of a government

• Financial Data• Contact Data

• Extended later for• Key Statistics• Capital Projects• Government Scoring / Grading

• User Preferences• Profile preferences including default search• Saved searches• Started with Government and Function “favorites” • Added To-Dos, Notes, Alerts and Notifications

Page 25: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: MongoDB Flexible Schema

{ _id: 'ga-atlanta-city', name: 'Atlanta City', population: [], city_officials: [], total_expenditures: [], …}

Page 26: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: MongoDB Flexible Schema

{ _id: 'ga-atlanta-city', name: 'Atlanta City', population: [], city_officials: [], total_expenditures: [], … capital_projects: [], …}

Page 27: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: MongoDB Map-Reduce

Traversing the Government data to distill useful information. A simple example:

db.governments.mapReduce( function() { for (var i = 0; i < this.population.length; i++) { emit(this.population[i].year, this.population[i].value); } }, function(k, v) { return Math.min.apply(Math, v); }, { out: "calculated_population_min” });

Page 28: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: MongoDB Map-Reduce

Additional uses of Map-Reduce in Govalytics include:

• Calculation of minimums, maximums, averages and benchmarking data across

• National

• MSAs

• State

• Population Brackets

Page 29: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: MongoDB Aggregation

Used to perform additional minimum, maximum and rollup calculations. Example:

var k = 'statistic.key’;

var r = db.governments.aggregate({ $group: { _id: 0, min: { $min: k }, max: { $max: k } }});

Page 30: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Implementation: MongoDB Functions and Forecasting

• Extended MongoDB with additional functions

• db.system.js for storing reusability

• db.loadServerScripts() to make them available

• We created Govalyitcs-specific functions for:

• Forecasting data (best-fit curve projection)

• Normalizing imported data

• Processing the importing data

• Flattening data for subsequent search

Page 31: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Roadmap

Page 32: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Roadmap

Govalytics will add in the near future:

• Revenue/Expenditure analysis charts• Whitepapers section• Article/content archive• Universal search• State-based data• Resources compilation• Olark instant messaging with sales/help desk• Linkedin contact info• APIs for various users

* A product is never “finished,” but in a constant state of improvement and positioning itself in the market to better aid its users.

Page 33: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

Screenshots

Page 38: Webinar: How Penton Uses MongoDB As an Analytics Platform within their Drupal Digital Platform

How Penton Won Local Government Business

Thank You!

Ian MarianoNorthPoint Digital www.northps.com

@northps

Ben WalkerPenton | Govalytics

www.govalytics.com @Govalytics