building practical analytics at the wall street journal ... · the wall street journal: rapid...
TRANSCRIPT
#AnalyticsXC o p y r ig ht © 201 6, SAS In st i tute In c. A l l r ig hts r ese rve d.
Building Practical Analytics at The Wall Street Journal: Rapid Development Using Amazon Redshift
Jeff ParkinsonVice President, Customer OperationsDow Jones
Deploying Customer Analytics in the Cloud
Jeff ParkinsonVice President, Customer Operations
Speaker Background – Jeff Parkinson
o Vice President of Customer Operations at Dow Jones (7 years)
o Over 20 years of CRM experience across multiple industries- Fidelity Investments – Director of Marketing Operations- Accenture – Marketing Transformation practice- JP Morgan – Program Management Office
o Deep usage of IBM, Oracle, SFDC and SAS operational and analytical stacks
o We explain the world and the world of businesso We are authoritative journalism and smart technologyo We provide a window on events, clarify issues, inspire new thinking and give readers and business
customers the insight they need to make informed decisionso We are newspapers, newswires, websites, apps, newsletters, magazines, proprietary databases,
conferences and more o Our premier brands include The Wall Street Journal, Dow Jones Newswires, Factiva, Barron’s,
MarketWatch and Financial News
Company Background
Product Focused to Customer Centric
Product
Customer
Organizational Paradigm Shift
Challenges - People
Product
Customer
o Customer channel data stored across
multiple external vendors and applications
o Aligning centralized data vision across
product groups
o Internal legacy systems requiring complex data transformations
o Various versions of the truth
o Competing data groups
o Lack of a true customer master record or single view of the customer
o Intuitive easy to use reporting interface
Challenges - Process
Product
Customer
o Multiple Analytics teams working on the
same analysis
o Different data stores producing different
results
o Complexity of standard data pulls require heavy data mining skill set
for simple requests
o Turn around time weeks to market as a result
o Inability to scale resource due to advanced skill set needed
Data Architecture Current State
Data Warehouse
Reporting Cubes
Usage Data
WSJ Barron’s MarketWatch
Data Mart
Legacy Systems
CRM Applications
Challenges - Technology
Data Warehouse
Reporting Cubes
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
o Legacy systems have different data models
Usage Data
Data Mart
Challenges - Technology
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o Reporting cubes hard coded and out of date
Usage Data
Data Mart
Challenges - Technology
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o No feedback from channels
Usage Data
Data Mart
Challenges - Technology
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o No connectivity between commerce and content systems (online activity
Usage Data
Data Mart
Challenges - Technology
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o Channels lack a hub to record all interactions
Usage Data
Data Mart
Data Warehouse
WSJ Barron’s MarketWatch
Legacy Systems
CRM Applications
Reporting Cubes
o Tagging suboptimal and not gathered at an individual level on a regular basis
Usage Data
Data Mart
Challenges - Technology
Personalizing Every Touchpoint
Mobile Print
Direct Mail Social Media
Telemarketing Web
Customer Service
Tablet
This project will provide the following business benefits:
- Expansion of the Data Provisioning team’s scale of resource to be applied to ad hoc requests
- Access by non-technical users to simple data visualization of key metrics
- Ability to scale resources as datamart will be accessible to all levels of data miners
- Migration to a SAS analytical stack enabling the team to increase their skill set in the world -class data tool
- Documentation of all business rules
- Enablement of vendor-supplied analysis/marketing via access to subscription data – we currently struggle to do this
o Create an accessible campaign datamart allowing quick and agile access to account-level data agnostic of source system
o Implement SAS applications to provide functionality required for segmentation, campaign management and reporting
Program Goals:
Initial Phase - SAS Datamart to build Customer Master
Key Challenges We Encountered
o Addressing lack of AWS knowledge within company
o Optimizing the movement of large data volumes
o Complexity of coordinating incremental updates versus full data overwrites (PC SAS)
o Outlining ETL process (SAS vs Lambda functions for master controller)
o Adhering to stricter data encryption requirements
o Storing Personally Identifiable Information (PII) data in the cloud
o Shifting resource mind sets entrenched in PC SAS to Server SAS
Project Scope
o Infrastructure phase was a little harder than expected
o Internal resources gaining experience with AWS and all major connection points are working together
o More Information Security hurdles than were expected
o Good news is that all delays took days not months and we now have a core team that is making rapid progress each day
o Static data set in AWS Red Shift
o Visualization being connected to data set (these cover Phases 1.1-1.2, 1.3 coming swiftly)
Next Steps
o Nightly Updates currently being worked on
o Extension of current resources
o Prioritization of Phase 2
✓ Scalable (up or down scale)
✓ Easy to administer (distribution, sort keys)
✓ Secure
✓ Good disaster recovery
✓ Integrates well with SAS
✓ Cost-efficient
✓ Easy to pick up skillset wise (SQL)
✓ Complements other components of solution S3 and EC2
Why AWS Redshift?
SAS Office AnalyticsSAS Enterprise Miner
SAS DataFlux
Possible Cloud Data Mart Configuration
x Prodx Dev
Potential Future Migration
Extract Staging BucketDelivery Staging Bucket
Data Warehouse
Conceptual Design
What does Maturity look like? - Peopleo Embedded analysis function in each key
stakeholder group to drive strategy from actionable insights
o Self-serve visualization tools to empower users such as Office Analytics and Visual Analytics
o A centralized data infrastructure and governance function producing consolidated data
o Appropriate communication and training to ensure usability and adoption
o Auditability process to ensure validity and instill user confidence
What does Maturity look like? - Process
o Encryption must be incorporated into all data transfers and when data is at rest
o Privacy Statements adjusted to allow for responsible insight gathering with subscriber permission
o All data transformations must be audited regularly
o Data Quality must become an obsession and a permanent group within the Data Management function both for new data streams and existing data stores
o Tagging and recording the correct activity data is essential (embarrassing to be wrong)
What does Maturity look like? - Technology
o Connectivity between systems must be solved
o Centralized customer record essential starting point
o Build activity and interest data on top of customer record
o Record and store historical snapshots of customer record for modeling and regression analysis
o Connect firmographic and demographic information to customer master for 360 degree view
What does Maturity look like? - Technology
What does Maturity look like? - Technology
What does Maturity look like? - Technology
What does Maturity look like? - Technology
C o p y r ig ht © 201 6, SAS In st i tute In c. A l l r ig hts r ese rve d.
#AnalyticsX