data foundation for analytics excellence

26
Data Foundation for Analytics Excellence Predictive Analytics & Business Insights – 2015 , Chicago Mudit Mangal Project Lead, Data Analytics, Supply Chain Sears Holdings Corporation 06/11/2015

Upload: mudit-mangal

Post on 09-Feb-2017

18 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: Data Foundation for Analytics Excellence

Data Foundation for Analytics Excellence

Predictive Analytics & Business Insights – 2015 , Chicago

Mudit MangalProject Lead, Data Analytics, Supply Chain

Sears Holdings Corporation

06/11/2015

Page 2: Data Foundation for Analytics Excellence

AgendaWHAT IS HAPPENINGWHAT IS DATA ANALYTICS AND ITS CHALLENGES WHY DATA FOUNDATION HOW TO APPROACH WHY DATA GOVERNANCEWHAT ARE SECURITY ISSUES WITH DATA USE CASEWHAT IS IN FUTUREQUSESTIONS

Page 3: Data Foundation for Analytics Excellence

New Data FrontiersFueled by growing demand for anytime anywhere access

to information, technology is disrupting all areas of enterprise, driving myriad opportunities and challenges.

Enormous opportunities exist for enterprises to take advantage of connected devices enabled by the “Internet of Things” to capture vast amounts of information.

Digital transformation is changing business models –pricing strategies, processes, relationship between businesses and customers.

Declining PC usage and increasing mobile device adoption is driving a “mobile first” world.

However, the evolution of the digital enterprise also presents significant challenges, including new competition, changing customer engagement and business models, unprecedented transparency, privacy concerns and cybersecurity threats.

Page 4: Data Foundation for Analytics Excellence

Digital Frontiers

Page 5: Data Foundation for Analytics Excellence

Understanding Analytics

Page 6: Data Foundation for Analytics Excellence

360-degree view360-degree view of all the data is important to know what’s happening in a marketplace—the combination of structured information, human interactions, and machine-to-machine data.

Page 7: Data Foundation for Analytics Excellence

360-degree view in practice In IT operations management, a 360-degree view allows to

see logs, performance ,it also lets you see what the test team said to the app dev team, what the customers said to the help desk, and what the app support team said to the help desk.

In security management, a 360-degree view lets you see security alerts from the network, applications, and infrastructure, while human interaction data allows you to see security threats in emails.

In retail, a 360-degree view allows you to analyze sales in stores and online, as well as understand consumers’ expectations ,social sentiment regarding the store. What do people think about your service compared to that of your competitors?

Page 8: Data Foundation for Analytics Excellence

By its definition : “Data that was previously ignored because of technology limitations” examples includes unstructured data that companies have struggled to analyze in the past, documents, social data, customer surveys, web logs, and a lot of ‘dark’ structured data.

Dark Data

Page 9: Data Foundation for Analytics Excellence

Data Torturing Are you still torturing your data to get what you want?

Page 10: Data Foundation for Analytics Excellence

Data FoundationData is the foundation of all information solutions, BI and

analytical decisions and choosing the right technology is important.

The data foundation encompasses the integration of data from multiple, disparate sources into a trusted, understandable form for use in analytics and making data as an enterprise asset.

The escalating volume, variety, and velocity of information that is being generated today present with many critical challenges.

However, this overabundance of information can be an important asset to those organizations that choose to capitalize on it.

Page 11: Data Foundation for Analytics Excellence

Components of data foundation

Page 12: Data Foundation for Analytics Excellence

Benefits of robust data foundationA robust data foundation provides an organization with tremendous benefits in terms of efficiency and effectiveness in decision making.

One-stop shopping for data: Most significant uses of time in decision making is getting the data in usable format. A robust data foundation changes the 80 percent time spent on gathering to 80 percent time spent on analyzing the data.

Single version of the truth: Getting different answers to same questions is frustrating experience for decision makers. A robust data foundation provide single version of truth on which everyone can rely.

Drives common understanding across the enterprise: One of the key objectives of data foundation is to integrate data from disparate sources. A robust data foundation provides the structure and enforcement of these, resulting everyone in organization working on same page.

Page 13: Data Foundation for Analytics Excellence

Consequences of the lack of a robust data foundation Multiple answers to same questionsMaking less optimal business decisionsWasted time finding, collecting, summarizing data for use

in analytics

Page 14: Data Foundation for Analytics Excellence

Getting Started: Building OutAnalyzing data often requires a transformational approach

to many critical IT processes. Ask yourself what data points do I need, how I am going to

get them, and what am I going to do with them once I have them ?

Try building a Distributed platform that is small, low cost, fluent in all forms of data and analytics. E.g. data in motion.

Next, identify a low impact use case for implementation.Your application should be a good candidate for Distributed

Data Computing. If so, a successful POC will be assured.

Page 15: Data Foundation for Analytics Excellence

From Data Lakes To Data Swamps“By its definition, a data lake accepts any data, without governance. Without metadata and a mechanism to maintain it, the data lake risks turning into a data swamp and leads to hardest problem of data quality.”

Page 16: Data Foundation for Analytics Excellence

Big Data without GovernanceDumping data into Big Data Lake without repeatable

processes and data governance will create messy, uncontrollable data environment.

Insights harvested from ungoverned data lake is not reliable and trustworthy, so cannot make business decisions confidently.

In an industry where data is the most valuable asset, data integrity is essential. If the data is compromised, it can have vast consequences.

Data must be physically safe. Whether data is stored internally or within the cloud, Disaster recovery, security and other actions must be taken to ensure the physical integrity of data.

Humans make mistakes. Maintaining data integrity is difficult when humans enter free-form text into software systems.

Page 17: Data Foundation for Analytics Excellence

Governance Disciplines

Page 18: Data Foundation for Analytics Excellence

Evolving Data Governance

Page 19: Data Foundation for Analytics Excellence

Security Risk for Big DataAs cyber threats continue to multiply, it is becoming harder

to safeguard data, intellectual property, and personal information.

Greater use of the internet, smartphones and tablets in combination with bring-your-own-device policies has made organizations’ data more accessible and vulnerable.

More data implies higher risk of exposure.New data types may give rise to new security breach

scenarios.

Page 20: Data Foundation for Analytics Excellence

Data Lake Security Solutions

Page 21: Data Foundation for Analytics Excellence

Evolving Data SecurityApache Knox : Perimeter/ Network SecurityApache Ranger : Data Protection, Authorization, Audit

trackingApache Sentry : Authorization

Page 22: Data Foundation for Analytics Excellence

Retail Use CaseLet’s face it: when it comes to giving business users the information they need, retail is as tough. With multiple stores, myriad, ever-changing products and constant transactions, every day is a new challenge.

Most retailers already have systems in place to provide business users with information. The question is, how to do better?

Web-based retail analytic applications extend existing business reporting and analytic solutions. Helps in understanding Customers.

Sears has a very intensive big data program to drive customer loyalty, Sears is doing amazing things with technology and Competes On Big Data.

Page 23: Data Foundation for Analytics Excellence

Predicting Future

Page 24: Data Foundation for Analytics Excellence

Future of DataWhat kind of data will be Big Data in the future? Structured data - This is the data companies store today:

sales transactions, maintenance details. Since Big Data technology allows us to store more data and analyze it much faster, there will be increase in amount of details stored ,in the time period for which data is kept.

Human interaction data - Unstructured data refers to the data of human interactions: emails, phone conversations, video, pictures, documents, social media interactions on Twitter, Facebook and other communities. This type of data represents 90 percent of useful data.

Machine to machine data - By 2020 there will have been a massive increase in the number of connected smart devices. Cooktops, shopping carts, home thermostats, cars, bicycles, and refrigerators will run applications that connect to the emerging Internet of Things. These devices will generate huge amounts of data. Collecting and analyzing this data will lead to new insights.

Page 25: Data Foundation for Analytics Excellence

ConclusionReporting and Analytics can be transformational for an

organization. However, having the proper data foundation that provides trusted, well-integrated and well-managed data is essential to realize the desired reporting and analytical capabilities.

Mapping out a strategy and plan to establish the data foundation is time well spent and will provide a return many times over.

Page 26: Data Foundation for Analytics Excellence

Questions?

www.linkedin.com/pub/mudit-mangal/25/737/931

[email protected]