Big Data Solutions in the Real WorldB15 - PHẠM MINH MẪNIU-ISS

Content◦Big data introduction( video)◦The importance of Big data to business◦Analyzing data in motion: a real-world view◦Improving business processes with big data analytics: a real-world view

◦Summary◦References◦Q&A

The importance of Big data to business◦Big data problems◦Big data as a Business planning tool◦New dimensions to the planning cycle◦Keeping data analytics in perspective◦Getting started with the right foundation◦Planning for Big Data

Big data problemsFactors make businesses not be able to get the most value from Big data◦ Expense if purchasing systems and storage to contain data◦ Manage, back up, query very huge database◦ Performance – manage data at the right speed◦ Difficulty of programming to integrate data and maintain code◦ Complexity of keeping data up-to-date and emerging business requirement

Big data as business planning tool◦ Big data is considered as way to plan and execute

for business strategy◦ Challenge for business is ability look into future

and anticipate what might change and why◦ Planning with data:

◦ Understand how data sources are related◦ Determine what data is needed to plan for new

strategies and new directions◦ Case study: Company want expand service offering to

existing customer

Big data as business planning tool(Cont.)◦ Doing the analysis:

◦ Applying big data will be able to identify business nuances and changes that can impact bottom line of business(net profit)

◦ Example: e-commerce company wants to see reaction to the new services on social media sites, understand what competitors are offering could impact revenue

◦ Checking the results:◦ Does analysis reflect business outcome? Is the using data accurate enough or do problems

exist? Are data sources going to truly help with planning?◦ Make sure not rely on data sources taking in the wrong direction◦ When planning and making business decisions based on analysis, make sure that you are on

a strong foundation

Big data as business planning tool(Cont.)◦ Acting on the plan:

◦ When business initiates new strategy, it is critical to constantly create a big data business evaluation cycle

◦ This will based on result of big data analytics and testing the result of executing business strategy verify strategy is working as intended

◦ Sometimes, results do not match expectations, this will mean resetting the strategy.◦ Or lead a company in new direction might have better outcome

New dimensions◦ Monitoring in real time:

◦ Big data analytics enables to monitor data in near real time proactively◦ Adjusting the impact:

◦ Monitor quickly, change process early, and result better overall quality◦ Find out why a problem happened, why product failed or why service did not satisfy

customer able to avoid mistake◦ Enabling experimentation:

◦ Trying out new product and service is not without risk.◦ Combining experimentation with real-time monitoring and rapid adjustment can transform

business strategy◦ Can change directions and outcome more easily when arming right data less risk

Keeping data analytics in perspective◦ Predictive analytics makes company to understand small and

subtle changes in customer buying patterns can make changes in strategy earlier

◦ It is dangerous to rely too much on data alone◦ Make sure that business leaders do not trust the result of big data

analytics isolation from other factors◦ Business leaders bring intuition and knowledge to the table◦ Before assume big data is the panacea for all business strategy

issues, make sure taking balanced approach

Getting started with right foundation◦ Understand various types of data that are important to

organization◦ Understand new types of data management environments◦ Building right team of individuals with both technical and business

knowledge such as business leaders who involve in planning strategy, consultants who have experience working on big data,…

◦ Make sure that metadata is consistent◦ Progress with the analytics that have always been doing

Planning for big data◦ Every industry has capability or potential to collect and analyze

data to improve business outcomes◦ Companies use social media data to improve business planning

and execution◦ Analyzing in all different types of structured and unstructured data

to be able to predict outcomes◦ Big data has the potential to help companies get a handle on both

risk and opportunities in the best way

Analyzing data in motion◦Data in motion◦Why companies need data in motion◦Environment impact◦Public policy impact◦Health impact◦Energy impact

Data in motion◦ Limitation placed on problem-solving techniques: lack of data, limitation on

storage or computer power, high cost◦ Data in motion – Streaming data: type of information from social media, news

or stock market data feeds, log files, spatial data from sensors, medical device data, or GPS data, constantly in motion

◦ Streaming data provides a way to understand an event at the moment it occurs, add new insight to some challenging question because of the immediacy of the knowledge

◦ Data in motion is constantly flowing across various interconnected channels of communication

◦ Data is in motion when it is transit from one resting location to another

Why companies need data in motion◦ Companies need data in motion to react quickly to the current

stage of data.◦ Case studies:

◦ Sensors are connected to highly sensitive medical equipment to monitor performance and alert technicians of any deviations from expected performance. Streaming data ensure that technician receive information about potential faults to correct before harming patients

◦ Telecommunication equipment ensure service levels meet customer expectation◦ Information from sensors in a security-sensitive area can differentiate between movement

of harmless rabbit and car moving rapidly toward a facility

Why companies need data in motion(Cont.)◦ Data in motion is important to make decisions when speed is critical factor.

The challenge with streaming data is extracting useful information before it come to resting location

◦ Processing and analyzing streaming data in real time can react to the current stage of the data

◦ Need to have clear understanding of nature of streaming data and what result are looking for

◦ However, a lot of streaming data doesn’t mean that will solve problem◦ Using streaming data to improve outcomes for customer, patients, city

residents, and to influence customer decision making at the point of sales

Streaming data with environmental impact◦ Change in natural environments can have enormous impact on the economic,

physical, and cultural well-being of individuals and communities around the world

◦ Improving to predict environment impact, scientists are beginning to include analysis of data in motion in their research such as collection of large volumes of time-sensitive information about water resource, weather

◦ Mathematical models are used to make predictions to against disasters impacting natural resources

◦ Many research programs are in place to improve understanding of how to protect natural water resource using data in motion to analyze newly acquired data with historical information

Streaming data with environmental impact(Cont.)◦ Using sensors to provide real-time information about rivers and

oceans such as physical, chemical, and biological data from rivers◦ Sensors monitor spatial changes in temperature, pressure, salinity,

turbidity, and chemistry of water◦ Creating real-time monitoring network for rivers will be able to predict

changes in the river as same as weather predictions◦ Others use radio-equipped buoy containing sensors to collect data

about ocean( wave height and action) provide real-time ocean conditions to fisherman and marine researchers

Streaming data with environmental impact(Cont.)◦ Benefit of real-time data:

◦ Understand of major ecological challenges. Scientists may looking at data they may have in the past in new way and be able to collect new types of data sources

◦ Analyzing streaming data is possible to pick up on patterns that have missed so far

◦ Capability to do more in-depth analysis and better job of predicting future outcomes if combining collected data with real-time data

Streaming data with public policy impact◦ A huge amount of data ( tax records, sensors on building and bridges, traffic

pattern monitoring, location data, data about criminal activity, annual census data, police records) are collected in various city agencies separately and not easy to be shared across the city and it surrounding communities, even within one specific agency ( police department) to make real-time decision to improve city life

◦ The problem is that data is managed and stored in separate silos ◦ change can only happen if policy maker can use available data and data from

best practice◦ Improving city traffic congestion analyze both unstructured and structured data to assess current travel

conditions and make suggestions on alternative routes that will cut down on traffic◦ A significant impact on lower crime rates in the cities Analyzing changes in criminal behavior patterns

Streaming data with health impact◦ Big data is used from genetic research to advanced medical imaging and

research on improving quality of care◦ Major benefit of Big Data is applying practically and quickly at the right time to

clinical medicine to help save lives, to speed decision making of clinicians and researchers, and improve healthcare outcomes for patients

◦ Devices provide large amount time-sensitive data( results of lab tests, X-rays, digital imaging, blood pressure, heart rate, temperature) to monitor patient’s vital sign, alert doctors when readings go out of normal range

◦ It’s hard to pick up subtle changes in patient’s conditions with physical exam, but could be done by monitoring devices if a way existed to have more immediate access to the data

Streaming data with health impact(Cont.)◦ Monitoring devices used in intensive care units generate thousands

of readings every seconds, but because of technology limitation, much of those data was not available for analysis in the past

◦ Using streaming technology is able to capture data stream from bedside monitors and to look for early warning signs of serious infection

◦ Data is used in real time to provide early warnings of changes in patient’s condition doctors can take corrective action to help patient almost 24-36 hours earlier, and compare the analysis to database of patient outcomes that could provide additional insight

Streaming data with energy impact◦ Large volumes of data in motion are increasingly being monitored

and analyzed in real time to help reduce energy consumption, find new sources of renewable energy, and increase energy efficiency protecting environment and sustaining economic growth

◦ Many large organizations use streaming data to measure and monitor energy demand and supply to improve their understanding of their energy requirement, and make real-time decision about energy consumption

Streaming data with energy impact(Cont.)◦ Organizations are beginning to use streaming data to increase

energy efficiency:◦ Monitoring streaming data on its energy consumption and integrate it with

weather data to make real-time adjustments in energy use and production◦ Sharing and analyzing streaming energy use data to consume energy more

efficiently and reduce energy cost while ensuring that changes in demand are anticipated and kept in balance with supply

Streaming data with energy impact(Cont.)◦ Organizations are beginning to use streaming data to help advance

research and efficient production of alternative energy sources:◦ Research institution is using streaming data to understand viability of using

wave energy as renewable energy( using temperature, geospatial data, moon and tide data, to monitor and analyze noise made by wave energy technology and its impact on fish and other marine life)

◦ Wind-farm company collects turbine data, temperature, barometric pressure, humidity, wind direction,… to create prediction about energy

production. The resulting analytics are used to select the best location for wind turbines and to reduce cost per kilowatt- hour of energy produced

Improving business processes◦Why companies need big data analytics◦Customer experience with text analytic◦Big data to determine next best action◦Preventing fraud◦Business benefit of integrating new sources of data

Why companies need big data analytics◦ If the business leaders can quickly analyze information that is

unstructured( either in the form of text from customer support system or social media site), they can gain important insight, huge revenue increase

◦ leveraging a combination of structured and unstructured data as part of business process can transform business’s capability to be agile and nimble, and profitable

Customer experience with text analytic◦ Unstructured data is the text found in email, text message, call center notes, comments in

surveys, tweets, blogs, and facebook◦ This type of data represent 80% of data available to companies, and continuing to grow◦ It takes a long time( an hour, a day, a week,..) to review manually, and in many companies it

has never have been enough analyzed◦ If this data is analyzed at the right time it may help to identify customer dissatisfaction or a

potential product detect so that right action can be taken before it’s too late◦ Text analytic is the process of analyzing unstructured text, extracting relevant information,

and transforming it into structured information leverage in various ways◦ Case study: Car rental company wants to compete with emerging companies by analyze

feedback of customer on its service in online surveys or by email or text. They change from manual analytic into text analytic solution to improve their service and customer satisfaction as well

◦ provide managers with an early identification of problems, and they will be able to make changes and improve customer satisfaction before it gets worse

Determine next best action◦ Big data is secret weapon for business to make prediction and take next best action in highly

competitive environment◦ Using analytics to understand customer requirements and catch up those one◦ Increase company understanding of each customer’s unique needs. Provide these in-depth customer

insights at the right time to make them actionable◦ Improve responsiveness to customers at the point of interaction◦ Integrate real-time purchase data with large volumes of historical purchase data and other sources of

data to make a targeted recommendation at the point of sale◦ Provide customer service representatives with the knowledge to recommend the next best action for

the customer◦ Improve customer satisfaction and customer retention. ◦ Deliver the right offer so that it is most likely to be accepted by the customer ◦ Predict best action very quickly to be success in this fast-paced mobile world

Determine next best action◦ Predicting the next best action often requires the use of sophisticated

machine-learning algorithms from a cognitive computing environment◦ Examples:

◦ Insurance company wants to increase efficiency and effectiveness of its call center representatives. Agents need to identify top customer for special attention, solve problem quickly and meet customer satisfaction. They transform conversations from recorded calls in to text, identify and analyze key word, combine historical data to identify high priority customers who require immediate attention and deliver more timely and appropriate response to all customers

◦ Global bank wants to shorten time taking to access customer information. The bank implemented a big data analytics solution that improves the way its representatives support customers by providing them with an early indication of each customer’s needs before they got on the phone. The platform uses social media data to understand relationships and can determine whom the customer connected to. The solution combines multiple sources of data, both internal and external. As a result, agents are able to take the next best action such as agent might consult customer to discuss college loan for their children when customers have children ready to graduate from high school

Preventing fraud◦ Insurance companies want to stop fraud early before they get involved in the processing of

the claim. By developing predictive models based on both historical and real-time data, companies are in a better position to identify suspected fraudulent claims in the early stages of interaction

◦ Big data analysis can quickly look for patterns in historical claims and identify similarities or bring up questions in a new claim before the process gets too far along

◦ Big data analytics have the potential to deliver a huge benefit by helping to anticipate and decrease attempted fraud

◦ The company implemented a big data analytics platform to provide the integration and analysis of data from multiple sources. The platform incorporates extensive use of social media data and streaming data to help provide a real-time view. Call center agents are able to have a much deeper insight into possible patterns of behavior and relationships between other claimants and service providers when a call first comes in

Business benefit of integrating new sources of data◦ By integrating new sources of unstructured data such as web logs,

call center notes, e-mails, log data, and geospatial data with traditional sources of transaction, customer, and operational data, companies can look at their businesses much differently.

◦ They can gather data they were not able to collect previously and use this data to look for patterns of behavior that provide a great insight to the business.

◦ Integrating all these sources of data provides a way for companies to deepen their understanding of customers, products, and risk.

Summary◦Review big data utilization models◦Use big data to detect fraud◦Integrate social media feedback into corporate decision making

◦Understand data in motion in some impact◦Big data help to predict and determine next best action

References◦ Judith Hurwitz, Alan Nugent, Dr. Fern Halper, Marcia Kaufman, 2013, “Big Data for Dummies”, Part VI,

“Big Data Solution in the Real World”, pages 235-262