big data testing (1)
TRANSCRIPT
![Page 1: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/1.jpg)
TESTING BIG DATA
Adoniram MishraBenazeer Khan
![Page 2: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/2.jpg)
Agenda
❖ What is Big Data?❖ Hadoop Ecosystem❖ Testing Challenges❖ Test Pyramid
![Page 3: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/3.jpg)
What is Big Data?
![Page 4: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/4.jpg)
![Page 5: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/5.jpg)
What is Hadoop
![Page 6: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/6.jpg)
Hadoop Ecosystem
![Page 7: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/7.jpg)
HDFS
![Page 8: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/8.jpg)
MapReduce
![Page 9: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/9.jpg)
High Level Example
Data Source
HBase HiveData Source
Data Source
IMPORT HQL
UI
Reports
Other applications
MR
MR
MR
![Page 10: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/10.jpg)
Testing Challenges
![Page 11: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/11.jpg)
Long Feedback Cycle
![Page 12: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/12.jpg)
Data
![Page 13: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/13.jpg)
Deadlines
![Page 14: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/14.jpg)
Automation
![Page 15: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/15.jpg)
Countering Challenges
❖ Sampling.
❖ Basic testing concepts - Boundary Value testing, Equivalence class partitioning.
❖ Automation
![Page 16: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/16.jpg)
Nightly Build Tests
Hive Query Validator
Unit Tests
UI Test
Test Pyramid
![Page 17: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/17.jpg)
Hive Query Validator
Hive Query Validator
Query 1
Query 2
Query 3
Correct
Correct
Incorrect
![Page 18: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/18.jpg)
Nightly tests & Smoke Tests
SMOKE
MODULE 3MODULE 1
MODULE 2MODULE 4
![Page 19: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/19.jpg)
![Page 20: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/20.jpg)
About Us
● Adoniram Mishra ([email protected])● Benazeer Khan ([email protected])
Follow us at:Facebook: https://www.facebook.com/groups/vodqa/Twitter: https://twitter.com/vodqancr
![Page 21: Big data testing (1)](https://reader031.vdocuments.site/reader031/viewer/2022030304/5876ff101a28abf3398b6ebf/html5/thumbnails/21.jpg)