Download - How we Test at eBay Europe
![Page 1: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/1.jpg)
How we test at eBay
Michael Palotas Head of Quality Engineering Europe Zürich, 17 August 2011
![Page 2: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/2.jpg)
Produced by: Michael Palotas
The Company
2
![Page 3: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/3.jpg)
Produced by: Michael Palotas
eBay Inc.
3
![Page 4: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/4.jpg)
Produced by: Michael Palotas
Setup
4
EU
Quality Engineering
(Zürich, London,
Berlin, Paris)
Global Product
San Jose
USA
Global Product
Shanghai
China
Global Product
Chennai
India
Local Product
Zürich, London
Mobile
Portland USA
Adjacencies (i.e. Brands for Friends)
![Page 5: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/5.jpg)
Produced by: Michael Palotas
Not only the Web
5
Web
Windows Desktop
Mobile
![Page 6: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/6.jpg)
Produced by: Michael Palotas
Landscape
eBay Inc 6
90.000 development days / year
Continuous Integration not consistent
1000+ new bugs found every week
Teams all around the globe in all possible time zones
Major pieces outsourced to India + China
300 test engineers around the globe
Unstable test environment
![Page 7: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/7.jpg)
Produced by: Michael Palotas
Waterfall...SCRUM…and everything in between
7
![Page 8: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/8.jpg)
Produced by: Michael Palotas
Testing at eBay in the past
8
Very manual
Manual regression testing for 22 sites, every 2 weeks
Teams did not rely on automation
Skill set of QA Engineers mainly non-technical
Sources: www.linesofflight.net www.generallyAwesome.com
![Page 9: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/9.jpg)
Produced by: Michael Palotas
Testing Today – Ownership of Quality
9
Manual Test
Automation
Consulting +
Coaching
![Page 10: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/10.jpg)
Produced by: Michael Palotas
Manual Test
Still very important, mainly exploratory type
All new features are tested manually first
Edge cases
Not automatable tests
Manual testing still yields the highest number of bugs
10
![Page 11: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/11.jpg)
Produced by: Michael Palotas
Automation
Automation is not about eliminating jobs !!!
Save time and reinvest into manual testing
Regression Testing (fully) automated
Programmatic approach (JAVA)
Open source only (WebDriver/Selenium 2 GRID)
3500 end to end testcases running in less than 1 hour 11
![Page 12: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/12.jpg)
Produced by: Michael Palotas
GRID
12
Para
llel
exec
utio
ns
Test 1
Test 2
Test 3
Test …
Test …
Test3500
Execution Time
Sequential Execution
Execution time 7000 minutes (~116 hours = ~5 days)
eBay Europe: - 3500 e2e regression testcases - 2-3 minutes execution time (each)
Test
Test
Test
Test
Test
Test
Execution Time
Para
llel e
xecu
tions
Parallel execution in Grid
Test
Test
Test
Test
Test
Test
Test
Test
Test 150 Virtual Machines / Browsers in
the WebDriver GRID Execution time: 60 minutes
![Page 13: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/13.jpg)
Produced by: Michael Palotas
Coaching / Consulting
EU team conducts project audits
Improvement initiatives
Consulting for dev and test teams
13
![Page 14: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/14.jpg)
Produced by: Michael Palotas 14
Software Measurement - SONAR
![Page 15: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/15.jpg)
Produced by: Michael Palotas
Project Audits
15
![Page 16: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/16.jpg)
Produced by: Michael Palotas
EU Quality Engineering Team – Automation Impact
16
.00% 2.00% 4.00% 6.00% 8.00%
10.00% 12.00% 14.00% 16.00% 18.00% 20.00%
1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13
Week
Team % of global Bugs 2009 Start of automation efforts in early 2009
" Team makes up for only 2% of global test engineer headcount
" Clear correlation between global bug contribution and automation investment / coverage
headcount share
headcount share
Contribution Trendline
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
16.0%
18.0%
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51
Week
Team % of global Bugs 2010
headcount share
![Page 17: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/17.jpg)
Produced by: Michael Palotas
Bug Quality
17
Bug Quality = real bugs / submitted bugs
75
80
85
90
95
100
Q1 08 Q2 08 Q3 08 Q4 08 Q1 09 Q2 09 Q3 09 Q4 09 Q1 10 Q2 10 Q3 10 Q4 10
Bug Quality in % (2008-2010)
Bug Quality 2008-2010
Start of automation efforts
![Page 18: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/18.jpg)
Produced by: Michael Palotas
eBay checkbox – vs. HTML checkbox
18
<div align="center”> <input type="checkbox" name=”myName" value=”eBay"> eBay<br> </div>
![Page 19: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/19.jpg)
Produced by: Michael Palotas
Summary
Testing is changing
Great career opportunities in testing
More technical skill sets required
Testautomation can have a big impact
….
or make your life hell
Essential to work with developers
Measure and improve
19
![Page 20: How we Test at eBay Europe](https://reader034.vdocuments.site/reader034/viewer/2022051816/547388bab4af9fc30a8b52eb/html5/thumbnails/20.jpg)
Produced by: Michael Palotas
C o n t a c t : m p a l o t a s @ e b a y . c o m
THANK YOU! QUESTIONS?
20