От хаоса к автоматизации тестирования на примере backend
Post on 09-Jan-2017
147 Views
Preview:
TRANSCRIPT
1CONFIDENTIAL
Testing of the backend systemOctober 15, 2016
FROM CHAOS TO TEST AUTOMATION
2CONFIDENTIAL
ROMAN SOROKA• Lead test automation engineer in EPAM
• Activist of COMAQA.by community• Over 7 years in IT.
• Relocated to Minsk from Moscow
• Roles in the career: developer, software engineer in test, QA lead, delivery manager.
3CONFIDENTIAL
I am from Sarov
4CONFIDENTIAL
Why Minsk
Minsk =
= Moscow
5CONFIDENTIAL
Why Minsk
Minsk =
= Moscow
6CONFIDENTIAL
Why Minsk
Minsk =
= Moscow
7CONFIDENTIAL
Why Minsk
Minsk =
= Moscow
8CONFIDENTIAL
Why Minsk
Minsk =
9CONFIDENTIAL
Why Minsk
MinskEPAM =
>4000epamers
11CONFIDENTIAL
The ground rules
25 minutes 15
minutes
12CONFIDENTIAL
1 2 3 4
Agenda
The first steps on the project
Dealing with chaos Selling the test automation
The framework architecture inside
out
13CONFIDENTIAL
Intro
14CONFIDENTIAL
Intro
15CONFIDENTIAL
The first steps on the project
16CONFIDENTIAL
Expectation vs Reality
17CONFIDENTIAL
Expectation vs Reality
18CONFIDENTIAL
EXPECTATION REALITY
Expectation vs Reality
Clear workflow
Specified acceptance criteriaUsable documentation
Good working autotests
19CONFIDENTIAL
•Ask questions•Confirm answers or their absence•Fill the knowledge base
Solutions
20CONFIDENTIAL
Knowledge transfer
Find out what was expected from your predecessor!Make sure that Customer knows how it was done!
21CONFIDENTIAL
Ask questions
22CONFIDENTIAL
Get confirmation
23CONFIDENTIAL
Ask questions again
24CONFIDENTIAL
Summary for the first steps
•Gather knowledge as many as possible!
•There are no too many stupid question to ask!
25CONFIDENTIAL
Kb.e
pam
.co mThink
about yourself in the future and other people!
Fill the knowledge base
26CONFIDENTIAL
Dealing with chaos
27CONFIDENTIAL
•Earn trust and respect•Make friends•Find out the politics behind•Start offering incremental improvements
Solutions
28CONFIDENTIAL
Earn respect
Do some simple and required task much better then it was done before!Use it as an example of good work!
29CONFIDENTIAL
Earn respect
Get known for your knowledge and understanding of the job
30CONFIDENTIAL
Earn trust
Help out people if you can!It will earn their trust.
31CONFIDENTIAL
Make friends
Invest in good relationships! They help in achieving all goals!
32CONFIDENTIAL
Politics behind
33CONFIDENTIAL
Find allies
34CONFIDENTIAL
Find allies
Find ones who can be trusted.
35CONFIDENTIAL
Find out why the things are in the current state of affairs!
Find out reasons
36CONFIDENTIAL
Find out reasons
We need to show the Client that we can achieve the expected results!
37CONFIDENTIAL
Incremental improvements
Propose to improve some small thing which will give the meaningful value!
38CONFIDENTIAL
Incremental improvements
39CONFIDENTIAL
Incremental improvements
40CONFIDENTIAL
Selling test automation
41CONFIDENTIAL
•Ask advices•Gather facts•Calculate benefits•Present the proposition to the customer
Solutions
42CONFIDENTIAL
Ask advices
43CONFIDENTIAL
• The number of defects detected in production in Jan-Jul 2016 is twice higher than in the same period 2015.
• The ratio of defects detected in production amounts to 31.5%. (The industry recommended value is less than 10%)
• The total sanity test coverage amounts to 34%. (The value recommended by customer standards is above 80%.)
• The immediate test automation technical debt is more than 1000h.
• The development team spends up to 30% on the bug fixing per iteration.
Gather facts
44CONFIDENTIAL
Calculate benefits
Time to implement the suggested TAF
Elimination of the technical debt
Time savings with the new TAF
1 key tester works 64 hours per iteration in the ideal time in both cases for the elimination of the debt only
Existing TAF 25
i1 i3 i5 i7 i9 i11 i13 i15 i17 i19 i21 i23
Suggested TAF 5 7
13
i1 i3 i5 i7 i9 i11 i12 i13 i15 i17 i19 i21 i23
45CONFIDENTIAL
Present proposition to the customer
46CONFIDENTIAL
Be flexible
47CONFIDENTIAL
The framework architecture inside out
48CONFIDENTIAL
BACK-END. What’s that?
49CONFIDENTIAL
•Allow the computer to do the math for you•Collaborate with developers•Use the cases as the documentation•Make it easy for other people to use•Create easy readable results of testing
Solutions
50CONFIDENTIAL
Allow the computer to do the math for you
Create a sanity test caseCreate all positive requests with
required fields
Create all negative requests without required fields
Create all positive requests within boundary conditions
Create all negative requests outside boundary conditions
Create all positive requests with correct type of arguments
…
51CONFIDENTIAL
Use TAF to generate all available permutations of test cases.People don’t need to think about each available combination it should be done automatically once the requirements are specified.
1
Soltuion#1
52CONFIDENTIAL
Collaborate with developers
53CONFIDENTIAL
Take the functional created by developers don’t reinvent the wheel. Use development code data base as well.
2
Soltuion#2
54CONFIDENTIAL
Use the cases as the documentation
55CONFIDENTIAL
Use TAF to generate the most actual and comprehensive documentation from your test cases.It will contain working request and real responses and be always up to date and be complied with the acceptance criteria for the task.
3
Soltuion#2
56CONFIDENTIAL
Make it easy for other people to use
Once the framework is created anyone without automation knowledge should be able to add test cases to it!
57CONFIDENTIAL
Create TAF to allow adding cases even to business people if they want it.Adding test cases should not require knowledge beyond the average computer user.
4
Soltuion#3
58CONFIDENTIAL
Create easy readable results of testingCreate easy readable results of testing
59CONFIDENTIAL
Write in the main log only really useful information on the state of affairs.Logs and results should not contain all the information, but the necessary one which allows to quickly identify the root cause of the problem.
5
Soltuion#4
60CONFIDENTIAL
THANK YOU FOR ATTENTION!
??? QUESTIONS ??
?
61CONFIDENTIAL
Create easy readable results of testingTHE END
top related