testing after you’ve finished testing
Post on 13-Jul-2015
49 Views
Preview:
TRANSCRIPT
Testing… (After You’ve Finished Testing)
Jon BachQE Director, Live Site Qualityjobach@ebay.com
STAR Canada 2013
2
These are slides for the Conference CD
This deck is written to meet a deadline for the conference CD on January 25th.
If you’re reading this slide, please know that the content is likely out-of-date because I change content before, after (and even DURING) my presentations.
If you want the latest-and-greatest slides, go to my public Dropbox share:
https://www.dropbox.com/sh/bg1r6qz3wqhcil3/-R_hgcinfR
4
10 Testing-related activities after release
• Experimentation• Monitoring• Investigative• Surveys• Scenario• Reputation• Follow-up• Regression• Events• Competitive
8
This is a dip in bids we saw in Production.
What questions do you have to troubleshoot this?
Investigative Testing
9
Topics for investigation (FORCOSTTTIP)
• Feature – Do we know what happens when people bid?• Origin -- When did this start happening?• Recent – Did we just push code to Production?• Consistency / Correlation -- What happened yesterday,
tomorrow? Are there other graphs and metrics?• Overlap – Is there any other process happening?• Security – Is it malicious? Intentional?• Terminology -- What kinds of “bids”? US, UK, International?• Third-Party / Silent Partner – What’s not in the room? What sort
of information have we NOT heard from?• Truth is out there – does someone already know about this?• Instrumentation – is the graph wrong? Can I get more data?• Prediction – can we test a conjecture with a prediction
14
I go on eBay Radio (a weekly podcast for customers) as “The Bug Hunter” to elicit stories about technical problems in Production.
-------------------------------------------------------------------------------
“I have attached the screen shot for you. Try as I might, I can't get
the selling reminders to go to the top. The arrows don't work, but I
can move sold, unsold and active selling. I even set the default to summary, but no luck. FYI- I am using Safari on my trusty Mac.
Also, the default font size is too big. When I zoom it out, it's too small. It was a better size the way it was before. I sent a message
to the team through the 'what do you think' for that page.”
Follow-up
top related