webrtc and its impact on testing
TRANSCRIPT
WebRTC and its Impact on Testing
It isn’t the Web and it isn’t VoIP
&
Who are we?
2
Nikhil Kaul
Product Marketing Manager
SmartBear
@kaulnikhil
Tsahi Levent-Levi
Co-founder, testRTC
Founder, BlogGeek.me
@tsahil
On our plate today
• What’s WebRTC exactly?
• Testing challenges with WebRTC
• How do vendors test today?
• WebRTC testing best practices
3
What is WebRTC Exactly?
4
5
WebRTC offers real time communication natively from a web browser
WebRTC is a media engine with JavaScript APIs
6
WebRTC is a Technology
not a SolutionIt is up to you to build the solution
TheBusyBrain
How calls are made in WebRTC?
10/7/2015
7
12
34
5
Why all the fuss?
8
Reduce barrier of entry for new vendors
Friction reduction for users
WebRTC’s Job to be Done
9
No Plugins No Downloads No Installations No App Store
No SignalingFree (as in BSD) No Patents No Media Eng.
Where is it available?
And we almost forgot mobile apps – it is there for quite some time now…
10
2015
Chrome Firefox Opera Internet Explorer Safari
Client-side APIs
getUserMedia Access to the camera and microphone of the device• Grab a user’s photo• Offer voice interface to a service• Locate user’s head/eyes/hands
peerConnection Creation and management of voice and video sessions• Voice calls• Video calls
dataChannel Send arbitrary data across browsers• Low latency information (multiplayer gaming)• P2P streaming• Privacy
11
Server-side Considerations
• STUN/TURN servers
• Signaling server
• Media servers
• Gateways, Multiparty bridges, etc.
12
Testing Challenges with WebRTC
13
At an intersection of worlds
14
OurInternet
WebTraditional
VoIP
The world of VoIP testing
• Focused on DUTs (Devices Under Test)
• Rely heavily on slow moving standards and implementations
• Vast majority of testing solutions are on premise
• Assume limited geographies
15
The world of Web testing
16
• Focused on different browsers
• Rely heavily on the GUI
• Responsive design causes GUI changes
• GUI and API testing treated as separate
• Geography doesn’t matter for functionality
The 7 challenges of WebRTC testing
Challenge VoIP Internet
1. WebRTC’s improvement correlates with Internet updates
2. WebRTC is a work in progress
3. Introducing WebRTC, cloud, and unmanaged networks
4. Scaling
5. Global reach
6. Peer-to-peer networking and Internet KPIs
7. Downgrading communications from a service to a feature
10/7/2015
17
The 7 challenges of WebRTC testing
Challenge VoIP Web
1. WebRTC’s improvement correlates with Internet updates
2. WebRTC is a work in progress
3. Introducing WebRTC, cloud, and unmanaged networks
4. Scaling
5. Global reach
6. Peer-to-peer networking and Internet KPIs
7. Downgrading communications from a service to a feature
10/7/2015
18
The 8 7 challenges of WebRTC testing
Challenge VoIP Web
1. WebRTC’s improvement correlates with Internet updates
2. WebRTC is a work in progress
3. Introducing WebRTC, cloud, and unmanaged networks
4. Scaling
5. Global reach
6. Peer-to-peer networking and Internet KPIs
7. Downgrading communications from a service to a feature
Bonus point: Web testing is far from static
10/7/2015
19
How do vendors test today?
20
1. They don’t
• Test other aspects of their service
• But ignore the WebRTC part of it
• Or test it superficially
I wouldn’t call this testing…
21
2. Manual testing
• Put the body-count to work
• Regression every once in awhile, or when something breaks
• No real scale testing
Hard to keep pace with browser updates and no automation…
22
3. DIY-framework
• Start with Selenium
• Test the basics
Considerable investment in building a rather generic testing infrastructure…
23
WebRTC Testing Best Practices
24
Test 3 browser versions
• Latest
• Previous
• Beta
25
Configure the network
• Different firewall and NAT configurations
• Different bitrates (static and dynamic ones)
• Different packet loss
26
Test end-to-end
• Don’t focus on your DUT
• Start from the page load itself and not only from the actual video call session
27
Automate your testing
• Put automation to work with you
• But make sure you collect and analyze the relevant information
• Automate the collection and analysis processes as well
28
Check out what we’re doing at testRTC
Test & Monitor WebRTC-based services @ scale
29
Check out what we’re doing at SmartBear
30
31
Thank You!
Tsahi Levent-Levi
http://testrtc.com
Nikhil Kaul
http://smartbear.com