automated testing in call of duty jan van valburg · metrics tracking/graphing using...
TRANSCRIPT
![Page 1: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/1.jpg)
Automated Testing inCall of Duty®
Jan van Valburg
Technical Director, Activision CTN
![Page 2: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/2.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 3: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/3.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 4: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/4.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 5: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/5.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 6: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/6.jpg)
![Page 7: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/7.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 8: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/8.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Wishlist
●
●
●
![Page 9: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/9.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 10: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/10.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Some Stats
●
●
●
●
![Page 11: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/11.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
What we run
●●
●
●
●
●●
●●
●
●
●●
●
![Page 12: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/12.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
What Compass Looks Like
![Page 13: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/13.jpg)
![Page 14: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/14.jpg)
![Page 15: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/15.jpg)
![Page 16: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/16.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 17: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/17.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 18: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/18.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 19: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/19.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 20: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/20.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 21: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/21.jpg)
![Page 22: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/22.jpg)
![Page 23: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/23.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 24: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/24.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 25: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/25.jpg)
![Page 26: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/26.jpg)
![Page 27: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/27.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Screenshot / Perf Capture Method
●
●
●
●
![Page 28: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/28.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
How Compass Tasks Are Run
![Page 29: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/29.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 30: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/30.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
sh('py.test –junitxml=result.xml')
![Page 31: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/31.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
build_solution('PhantomTube\PhantomTube.sln',configuration='Release')
![Page 32: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/32.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
P4Repository('perforce.smi.sendmail.com:1666', username='dfu', client='build-Jenkins')
![Page 33: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/33.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
@task('my_task’)def test():
sync('dev')
sh('echo hello world')
Measurement.new(name='Example', group='Data’,value=data['example_measurement'])
store(['output.json’])
![Page 34: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/34.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
@task('my_task’)def test():
sync(‘dev’)
sh('echo hello world')
Measurement.new(name='Example', group='Data’,value=data['example_measurement'])
store(['output.json’])
![Page 35: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/35.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
@task(resources=[PC(), PS4()])def run_game():
sync(['game.elf'])with resource_manager.acquire_resource('PS4') as target:
target.Reset()
target.LaunchGame('game.elf')
sleep(20)
Screenshot.capture(target, 'Test', ‘Test screenshot')
![Page 36: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/36.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Benefits: Configuration as Code
- Debug scripts locally or on the server
- Easy to write. Easy to understand.
- All the power of the Python ecosystem
![Page 37: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/37.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Benefits: Configuration in SCM
- History
- Ability to re-run old checkins
- Test configuration using proof build
- Branching configurations
- Empower users to write and maintain their own tests
![Page 38: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/38.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Benefits: Configuration in SCM
- History
- Ability to re-run old checkins
- Test configuration using proof build
- Branching configurations
- Empower users to write and maintain their own tests
![Page 39: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/39.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Drawbacks
- Learning curve
- More difficult to get going initially
![Page 40: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/40.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Provisioning
- All software on our workers is installed using Puppet
- Puppet service on worker talks to the Puppet Master
- Will install/uninstall software as needed
- Looking at switching to Windows Containers (Docker)
![Page 41: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/41.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Provisioning
- All software on our workers is installed using Puppet
- Puppet service on worker talks to the Puppet Master
- Will install/uninstall software as needed
- Looking at switching to Windows Containers (Docker)
![Page 42: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/42.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Error Bucketing / Auto Retry
![Page 43: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/43.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Error Bucketing
- Need a way to track incidence of errors
- When did they first start happening
- How often do they happen
![Page 44: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/44.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Assert: (cent->prevState.apos.trType == TR_INTERPOLATE)
![Page 45: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/45.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Error Key Examples
- Assert: packedCheckSums[checkSumCount] != 0
- Connect failed: The console is not yet ready to process requests.
- gfx_d3d\r_scene_add.cpp(1228,24): error : use of undeclared
identifier 'R_WARM_WORLD_VOLUME_DECAL_EXCEEDED_LIMIT’
- Failed to upload file: Out of disk space
![Page 46: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/46.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 47: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/47.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 48: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/48.jpg)
![Page 49: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/49.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 50: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/50.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Auto Retry
![Page 51: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/51.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Auto Retry
![Page 52: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/52.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Pain Points
• Stability
• Scalability
• Maintenance
![Page 53: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/53.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Pain Points
• Stability
• Scalability
• Maintenance
• Stability is crucial
• Debugging live servers: stressful
![Page 54: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/54.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Pain Points
• Stability
• Scalability
• Maintenance
• Writing scalable code
• Dealing with bandwidth usage
![Page 55: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/55.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Pain Points
• Stability
• Scalability
• Maintenance
• Large scale automation ->
maintenance overhead
• Use self-healing systems
![Page 56: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/56.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Applying this to existing buildservers(What if I don’t want to write a Buildserver from scratch?)
![Page 57: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/57.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Applying this to existing buildservers
Compass Webapp UI: Python + Flask
![Page 58: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/58.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Applying this to existing buildservers
Compass Webapp UI: Python + Flask
![Page 59: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/59.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Applying this to existing buildservers
Metrics tracking/graphing
![Page 60: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/60.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Applying this to existing buildservers
Metrics tracking/graphing using InfluxDB/Grafana
![Page 61: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/61.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Applying this to existing buildservers
@task('my_task’)def test():
sync('dev')
sh('echo hello world')
Measurement.new(name='Example', group='Data’,value=data['example_measurement'])
store(['output.json’])
Configuration as code
![Page 62: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/62.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
![Page 63: Automated Testing in Call of Duty Jan van Valburg · Metrics tracking/graphing using InfluxDB/Grafana Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing,](https://reader030.vdocuments.site/reader030/viewer/2022041102/5ee0f06cad6a402d666bffc0/html5/thumbnails/63.jpg)
Programming and Technology Track, Digital Dragons 2018 ©2018 Activision Publishing, Inc.
Thank you!
Jan van Valburg
@janvanvalburg