erik p. debenedictis sandia is a multiprogram laboratory operated by sandia corporation, a lockheed...
TRANSCRIPT
![Page 1: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/1.jpg)
Erik P. DeBenedictis
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy under contract DE-AC04-94AL85000.
Testing Integration and Validation Group
![Page 2: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/2.jpg)
Agenda
• Hardware Inventory– Oak Ridge– Sandia– Berkeley– Argonne
• Intellectual Property– Process
• Documentation Standards– TEX– HTML– Embedded
• Bug tracking– Bugzilla
• Test Framework– Deja Gnu– QMTest
![Page 3: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/3.jpg)
Hardware
• Berkeley– Hardware
• 32 nodes P II 400
– Software• Redhat 6.2
• Argonne– Hardware
• On demand 256 nodes• P III 500s myrinet
– Software• Redhat 7.1
• Oak Ridge (posted in notebook)– Hardware
• 4 dual 450 with 1 gig ram• X Torque 64 nodes 2 gig
– Fast ethernet + gig ethernet
– Software• Oscar install 1.2.1
• Sandia– Hardware
• 8 Alpha + admin node
– Software• Cplant
![Page 4: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/4.jpg)
Servers
• Ames– CVS Directory
• E-mail Brett
• Argonne– Bitkeeper
• E-mail Narayan
– Bugzilla• E-mail Narayan
• Sandia– CVS
• E-mail Erik
![Page 5: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/5.jpg)
Intellectual Property
• Process– Fred (DOE) requests Open Source
– Labs “Assert Copyright” on Open Source License• What license• Needs to be same
– Contribute Source
– Publish Source
![Page 6: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/6.jpg)
Documentation Standards
• Objectives– Set standards for English
Language contributions to group’s product
• HTML– (Doesn’t paginate well --
Hargrove)
• Rusty’s TeX Macros– (Not browsable? --
Hargrove)– (Paginates well
Jackson)
• Microsoft Word– (Violent objection to this
one)
• ASCII
• What types of documentation?– Design documents– User guide– Man pages– XML Schema specs– Installation guide– Administration guide– Reference guide (optional)
• Format requirements– Source format
• Html, tex
– Distribution format• Pdf, html
![Page 7: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/7.jpg)
Bugtracking
• Bugzilla– Argonne has bugzilla installed and can create a
branch for SciDAC
![Page 8: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/8.jpg)
Test Harness
• The T-I-V-X group investigated test harness products– Selected Deja GNU (but did not review QMTest)
• Sandia investigated test harness products for Cplant– Selected QMTest
• The SciDAC SSS group finds QMTest interesting and support for Deja GNU has withered
![Page 9: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/9.jpg)
What is QMTest?
• Scriptable (Python) test driver• Test metadata (results) defined with XML• Web-based interface• Extensible webserver built in (Zope)• HTML contents customizable• New test types through inheritance• More: http://www.codesourcery.com/qm/qmtest
![Page 10: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/10.jpg)
Why QMTest?
• Opensource• Scriptable (you can extend it)• Object-oriented• Declarative results in XML• Web-based UI supports decentralized dev.• Active and growing development commun.• Good “separation of concerns” design
![Page 11: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/11.jpg)
How Does Cplant Use It?
• Mostly as intended • Subclass ExecTest• Declare test results using UI right now• Drive cases sequentially. PBS integration?• We’re coming up the learning curve…
![Page 12: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/12.jpg)
How QMTest Looks
![Page 13: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/13.jpg)
Example Test Report
![Page 14: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/14.jpg)
Step 1: Define a New Test
![Page 15: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/15.jpg)
Step 2: Describe It
![Page 16: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/16.jpg)
Who’s Doing It
• Lead: Sue Goudy [email protected]• Grunt: Mike Carifio [email protected]• Carifio is ½ time…
![Page 17: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/17.jpg)
When Did We Do This?
• 12/01 Carifio invest. test frameworks• 1/02 Goudy, Carifio select QMTest• Q1/02 Goudy,Carifio learn Python, QMT.• Q2/02 Carifio gathers test “packages”,• implements “build framework”• Q3/02 Drive packages with QMTest• Q4/02 UI mods? PBS integration?
![Page 18: Erik P. DeBenedictis Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of](https://reader036.vdocuments.site/reader036/viewer/2022081603/56649f305503460f94c4aca3/html5/thumbnails/18.jpg)
Conclusions
• QMTest is scriptable driver• Tests are defined in XML• UI is web-based, content generated• New test kinds derived from base classes• Best price• Active development• Seems to be “best in class” today