Testing &Verification

Alexander Menk HAW, GSM, WS 2018

[by Stephen Confer]

Motivation

●

●

●

●

●○○○

ÜberblickMaster

Forschung

●●●

●○○

●

Überblick: Testing

Definition Testing (Verification)

Software testing consists of the dynamic verification that a program provides

expected behaviors on a finite set of test cases, suitably selected from the usually

infinite execution domain.

- [1] Guide to the Software Engineering Body of Knowledge

Grundlagen: Testing ist ...

●●

●●

●●

●●●

●

Psychologie in Software Engineering?

●●

○○

●○○

●○○○

“I like offending people, because I think people who get offended should be offended.” -- Linus Torvalds

Das Orakel-Problem

●○○○

●●

“foo bar” <= substr “foo bar”,0,7“bar” <= substr “foo bar”,4,3“r” <= substr “foo bar”,6,1 nil? <= substr nil,4,2 ? <= substr Date.new,0,0

Unit Testing

●○

●●

Testabdeckung

●

●○

●

●

[A]

Statische Analyse

Source CodeModel

ExtractionIntermediate

Representations Analysis Result

[H, codesonar]

Statische Analyse

[I]

[I]

[G]

[G]

Testing und Nebenläufigkeit

●○

●○○○

●○○

⇒

●○○

Semaphoren und Mutual Exclusions

●

●

●○○

●○

●

[15]

Problem des kritischen Abschnitts

Thread 1

Variable

Thread 2

Lock Set (1997 anhand von “Eraser”[15])

●

● v v○ l v l

○ v l○ v○ v

●

[M]

Forschung

GONZALEZ, D.; SANTOS, J. C. S.; POPOVICH, A.; MIRAKHORLI, M. & NAGAPPAN, M.:

A Large-scale Study on the Usage of Testing Patterns That Address Maintainability Attributes: … [6]

< 1k LOC< 10k LOC< 00k LOC>= 100k LOC

[B]

Testing in Open Source

● Metriken○

● Entwurfsmuster○○○○

● Qualitätsattribute○○○

[F]

[B]

[B]

● 23,75% Pattern Nutzung● Assertion Message 41%● Test Case Per Class 33,3%

Zusammenfassung, Testing in Open Source

●

●

●

●○

●○

■■

○■

○■

●

Harald Altinger, Franz Wotawa, and Markus Schurius. 2014.

Testing methods used in the automotive industry: results from a survey. [8]

[C]

Testing in der Automobil-Industrie?

●○

○

●

●○

○

○

●○

○

[J]

Research (Re) Pre Development (PD) Series Development (SD)

[C]

Jahic, Jasmin & Kuhn, Thomas & Jung, Matthias & Wehn, Norbert. 2017.

Supervised Testing of Concurrent Software in Embedded Systems. [12]

[L]

Lock Set und Scheduling

●

●

●[L]

[L]

Konferenzen

● ISSTA 2019, Beijing, China, July 15-19, 2019conf.researchr.org/home/issta-2019

● ASE 2019, Urbana-Champaign, Illinois, USA, 11.10 – 3.112019.ase-conferences.org/

● CAST 2019?associationforsoftwaretesting.org/conference/

● European Testing Conference 2019, February 14-15th 2019, Valencia, Spain

europeantestingconference.eu/2019/

Master●

○

○

○

●

○

[K]

Abbildungen

A. https://www.eclemma.org/images/screen.png, eclEmma Code Coverage Tool, Aufruf 27.10.18 um 17:50B. GONZALEZ, D.; SANTOS, J. C. S.; POPOVICH, A.; MIRAKHORLI, M. & NAGAPPAN, M.: A Large-scale Study on the Usage of Testing Patterns That Address

Maintainability Attributes: Patterns for Ease of Modification, Diagnoses, and Comprehension. In:: IEEE Press. : Proceedings of the 14th International Conference on Mining Software Repositories., 2017, S. 391--401

C. Harald Altinger, Franz Wotawa, and Markus Schurius. 2014. Testing methods used in the automotive industry: results from a survey. In Proceedings of the 2014 Workshop on Joining AcadeMiA and Industry Contributions to Test Automation and Model-Based Testing (JAMAICA 2014). ACM, New York, NY, USA, 1-6. DOI=http://dx.doi.org/10.1145/2631890.2631891

D. http://www.b737.org.uk/maxflttestprog.htm, abgerufen 28.10.18 22:31E. https://apod.nasa.gov/apod/ap120926.html, abgerufen 28.10.18. 22:36, Space Shuttle Endeavor by Stephen ConferF. https://bitbakery.com/using/UsingMaven.html, abgerufen 29.10.18 13:52G. https://upload.wikimedia.org/wikipedia/commons/5/5c/Sonarqube-nemo-dashboard.png, abgerufen 29.10.18 14:45H. https://www.grammatech.com/products/source-code-analysis, abgerufen 29.10.18 um 14:37I. https://www.sonarlint.org/, abgerufen 29.10.18 um 14:55

J. https://www.legalexaminer.com/transportation/a-brief-history-of-crash-dummies/, abgerufen 29.10.18 17:29K. https://cdn0.iconfinder.com/data/icons/education-science-1-1/100/Education__seience_1_9-512.png, abgerufen 29.10.18 17:34L. Jahic, Jasmin & Kuhn, Thomas & Jung, Matthias & Wehn, Norbert. (2017). Supervised Testing of Concurrent Software in Embedded

Systems. 10.1109/SAMOS.2017.8344633.M. Stefan Savage, Michael Burrows, Greg Nelson, Patrick Sobalvarro, and Thomas Anderson. 1997. Eraser: a dynamic data race detector for

multithreaded programs. ACM Trans. Comput. Syst. 15, 4 (November 1997), 391-411.

Quellen

1. SOCIETY, I. E. E. E. C.; BOURQUE, P. & FAIRLEY, R. E.: Guide to the Software Engineering Body of Knowledge (SWEBOK(R)): Version 3.0, Hoboken, New Jersey, USA: IEEE Computer Society Press., 2014, pp 4 – 1 Introduction

2. BERTOLINO, Antonia. Software Testing Research and Practice. Berlin, Heidelberg: Springer-Verlag. 2003. .P. 9.3. WEINBERG, Gerald M.. The Psychology of Computer Programming. New York, NY, USA: John Wiley & Sons, Inc., 19854. International Software Testing Qualification Board Glossary.

https://www.astqb.org/certified-tester-resources/glossary-software-testing-terms/. Abgerufen 13.06.175. "Systems and software engineering – Vocabulary". ISO/IEC/IEEE 24765:2010(E). 2010, p. 1-418.6. GONZALEZ, D.; SANTOS, J. C. S.; POPOVICH, A.; MIRAKHORLI, M. & NAGAPPAN, M.: A Large-scale Study on the Usage of Testing Patterns That Address

Maintainability Attributes: Patterns for Ease of Modification, Diagnoses, and Comprehension. In:: IEEE Press. : Proceedings of the 14th International Conference on Mining Software Repositories., 2017, S. 391--401

7. TRAUTSCH, F. & GRABOWSKI, J.: Are There Any Unit Tests? An Empirical Study on Unit Testing in Open Source Python Projects. In:: . : 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)., 2017, S. 207-218

8. Harald Altinger, Franz Wotawa, and Markus Schurius. 2014. Testing methods used in the automotive industry: results from a survey. In Proceedings of the 2014 Workshop on Joining AcadeMiA and Industry Contributions to Test Automation and Model-Based Testing (JAMAICA 2014). ACM, New York, NY, USA, 1-6. DOI=http://dx.doi.org/10.1145/2631890.2631891

9. https://randoop.github.io/randoop/, aufgerufen 28.10.18 um 16:0410. https://www.sonarsource.com/, aufgerufen 28.10.18 um 16:0411. https://www.absint.com/astree/index_de.htm aufgerufen 28.10.18 um 16:0712. Jahic, Jasmin & Kuhn, Thomas & Jung, Matthias & Wehn, Norbert. (2017). Supervised Testing of Concurrent Software in Embedded

Systems. 10.1109/SAMOS.2017.8344633.13. https://www.grammatech.com/products/source-code-analysis abgerufen 29.10.18 um 14:3714. P. S. Kochhar, T. F. Bissyandé, D. Lo and L. Jiang, "An Empirical Study of Adoption of Software Testing in Open Source Projects," 2013 13th

International Conference on Quality Software, Najing, 2013, pp. 103-112.15. Stefan Savage, Michael Burrows, Greg Nelson, Patrick Sobalvarro, and Thomas Anderson. 1997. Eraser: a dynamic data race detector for

multithreaded programs. ACM Trans. Comput. Syst. 15, 4 (November 1997), 391-411.