![Page 1: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/1.jpg)
QCon London 2016An Introduction to Property Based Testing
Aaron Bedra Chief Security Officer, Eligible @abedra
![Page 2: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/2.jpg)
Why do we test?
• To better understand what we are building
• To help us think deeper about what we are building
• To ensure the correctness of what we are building
• To help us explore our design*
• To explain to others how our code should work
![Page 3: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/3.jpg)
How do we test?
• With compilers (type systems, static analysis, etc)
• Manual testing
• X-Unit style tests
• Property/generative based tests
• Formal modeling
![Page 4: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/4.jpg)
How do we test?
• With compilers (type systems, static analysis, etc)
• Manual testing
• X-Unit style tests
• Property/generative based tests
• Formal modeling
![Page 5: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/5.jpg)
What is it?
![Page 6: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/6.jpg)
An abstraction
![Page 7: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/7.jpg)
Property based testing eliminates the guess work on value and order of operations testing
![Page 8: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/8.jpg)
Magic numbers
![Page 9: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/9.jpg)
Instead of specifying how you specify what
![Page 10: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/10.jpg)
Testing over time
![Page 11: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/11.jpg)
When we start our test suite, things are usually
easy to understand
![Page 12: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/12.jpg)
public class Basic { public static Integer calculate(Integer x, Integer y) { return x + y; } }
![Page 13: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/13.jpg)
public class BasicTest { @Test public void TestCalculate() { assertEquals(Integer.valueOf(5), Basic.calculate(3, 2)); } }
![Page 14: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/14.jpg)
What other tests might we write for this code?
![Page 15: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/15.jpg)
Like all programs we start simple
![Page 16: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/16.jpg)
But over time things get more complicated
![Page 17: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/17.jpg)
What happens when our simple calculate function grows to include an entire domain?
![Page 18: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/18.jpg)
Our test suite will undoubtedly grow, but we have options to
control the growth
![Page 19: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/19.jpg)
And also maintain confidence in our tests
![Page 20: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/20.jpg)
![Page 21: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/21.jpg)
By changing our mental model just a bit we can
cover much more ground
![Page 22: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/22.jpg)
Let’s revisit our basic example
![Page 23: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/23.jpg)
public class Basic { public static Integer calculate(Integer x, Integer y) { return x + y; } }
![Page 24: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/24.jpg)
But instead of a unit test, let’s write a property
![Page 25: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/25.jpg)
@RunWith(JUnitQuickcheck.class) public class BasicProperties { @Property public void calculateBaseAssumption(Integer x, Integer y) { Integer expected = x + y; assertEquals(expected, Basic.calculate(x, y)); } }
public class BasicTest { @Test public void TestCalculate() { assertEquals(Integer.valueOf(5), Basic.calculate(3, 2)); } }
![Page 26: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/26.jpg)
@RunWith(JUnitQuickcheck.class) public class BasicProperties { @Property(trials = 1000000) public void calculateBaseAssumption(Integer x, Integer y) { Integer expected = x + y; assertEquals(expected, Basic.calculate(x, y)); } }
![Page 27: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/27.jpg)
This property isn’t much different than the unit test
we had before it
![Page 28: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/28.jpg)
It’s just one level of abstraction higher
![Page 29: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/29.jpg)
Let’s add a constraint to our calculator
![Page 30: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/30.jpg)
Let’s say that the output cannot be negative
![Page 31: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/31.jpg)
public class Basic { public static Integer calculate(Integer x, Integer y) { Integer total = x + y; if (total < 0) { return 0; } else { return total; } } }
java.lang.AssertionError: Property calculateBaseAssumption falsified for args shrunken to [0, -679447654]
![Page 32: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/32.jpg)
Shrinking
![Page 33: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/33.jpg)
@RunWith(JUnitQuickcheck.class) public class BasicProperties { @Property public void calculateBaseAssumption(Integer x, Integer y) { Integer expected = x + y; assertEquals(expected, Basic.calculate(x, y)); } }
public class Basic { public static Integer calculate(Integer x, Integer y) { Integer total = x + y; if (total < 0) { return 0; } else { return total; } } }
![Page 34: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/34.jpg)
Now we can be more specific with our property
![Page 35: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/35.jpg)
@RunWith(JUnitQuickcheck.class) public class BasicProperties { @Property public void calculateBaseAssumption(Integer x, Integer y) { assumeThat(x, greaterThan(0)); assumeThat(y, greaterThan(0)); assertThat(Basic.calculate(x, y), is(greaterThan(0))); } }
java.lang.AssertionError: Property calculateBaseAssumption falsified for args shrunken to [647853159, 1499681379]
![Page 36: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/36.jpg)
We could keep going from here but let’s dive into some of the concepts
![Page 37: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/37.jpg)
Refactoring
![Page 38: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/38.jpg)
This is one of my favorite use cases for invoking property based testing
![Page 39: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/39.jpg)
Legacy code becomes the model
![Page 40: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/40.jpg)
It’s incredibly powerful
![Page 41: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/41.jpg)
It ensures you have exact feature parity
![Page 42: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/42.jpg)
Even for unintended features!
![Page 43: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/43.jpg)
Generators
![Page 44: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/44.jpg)
You can use them for all kinds of things
![Page 45: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/45.jpg)
Scenario
![Page 46: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/46.jpg)
Every route in your web application
![Page 47: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/47.jpg)
You could define generators based on your
routes
![Page 48: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/48.jpg)
And create valid and invalid inputs for every
endpoint
![Page 49: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/49.jpg)
You could run the generators on every test
![Page 50: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/50.jpg)
Or save the output of the generation for faster
execution
![Page 51: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/51.jpg)
Saved execution of generators can even bring you to simulation testing
![Page 52: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/52.jpg)
There are tons of property based testing libraries
available
![Page 53: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/53.jpg)
But this is a talk in a functional language track
![Page 54: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/54.jpg)
So let’s have some fun
![Page 55: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/55.jpg)
Let’s pretend we have some legacy code
![Page 56: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/56.jpg)
Written in C
![Page 57: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/57.jpg)
And we want to test it to make sure it actually
works
![Page 58: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/58.jpg)
But there are no quickcheck libraries
available*
![Page 59: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/59.jpg)
Warning! The crypto you are about to see should not be
attempted at work
![Page 60: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/60.jpg)
Caesar’s Cipher
![Page 61: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/61.jpg)
Let’s start with our implementation
![Page 62: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/62.jpg)
#include <stdlib.h> #include <string.h> #include <ctype.h>
char *caesar(int shift, char *input) { char *output = malloc(strlen(input)); memset(output, '\0', strlen(input));
for (int x = 0; x < strlen(input); x++) { if (isalpha(input[x])) { int c = toupper(input[x]); c = (((c - 65) + shift) % 26) + 65; output[x] = c; } else { output[x] = input[x]; } }
return output; }
![Page 63: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/63.jpg)
Next we create a new implementation to test
against
![Page 64: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/64.jpg)
caesar :: Int -> String -> String caesar k = map f where f c | inRange ('A', 'Z') c = chr $ ord 'A' + (ord c - ord 'A' + k) `mod` 26 | otherwise = c
![Page 65: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/65.jpg)
We now have two functions that “should” do
the same thing
![Page 66: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/66.jpg)
But they aren’t in the same language
![Page 67: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/67.jpg)
Thankfully Haskell has good FFI support
![Page 68: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/68.jpg)
foreign import ccall "caesar.h caesar" c_caesar :: CInt -> CString -> CString
native_caesar :: Int -> String -> IO String native_caesar shift input = withCString input $ \c_str -> peekCString(c_caesar (fromIntegral shift) c_str)
![Page 69: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/69.jpg)
$ stack exec ghci caesar.hs caesar.so GHCi, version 7.10.3: http://www.haskell.org/ghc/ :? for help [1 of 1] Compiling Main ( caesar.hs, interpreted ) Ok, modules loaded: Main. *Main> caesar 2 "ATTACKATDAWN" "CVVCEMCVFCYP" *Main> native_caesar 2 "ATTACKATDAWN" "CVVCEMCVFCYP"
![Page 70: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/70.jpg)
We can now execute our C code from inside of
Haskell
![Page 71: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/71.jpg)
We can use Haskell’s quickcheck library to
verify our C code
![Page 72: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/72.jpg)
First we need to write a property
![Page 73: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/73.jpg)
unsafeEq :: IO String -> String -> Bool unsafeEq x y = unsafePerformIO(x) == y
genSafeChar :: Gen Char genSafeChar = elements ['A' .. 'Z']
genSafeString :: Gen String genSafeString = listOf genSafeChar
newtype SafeString = SafeString { unwrapSafeString :: String } deriving Show instance Arbitrary SafeString where arbitrary = SafeString <$> genSafeString
equivalenceProperty = forAll genSafeString $ \str -> unsafeEq (native_caesar 2 str) (caesar 2 str)
![Page 74: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/74.jpg)
unsafeEq :: IO String -> String -> Bool unsafeEq x y = unsafePerformIO(x) == y
genSafeChar :: Gen Char genSafeChar = elements ['A' .. 'Z']
genSafeString :: Gen String genSafeString = listOf genSafeChar
newtype SafeString = SafeString { unwrapSafeString :: String } deriving Show instance Arbitrary SafeString where arbitrary = SafeString <$> genSafeString
equivalenceProperty = forAll genSafeString $ \str -> unsafeEq (native_caesar 2 str) (caesar 2 str)
![Page 75: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/75.jpg)
unsafeEq :: IO String -> String -> Bool unsafeEq x y = unsafePerformIO(x) == y
genSafeChar :: Gen Char genSafeChar = elements ['A' .. 'Z']
genSafeString :: Gen String genSafeString = listOf genSafeChar
newtype SafeString = SafeString { unwrapSafeString :: String } deriving Show instance Arbitrary SafeString where arbitrary = SafeString <$> genSafeString
equivalenceProperty = forAll genSafeString $ \str -> unsafeEq (native_caesar 2 str) (caesar 2 str)
![Page 76: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/76.jpg)
unsafeEq :: IO String -> String -> Bool unsafeEq x y = unsafePerformIO(x) == y
genSafeChar :: Gen Char genSafeChar = elements ['A' .. 'Z']
genSafeString :: Gen String genSafeString = listOf genSafeChar
newtype SafeString = SafeString { unwrapSafeString :: String } deriving Show instance Arbitrary SafeString where arbitrary = SafeString <$> genSafeString
equivalenceProperty = forAll genSafeString $ \str -> unsafeEq (native_caesar 2 str) (caesar 2 str)
![Page 77: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/77.jpg)
unsafeEq :: IO String -> String -> Bool unsafeEq x y = unsafePerformIO(x) == y
genSafeChar :: Gen Char genSafeChar = elements ['A' .. 'Z']
genSafeString :: Gen String genSafeString = listOf genSafeChar
newtype SafeString = SafeString { unwrapSafeString :: String } deriving Show instance Arbitrary SafeString where arbitrary = SafeString <$> genSafeString
equivalenceProperty = forAll genSafeString $ \str -> unsafeEq (native_caesar 2 str) (caesar 2 str)
![Page 78: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/78.jpg)
unsafeEq :: IO String -> String -> Bool unsafeEq x y = unsafePerformIO(x) == y
genSafeChar :: Gen Char genSafeChar = elements ['A' .. 'Z']
genSafeString :: Gen String genSafeString = listOf genSafeChar
newtype SafeString = SafeString { unwrapSafeString :: String } deriving Show instance Arbitrary SafeString where arbitrary = SafeString <$> genSafeString
equivalenceProperty = forAll genSafeString $ \str -> unsafeEq (native_caesar 2 str) (caesar 2 str)
![Page 79: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/79.jpg)
*Main> quickCheck equivalenceProperty *** Failed! Falsifiable (after 20 tests): "QYMSMCWTIXNDFDMLSL" *Main> caesar 2 "QYMSMCWTIXNDFDMLSL" "SAOUOEYVKZPFHFONUN" *Main> native_caesar 2 "QYMSMCWTIXNDFDMLSL" “SAOUOEYVKZPFHFONUN/Users/abedra/x“
![Page 80: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/80.jpg)
#include <stdlib.h> #include <string.h> #include <ctype.h>
char *caesar(int shift, char *input) { char *output = malloc(strlen(input)); memset(output, '\0', strlen(input));
for (int x = 0; x < strlen(input); x++) { if (isalpha(input[x])) { int c = toupper(input[x]); c = (((c - 65) + shift) % 26) + 65; output[x] = c; } else { output[x] = input[x]; } }
return output; }
![Page 81: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/81.jpg)
We’ve found a memory handling issue in our C
code!
![Page 82: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/82.jpg)
In reality there are more issues with this code, but our issue was quickly exposed
![Page 83: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/83.jpg)
And easily reproduced
![Page 84: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/84.jpg)
Wrapping up
![Page 85: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/85.jpg)
Not all testing is created equal
![Page 86: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/86.jpg)
You should use as many different testing
techniques as you need
![Page 87: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/87.jpg)
Remember to think about the limits of your tools
![Page 88: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/88.jpg)
And use tools that help you achieve your results
more effectively
![Page 89: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/89.jpg)
And more efficiently
![Page 90: QCon London 2016QCon London 2016 An Introduction to Property Based Testing Aaron Bedra Chief Security Officer, Eligible ... There are tons of property based testing libraries available](https://reader036.vdocuments.site/reader036/viewer/2022081607/5ece2d4cee11c142a623dd11/html5/thumbnails/90.jpg)
Questions?