the case for consumer-driven contracts
TRANSCRIPT
The case for consumer-driven contracts@matthewfellows
Picture the scene...
Monday morning
9am
Phone rings
It’s our VC
We just got the news....
We’re going to build a startup!
Soundify®�
Everyone is doing microservices!
Metric Value
No. teams 1
No. components 1
Test environments 1
Time in pipeline (commit to prod) 5 minutes
Risk 2.5%
Deployments per day 10
CD health check
Metric Value
No. teams 2
No. components 2
Test environments 4
Time in pipeline (commit to prod) 10 minutes
Risk 5%
Deployments per day 10
CD health check
Metric Value
No. teams 3
No. components 6
Test environments 18
Time in pipeline (commit to prod) 30 minutes
Risk 10%
Deployments per day 10
CD health check
ACQUIRED
Metric Value
No. teams 10
No. components 20
Test environments 20 200
Time in pipeline (commit to prod) 120+ minutes
Risk 20%
Deployments per day 1
CD health check
So what did we learn?
Where did we go wrong?
We pretended microservices were colocated libraries
Cohesive services, loosely coupled
vs
Cohesive teams, tightly coupled by services
Current tooling and strategies are not good enough
“Integration tests are a scam” - JB Rainsberger
Scam, you say? Justify!Integrated tests are:
● Slow● Fragile ● Hard to manage
When they fail, you can’t point to the problem!
Scam, you say? Justify!
“But my integration tests run in Docker, why can’t I use them?”
- People
Scam, you say? Justify!
“Because Maths” - Me
Branches per box vs test cases required
2 code branches = 128 tests5 code branches = 78,125 tests10 code branches = 10M tests
Good tests have the exact opposite properties
Dictator Driven Contracts
Dictator Driven Contracts
1. Sit in ivory tower and postulate2. Document perfect API (Swagger, API blueprint etc.)3. Create said API4. Publish said document to consumers5. Request dictate consumers update6. Repeat steps 1-5
How to: Dictator Driven Contracts
Crap, this didn’t work either!
Dictator Consumer Driven Contracts
Benefits?
You’ll know when you break a consumer
You have a form of documentation
You can test things independently
Pactwww.pact.io
Evolved from combining these two principles
Step 1: Define Consumer expectations
Step 1: Define Consumer expectations
Step 1: Define Consumer expectations
Step 1: Define Consumer expectations
Step 2: Verify expectations on Provider
Start with a consumer test
Given “User A exists”When I Receive “a GET request for user A”
With “these headers and query”Respond with “200 OK”
And “User A’s details in the body”
Given “User A does not exist”When I Receive “a GET request for user A”
Respond with “404 Not Found”
Example
// Setup our expected interactions on the Mock Service.
pact.
AddInteraction().
Given("User billy exists").
UponReceiving("A request to login with user 'billy'").
WithRequest(dsl.Request{
Method: "POST",
Path: "/users/login",
Body: loginRequest,
}).
WillRespondWith(dsl.Response{
Status: 200,
Headers: map[string]string{
"Content-Type": "application/json",
},
Body: `
{
"user": {
"name": "billy"
}
}
`,
})
// Run the test and verify the interactions.
err := pact.Verify(func() error {
client := Client{
Host: fmt.Sprintf("http://localhost:%d", pact.Server.Port),
}
client.loginHandler(rr, req)
// Expect User to be set on the Client
if client.user == nil {
return errors.New("Expected user not to be nil")
}
return nil
})
if err != nil {
t.Fatalf("Error on Verify: %v", err)
}
// Write pact to file `<pwd>/pacts/my_consumer-my_provider.json`
// NOTE: This also is a good candidate for use in TestMain(m *testing.M)
pact.WritePact()
Specification by example
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name for the test",
"provider_state": "Some state",
"request": {
"method": "GET",
"path": "/foobar",
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "foo"
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "bar"
}
}
}
...
]
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name for the test",
"provider_state": "Some state",
"request": {
"method": "GET",
"path": "/foobar",
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "foo"
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "bar"
}
}
}
...
]
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name for the test",
"provider_state": "Some state",
"request": {
"method": "GET",
"path": "/foobar",
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "foo"
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "bar"
}
}
}
...
]
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name for the test",
"provider_state": "Some state",
"request": {
"method": "GET",
"path": "/foobar",
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "foo"
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "bar"
}
}
}
...
]
{
"consumer": {"name": "MyConsumer"},
"provider": {"name": "MyProvider"},
"interactions": [
{
"description": "Some name for the test",
"provider_state": "Some state",
"request": {
"method": "GET",
"path": "/foobar",
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "foo"
}
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"s": "bar"
}
}
}
...
]
Next publish your pacts
// Publish the Pacts...
p := dsl.Publisher{}
err := p.Publish(types.PublishRequest{
PactURLs: []string{"../pacts/myconsumer-myprovider.json"},
PactBroker: os.Getenv("PACT_BROKER_HOST"),
ConsumerVersion: "1.0.0",
Tags: []string{"latest", "production"},
BrokerUsername: os.Getenv("PACT_BROKER_USERNAME"),
BrokerPassword: os.Getenv("PACT_BROKER_PASSWORD"),
})
Then verify your provider
// Verify the Provider from tagged Pact files stored in a Pact Broker
response = pact.VerifyProvider(types.VerifyRequest{
ProviderBaseURL: fmt.Sprintf("http://localhost:%d", providerPort),
BrokerURL: brokerHost,
Tags: []string{"latest", "prod"},
ProviderStatesURL: fmt.Sprintf("http://localhost:%d/states", providerPort),
ProviderStatesSetupURL: fmt.Sprintf("http://localhost:%d/setup", providerPort),
BrokerUsername: os.Getenv("PACT_BROKER_USERNAME"),
BrokerPassword: os.Getenv("PACT_BROKER_PASSWORD"),
})
if response.ExitCode != 0 {
t.Fatalf("Got %d, Want exit code 0", response.ExitCode)
}
Verifying a pact between billy and bobby
Given User billy exists
A request to login with user 'billy'
with POST /users/login
returns a response which
has status code 200
has a matching body
includes headers
"Content-Type" with value "application/json"
Given User billy does not exist
A request to login with user 'billy'
with POST /users/login
returns a response which
has status code 404
includes headers
"Content-Type" with value "application/json"
...
Finished in 0.03042 seconds
7 examples, 0 failures
Verifying a pact between billy and bobby
Given User billy exists
A request to login with user 'billy'
with POST /users/login
returns a response which
has status code 200
has a matching body
includes headers
"Content-Type" with value "application/json"
Given User billy does not exist
A request to login with user 'billy'
with POST /users/login
returns a response which
has status code 404
includes headers
"Content-Type" with value "application/json"
...
Finished in 0.03042 seconds
7 examples, 0 failures
Verifying a pact between billy and bobby
Given User billy exists
A request to login with user 'billy'
with POST /users/login
returns a response which
has status code 200
has a matching body
includes headers
"Content-Type" with value "application/json"
Given User billy does not exist
A request to login with user 'billy'
with POST /users/login
returns a response which
has status code 404
includes headers
"Content-Type" with value "application/json"
...
Finished in 0.03042 seconds
7 examples, 0 failures
Verifying a pact between billy and bobby
Given User billy exists
A request to login with user 'billy'
with POST /users/login
returns a response which
has status code 200
has a matching body (FAILED - 1)
includes headers
"Content-Type" with value "application/json"
Failures:
1) Verifying a pact between billy and bobby Given User billy exists A request to login with user 'billy' with POST
/users/login returns a response which has a matching body
Failure/Error: expect(response_body).to match_term expected_response_body, diff_options
Actual: {"user":{"user":"billy"}}
@@ -1,6 +1,5 @@
{
"user": {
- "name": "billy"
}
}
No Silver Bullet
Does not replace communication
What about systems maintained by other teams?
What about systems built in outdated technologies?
Scary outsideworld!
3rd Party
Mainframe
Recapping...
● Business impact of integrated tests● Cause and explanation of those effects● Alternative approach - isolation + contracts● Contract-testing as an approach, Pact as a tool (in this order!)
Thank you
- @matthewfellows
Given “The presentation is over”Upon Receiving “A request for an answer”With “A valid question”Respond With “A valid answer”