patterns in go complex concurrency - evan huus · goroutine input output goroutine goroutine...
TRANSCRIPT
![Page 1: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/1.jpg)
Complex Concurrency Patterns in Go Evan Huus - Shopify, Inc.
@eapache
![Page 2: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/2.jpg)
Goroutine
Input Output
Goroutine
Goroutine
Goroutine
Goroutine
Goroutine
Goroutine
Goroutine
In Theory
![Page 3: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/3.jpg)
Input Output
In Practice
![Page 4: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/4.jpg)
Input Output
Do I need a mutex here?
In Practice
![Page 5: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/5.jpg)
Yes, Go makes concurrency easier.
![Page 6: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/6.jpg)
Yes, Go makes concurrency easier.
It’s still really hard.
![Page 7: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/7.jpg)
Overview● A little bit of context● A lot of case study
![Page 8: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/8.jpg)
Literary Giants
![Page 9: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/9.jpg)
Kafka (https://kafka.apache.org/)
● Java-based Apache project for distributed publish-subscribe messaging.
● Messages are grouped into topics, topics are subdivided into partitions, and partitions are led or replicated by brokers.
● Clients are thick.
![Page 10: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/10.jpg)
Sarama.go (https://github.com/Shopify/sarama)
● Native Golang client for producing and consuming messages via Kafka.
● Implements wire protocol, producer and consumer.
● First version was a proof-of-concept, kept it simple, but...
![Page 11: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/11.jpg)
Knuth
“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.”
![Page 12: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/12.jpg)
Second Draft Requirements● Fast● Configurable● Resilient● Correct
![Page 13: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/13.jpg)
Producerdispatcher
topic producer
partition producer
topic producer
partition producer
aggregator
broker producer
aggregator
broker producer
retrier
...
... ...
...
...
KafkaCluster
![Page 14: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/14.jpg)
dispatcher
topic producer
partition producer
topic producer
partition producer
...
... ...
Resiliency and Isolation
KafkaCluster
![Page 15: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/15.jpg)
Resiliency and Isolation- fan-out (dispatcher)
handlers := make(map[string]chan<- *Message)
for msg := range input {handler := handlers[msg.Topic]
if handler == nil {handler = p.newTopicProducer(msg.Topic)handlers[msg.Topic] = handler
}
handler <- msg}
![Page 16: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/16.jpg)
Resiliency and Isolation- circuit-breakers (https://github.com/eapache/go-resiliency)
partitions, err = client.Partitions(msg.Topic)
versus
breaker := breaker.New(3, 1, 10*time.Second)
var partitions []int32err := breaker.Run(func() (err error) {
partitions, err = client.Partitions(msg.Topic)return
})
![Page 17: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/17.jpg)
Dynamic Multiplexing
Partition->
Broker
*
1
![Page 18: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/18.jpg)
Dynamic Multiplexing- global, locked, reference-counted map
Partition->
Broker
*
1
![Page 19: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/19.jpg)
Dynamic Multiplexing- acquire-broker
p.brokerLock.Lock()defer p.brokerLock.Unlock()
bp := p.brokers[broker]if bp == nil {
bp = p.newBP(broker)p.brokers[broker] = bp
}p.brokers[broker].refs++
return bp
- release-broker
p.brokerLock.Lock()defer p.brokerLock.Unlock()
p.refs[bp]--if p.refs[bp] == 0 {
close(bp.input)delete(p.brokers, bp.broker)
}
![Page 20: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/20.jpg)
Batching and I/O
aggregator
broker producerKafkaCluster
![Page 21: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/21.jpg)
Batching and I/O- aggregator
for {select {case msg := <-input:
req.addMessage(msg)if req.full() { output = realOutput }
case <-timer:output = realOutput
case output <- req:output = nilreq = new(Request)
}}
![Page 22: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/22.jpg)
Batching and I/O- buffer-producer
for request := range input {response, err := broker.Produce(request)
switch err.(type) {// ...
}
p.handleResponse(response)}
![Page 23: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/23.jpg)
Try, Try Again
dispatcherpartition producer
partition producer...
broker producer broker producer... broker producer broker producer...
![Page 24: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/24.jpg)
Try, Try Again (option #1)for {
select {case msg := <-input:
buf = append(buf, msg)case ack := <-acks:
// ...}
for partition := range response {if partition.success {
partition.sendAck()} else {
partition.sendNack()}
}
partition producer
partition producer...
broker producer broker producer...
![Page 25: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/25.jpg)
Try, Try Again (option #2)if msg.retries > 0 {
// ...}
for partition := range response {if !partition.success {
for msg := range partition {msg.retries++dispatcher <- msg
}}
}
dispatcher
broker producer broker producer...
![Page 26: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/26.jpg)
A
B
Try, Try Again (continued)
![Page 27: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/27.jpg)
Try, Try Again (continued)
A
B
select {case msg := <-input:
// ...case output <- msg:
// ...}
![Page 28: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/28.jpg)
Try, Try Again (continued)
A
B
![Page 29: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/29.jpg)
Try, Try Again (continued)- if it’s stupid, but it work (https://github.com/eapache/channels/)for {
if len(buf) == 0 {msg = <-p.retries
} else {select {case msg = <-p.retries:case p.input <- buf[0]:
buf = buf[1:]continue
}}buf = append(buf, msg)
}
dispatcher
retrier
broker producer broker producer...
![Page 30: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/30.jpg)
Putting it all togetherdispatcher
topic producer
partition producer
topic producer
partition producer
aggregator
broker producer
aggregator
broker producer
retrier
...
... ...
...
...
KafkaCluster
![Page 31: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/31.jpg)
Consumer
subscription manager
broker consumer
dispatcher
response feeder
dispatcher
response feeder
subscription manager
broker consumer
...
...
...
...
KafkaCluster
![Page 32: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/32.jpg)
Structure Your Goroutines
Anonymous -> Named -> Structured
![Page 33: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/33.jpg)
Structure Your Goroutines
Anonymous
go func() {
// ...
}()
Named
func foo() {
// ...
}
go foo()
![Page 34: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/34.jpg)
Structure Your Goroutines
Structured
type foo struct {
// ...
}
func (f *foo) run() {
// ...
}
func newFoo(...) {
foo := &foo{
// ...
}
go foo.run()
}
![Page 35: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/35.jpg)
Ownership Semantics
dispatcher dispatcher...
...
![Page 36: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/36.jpg)
Ownership Semantics- dispatcher
trigger := make(chan struct{}, 1)
for _ = range trigger {broker, err := findNewLeader()if err != nil {
time.Sleep(...)trigger <- struct{}{}
} else {broker.subscribe <- partition
}}
![Page 37: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/37.jpg)
Ownership Semantics- broker
for partition, messages, err := range response {if err != nil {
delete(subscriptions, partition)partition.trigger <- struct{}{}continue
}
sendToUser(messages)}
![Page 38: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/38.jpg)
Isolating I/O (redux)
subscription manager
broker consumer Kafka
Cluster
![Page 39: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/39.jpg)
Feeding the User
broker consumer
response feeder
response feeder
broker consumer
...
... KafkaCluster
![Page 40: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/40.jpg)
Feeding the User (response-feeder)for messages := range input {
for msg := range messages {select {case output <- msg:case <-time.After(timeout):
delete(broker.subscriptions, partition)broker.acks.Done()// feed remaining messagesbroker.subscribe <- partitioncontinue outerLoop
}}broker.acks.Done()
}
![Page 41: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/41.jpg)
Feeding the User (broker-consumer)
broker.acks.Add(len(subscriptions))
for sub := range subscriptions {sub.feeder <- response.messages[sub]
}
broker.acks.Wait()
![Page 42: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/42.jpg)
Consumer
subscription manager
broker consumer
dispatcher
response feeder
dispatcher
response feeder
subscription manager
broker consumer
...
...
...
...
KafkaCluster
![Page 43: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/43.jpg)
1. Channels are primitives.2. Structure your goroutines.3. Don’t trust the network or the user.4. Infinite buffers smell.5. Don’t be afraid of locks and “anti-go” tricks.
Lessons Learned
![Page 44: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/44.jpg)
Credits● Photo of Franz Kafka: public domain (via Wikimedia
Commons).● Photo of José Saramago: CC-BY 2.0 (from the website of
the Presidencia de la Nación Argentina, via Wikimedia Commons)
● Photo of Donald Knuth: CC-BY-SA 2.5 (by Jacob Appelbaum, via Wikimedia Commons)
● Tweet from @caitie: used with permission.
![Page 45: Patterns in Go Complex Concurrency - Evan Huus · Goroutine Input Output Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine Goroutine In Theory](https://reader031.vdocuments.site/reader031/viewer/2022022016/5b5ebffa7f8b9a51328cf26e/html5/thumbnails/45.jpg)
Questions?@eapache
[email protected]://eapache.github.io
(feedback: https://joind.in/talk/view/14954)