lca2014 - introduction to go
DESCRIPTION
Google's Go is a relatively new systems programming language that has recently gained a lot of traction with developers. It brings together the ease and efficiency of development in modern interpreted languages like Python, Perl, and Ruby with the efficiency and safety of a statically typed, compiled language like C/C++ and Java. On top of that, Go is a language built for modern hardware and problems. With built-in support for concurrency, programmers can easily build software to scale up to today's many-core beasts. Programming in Go is really nice, and in this tutorial, you will learn why. We will cover an introduction to the Go programming language, and together we will build a multi-user network service demonstrating all of the major principles of programming in Go.TRANSCRIPT
Introduction to Goa tutorial for developers
Hello! While you’re sitting down, please make sure your Go environment is set up.
!Grab a USB stick that’s floating around and copy
the gostick directory somewhere local. !
Then read the included README please!
Autobiography
• Mark Smith <[email protected]> @zorkian
• Site Reliability Engineer at Dropbox
• Contributes to Dreamwidth Studios
• github.com/xb95
Today’s Plan• What is Go? Where does it fit?
• Obligatory Hello World
• Contrived example program
• Anti-unicorns
• Wrap-up
Go in a Nutshell
• Designed for building systems
• Statically compiled, garbage collected
• Static typing (w/some duck typing)
• Built-in maps (dicts) and strings
Go in a Nutshell
• Multi-core support
• Concurrency primitives
• Closures, etc. etc.
• “Somewhere between C and Python”
The Go Way
• Go is an opinionated language
• gofmt, mixedCaps, capitalization for privacy…
• Simple is better than complex
• The compiler should help with the heavy lifting
Hello World
Hello World
package main !import "fmt" !func main() { fmt.Println("Hello, LCA 2014!") }
Hello World
package main !import "fmt" !func main() { fmt.Println("Hello, LCA 2014!") }
Hello World
package main !import "fmt" !func main() { fmt.Println("Hello, LCA 2014!") }
Hello World
package main !import "fmt" !func main() { fmt.Println("Hello, LCA 2014!") }
Hello World
package main !import "fmt" !func main() { fmt.Println("Hello, LCA 2014!") }
Building Hello World
# Do this in your gostick/ copy
# You did follow the README? :-)
cd helloworld/
go build
./helloworld
Getting Real
• Go is particularly suited for network services
• The standard library is fairly comprehensive
• Let’s build an echo server!
Echo Server
1. Listen on port for TCP connections
2. Accept connections
3. Read text from connection
4. Write it back to connection
Standard Library
• Go has a decent standard library
• The ecosystem is still fairly young, so it has some holes and some things aren’t well optimized
• Well built for network services
• http://golang.org/pkg/
Tip: Searching
When you search the Internet for information, “Go” is a bad keyword;
everybody uses “golang”.
Echo Server
1. Listen on port for TCP connections
2. Accept connections
3. Read text from connection
4. Write it back to connection
Listening for Connections
func Listen(net, laddr string) (Listener, error)
In the “net” package, you’ll find many useful things, including:
Listening for Connections
package main !import "net" !func main() { !}
func Listen(net, laddr string) (Listener, error)
In the “net” package, you’ll find many useful things, including:
Listening for Connections
package main !import "net" !func main() { ... := net.Listen("tcp", ":9000") }
func Listen(net, laddr string) (Listener, error)
In the “net” package, you’ll find many useful things, including:
Variable Definition• Go has two ways of declaring variables, explicit and implicit
• Explicit:var foobar uint64
• Implicit (type is inferred automatically):foobar := thingThatReturnsUint64()
• Go strives to save on typing redundant information that the compiler can figure out
Error Handling• Using multiple return values to get errors
• Idiomatically, you will write this a lot:results, err := pkg.DoSomething(1, 2) if err != nil { log.Fatalf(“Failed: %s”, err) } // carry on and use results
• Yes, this gets very verbose… oh well
Echo Server
1. Listen on port for TCP connections
2. Accept connections
3. Read text from connection
4. Write it back to connection
Connection Pump
func main() { listener, err := net.Listen("tcp", ":9000") !!!!!!!!!!!}
func Accept() (Conn, error)
Connection Pump
func main() { listener, err := net.Listen("tcp", ":9000") if err != nil { panic(err) } !!!!!!!!}
func Accept() (Conn, error)
Connection Pump
func main() { listener, err := net.Listen("tcp", ":9000") if err != nil { panic(err) } ! for { !!!! } }
func Accept() (Conn, error)
Connection Pump
func main() { listener, err := net.Listen("tcp", ":9000") if err != nil { panic(err) } ! for { client, err := listener.Accept() !!!! } }
func Accept() (Conn, error)
Connection Pump
func main() { listener, err := net.Listen("tcp", ":9000") if err != nil { panic(err) } ! for { client, err := listener.Accept() if err != nil { continue } ! } }
func Accept() (Conn, error)
Connection Pump
func main() { listener, err := net.Listen("tcp", ":9000") if err != nil { panic(err) } ! for { client, err := listener.Accept() if err != nil { continue } handleClient(client) } }
func Accept() (Conn, error)
Echo Server
1. Listen on port for TCP connections
2. Accept connections
3. Read text from connection
4. Write it back to connection
Client Handlerfunc Read(b []byte) (int, error)
func handleClient(client net.Conn) { for { // read from our client // write it back to our client } }
Client Handlerfunc Read(b []byte) (int, error)
func handleClient(client net.Conn) { for { // read from our client // write it back to our client } }
Okay, so what’s a []byte?
Primitive Go Types
• All of the usual suspects (ints, uints, floats)
• Built-in string type (Unicode, immutable)
• int and uint are architecture-width
• byte is just a synonym for uint8
More Types• arrays: of a declared, fixed length
• slice: a segment (“slice”) of an array
• map: key/value storage
• pointer : much like C (uses & and *)
• (more to come later)
So, []byte…
• Let’s make an array of bytes:var ary [4096]byte
• What if we don’t know the size? Or don’t care? Slices solve this problem:var aryslice []byte
• This is great, but what is it?
Slices Explained
var a [16]byte is
Slices Explained
var a [16]byte is
a[3] is
Slices Explained
var a [16]byte is
a[3] is
a[6:8] is
Slices Explained
var a [16]byte is
a[3] is
a[6:8] is
var s []byte is nothing to start with.
Slices Explained
var a [16]byte is
a[3] is
a[6:8] is
var s []byte is nothing to start with.
s = a[6:8]; s is
Slices Explained
var a [16]byte is
a[3] is
a[6:8] is
var s []byte is nothing to start with.
s = a[6:8]; s is
s[0] is the same as a[6], etc!
A slice is simply a window into a backing array.
Slices pt. 2
• Slices are internally a tuple of (array, start, length)
• A slice can be moved around (a sliding window) and resized (limited to the backing array size)
• Slices are reference types; great for passing to functions
Echo Server
1. Listen on port for TCP connections
2. Accept connections
3. Read text from connection
4. Write it back to connection
Client Handlerfunc Read(b []byte) (int, error)
func handleClient(client net.Conn) { for { !!!!!! } }
Client Handlerfunc Read(b []byte) (int, error)
func handleClient(client net.Conn) { for { buf := make([]byte, 4096) !!!! } }
Client Handlerfunc Read(b []byte) (int, error)
func handleClient(client net.Conn) { for { buf := make([]byte, 4096) numbytes, err := client.Read(buf) !!!! } }
Client Handlerfunc Read(b []byte) (int, error)
func handleClient(client net.Conn) { for { buf := make([]byte, 4096) numbytes, err := client.Read(buf) if numbytes == 0 || err != nil { return } ! } }
Client Handlerfunc Read(b []byte) (int, error)
func handleClient(client net.Conn) { for { buf := make([]byte, 4096) numbytes, err := client.Read(buf) if numbytes == 0 || err != nil { return } client.Write(buf) } }
Build & Test# Do this in your gostick/ copy
cd part1/
go build
./part1 &
telnet localhost 9000
# say something and hit enter
More Power
• The echo server is great, but can only serve one user at a time!
• Concurrent network programming… should we fork for each child? use a threading library? maybe we can implement it using non-blocking I/O…
• Stop, stop, stop!
Concurrent, Go Style
func main() { listener, err := net.Listen("tcp", ":9000") if err != nil { panic(err) } ! for { client, err := listener.Accept() if err != nil { continue } handleClient(client) } }
Remember our main accept loop? Let’s tweak it from this…
Concurrent, Go Style
func main() { listener, err := net.Listen("tcp", ":9000") if err != nil { panic(err) } ! for { client, err := listener.Accept() if err != nil { continue } go handleClient(client) } }
…to this!
Goroutines
• A goroutine is a function executing concurrently with other goroutines
• Go multiplexes M goroutines onto N processes, and scales to 100,000+ goroutines (millions possible, use case dependent)
• N is by default only 1, you can tune it
Goroutines pt. 2
• Everything in Go is designed to be blocking, in essence, and the idea is to use goroutines
• This makes reasoning about software much easier
• Go also provides deadlock detection and backtraces all living goroutines
Echo v2.0, “Chat”
• Let’s make a chat server out of our echo server
• Design goal: any text from one client is echoed immediately to all connected clients
• Well, clearly, our goroutines have to communicate
Communication
• The common idiom for communicating between threads in most languages is shared memory
• This kind of work is notoriously racy, error prone, and hard to implement correctly
Communication in Go
• “Don’t communicate by sharing memory; share memory by communicating!”
• Enter the concept of channels
• A built-in goroutine safe method of passing data
Basic Channel Use
ch := make(chan int)
ch <- 5
i := <-ch
Create a new channelWrite to a channelRead from a channel
The channel in this example is an unbuffered channel. Reads block until data is available, writes block until someone reads. Synchronous.
Buffered Channels
ch := make(chan int, 10)
ch <- 5
i := <-ch
Create a new channelWrite to a channelRead from a channel
The difference: buffered channels won’t block when inserting data (unless they’re full).
Chat Server• We have to make three main modifications:
1. Store a list of connected clients
2. Get input out of the clients
3. Write each input back to every client
• Note: this implementation is a reduced example, so it’s a little unsafe in one place :-)
Chat Server• We have to make three main modifications:
1. Store a list of connected clients
2. Get input out of the clients
3. Write each input back to every client
• Note: this implementation is a reduced example, so it’s a little unsafe in one place :-)
Connected Clientsfunc main() { ... !!! for { client, err := listener.Accept() if err != nil { continue } ! go handleClient(client) } }
Connected Clientsfunc main() { ... ! var clients []net.Conn ! for { client, err := listener.Accept() if err != nil { continue } clients = append(clients, client) go handleClient(client) } }
Build a slice of clients and then append
each new client to the slice.
!The append built-in handles automatically
allocating and growing the backing array as necessary.
Chat Server• We have to make three main modifications:
1. Store a list of connected clients
2. Get input out of the clients
3. Write each input back to every client
• Note: this implementation is a reduced example, so it’s a little unsafe in one place :-)
Getting Inputfunc main() { ... ! for { ... clients = append(clients, client) go handleClient(client) } } !func handleClient(client net.Conn) { for { ... client.Write(buf) } }
Getting Inputfunc main() { ... input := make(chan []byte, 10) ! for { ... clients = append(clients, client) go handleClient(client) } } !func handleClient(client net.Conn) { for { ... client.Write(buf) } }
Getting Inputfunc main() { ... input := make(chan []byte, 10) ! for { ... clients = append(clients, client) go handleClient(client, input) } } !func handleClient(client net.Conn) { for { ... client.Write(buf) } }
Getting Inputfunc main() { ... input := make(chan []byte, 10) ! for { ... clients = append(clients, client) go handleClient(client, input) } } !func handleClient(client net.Conn, input chan []byte) { for { ... client.Write(buf) } }
Getting Inputfunc main() { ... input := make(chan []byte, 10) ! for { ... clients = append(clients, client) go handleClient(client, input) } } !func handleClient(client net.Conn, input chan []byte) { for { ... client.Write(buf) input <- buf } }
Getting Inputfunc main() { ... input := make(chan []byte, 10) ! for { ... clients = append(clients, client) go handleClient(client, input) } } !func handleClient(client net.Conn, input chan []byte) { for { ... client.Write(buf) input <- buf } }
Removed the Write!
What is Happening?
• handleClient still is a blocking read loop, but instead of writing back to the client it writes the bytes onto a channel
• main now has a list of all clients and a channel that everybody is writing input to
• Final piece: somebody to read from the channel!
Chat Server• We have to make three main modifications:
1. Store a list of connected clients
2. Get input out of the clients
3. Write each input back to every client
• Note: this implementation is a reduced example, so it’s a little unsafe in one place :-)
The Chat Manager
func main() { ... var clients []net.Conn input := make(chan []byte, 10) !!!!!!!! ... }
We’ve seen this all before…
The Chat Manager
func main() { ... var clients []net.Conn input := make(chan []byte, 10) go func() { !!!!! }() ... }
You can create goroutines out of closures, too.
!
The Chat Manager
func main() { ... var clients []net.Conn input := make(chan []byte, 10) go func() { for { message := <-input !! } }() ... }
!!!
This is all blocking!
The Chat Manager
func main() { ... var clients []net.Conn input := make(chan []byte, 10) go func() { for { message := <-input for _, client := range clients { ! } } }() ... }
!!!
This is all blocking!
range and _for _, client := range clients {..}
• The range keyword iterates over maps/slices/arrays and returns the key (or index) and value
• Underscore (_) is the anonymous variable (“blank identifier”), you can use it to discard results
The Chat Manager
func main() { ... var clients []net.Conn input := make(chan []byte, 10) go func() { for { message := <-input for _, client := range clients { client.Write(message) } } }() ... }
Build & Test# Do this in your gostick/ copy
cd part2/
go build
./part2 &
telnet localhost 9000
# say something and hit enter, now connect
# again in another window and chat! :-)
Echo 3.0, “Frotzer”
• We’re entering the land of extremely contrived tutorial examples, but…
• Let’s give our chat server some behaviors!
• When a user chats, we apply some formatting rules before sending it out to other users
But First: More Type Talk!
• Go doesn’t have classes (or objects, really)
• However, you have named types, and methods are attached to named types:
type Uppercaser struct{} !func (self Uppercaser) Frotz(input []byte) []byte { return bytes.ToUpper(input) }
Um, struct{}?
• Go supports structs, very much like any other language that has structs
• The empty struct is often used for hanging methods
• Instantiating a struct type:foobar := Lowercaser{} foobar.Frotz(“HELLO”) // “hello”
Interfaces
• In Go, an interface specifies a collection of methods attached to a name
• Interfaces are implicit (duck typed)
type Frotzer interface { Frotz([]byte) []byte } !type Uppercaser struct{} func (self Uppercaser) Frotz(input []byte) []byte { return bytes.ToUpper(input) }
Implementing Frotzing
func chatManager(clients *[]net.Conn, input chan []byte ) { for { message := <-input for _, client := range *clients { client.Write(message) } } }
This is the input handler loop we built a few minutes ago, except now we pulled it out of main and made it a
function.
Implementing Frotzing
func chatManager(clients *[]net.Conn, input chan []byte, frotz Frotzer) { for { message := <-input for _, client := range *clients { client.Write(frotz.Frotz(message)) } } }
Now it takes a third argument: an interface. Any type that implements Frotzer can be used!
Making Frotzers
These are both named types, and both implement (implicitly!) the Frotzer interface. Thanks, compiler!
type Uppercaser struct{} func (self Uppercaser) Frotz(input []byte) []byte { return bytes.ToUpper(input) } !type Lowercaser struct{} func (self Lowercaser) Frotz(input []byte) []byte { return bytes.ToLower(input) }
A New Mainfunc main() { ... var clients []net.Conn input := make(chan []byte, 10) go chatManager(&clients, input) !! for { client, err := listener.Accept() if err != nil { continue } clients = append(clients, client) go handleClient(client, input) } }
A New Mainfunc main() { ... var clients []net.Conn input := make(chan []byte, 10) go chatManager(&clients, input, Lowercaser{}) go chatManager(&clients, input, Uppercaser{}) ! for { client, err := listener.Accept() if err != nil { continue } clients = append(clients, client) go handleClient(client, input) } }
A New Main pt. 2
• The chatManager functions are spawned into separate goroutines
• Channels are goroutine safe, so they end up interleaving reads (not guaranteed!)
• chatManager doesn’t know what you’re passing it as a Frotzer, it just knows it can call Frotz on it
Build & Test# Do this in your gostick/ copy
cd part3/
go build
./part3 &
telnet localhost 9000
# say something and hit enter, your text
# should alternate UPPER/lower
Review
• A multi-user chat server demonstrating many core parts of the Go programming language
• Built-in concurrency that is easy to use and trivially allows relatively powerful constructions
• A classless, statically type system that still enables many OO concepts
Go is Awesome :-)
This slide should have unicorns and bunnies on it
Things That Aren’t Unicorns Yet
• The GC is generally pretty solid, but if you are doing many allocations and need performance, you have to do the usual tricks
• Standard library is pretty young and not optimized
• Overall ecosystem is small (but growing, hi!)
Also: The Scheduler
• It’s new and has room for improvement
• Heavy use of channels between goroutines is probably faster in one process
• You’ll have to play with GOMAXPROCS and understand your application
Topics Undiscussed
• Building your own packages (it’s easy)
• Importing from external repositoriesimport pcap “github.com/akrennmair/gopcap”
• Using gofmt (do it!)
• switch, select, etc.
• Type assertions, reflection, even more etc.
Thank you!
• Some recommended reading:Effective Go golang.org/doc/effective_go.htmlFAQ golang.org/doc/faq
• Many more talks and presentations:code.google.com/p/go-wiki/wiki/GoTalks
• The Go Playground! play.golang.org
Mark Smith <[email protected]> @zorkianSRE at Dropbox (we’re hiring!)