演讲：Go, a global phenomenon (英语演讲)

A Global Phenomenon 深圳市 China – July 7 2018 Steve Francia Google @spf13

Google’s Journey The Journey to Go Go’s adoption & growth Go’s journeys in open source Go’s journeys into production Go’s journey forward Your journey

Google’s Journey SECTION 1

2 kinds of scale

Machine

Human

Urs Hölzle
Google: ‘At scale, everything breaks’ – 2011 Keeping things simple and yet scalable is actually the biggest challenge.

Google’s scale brought complexity 03. REAL WORLD GO

• Massive codebases with many dozens of programmers
• Very large build farms
• 45+ min builds

Brian Kernighan
Software Tools – 1976 Controlling complexity is the essence of computer programming.

During a 45 minute C++ build in 2007

Go’s Journey SECTION 2

Rob Pike Rob Pike’s words: Back around September 2007, I was doing some minor but central work on an enormous Google C++ program, one you’ve all interacted with, and my compilations were taking about 45 minutes on our huge distributed compile cluster. An announcement came around that there was going to be a talk presented by a couple of Google employees serving on the C++ standards committee. They were going to tell us what was coming in C++0x, as it was called at the time. (It’s now known as C++11).

In the span of an hour at that talk we heard about something like 35 new features that were being planned. … At this point I asked myself a question: Did the C++ committee really believe that was wrong with C++ was that it didn’t have enough features? Surely…, it would be a greater achievement to simplify the language rather than to add to it. Of course, that’s ridiculous, but keep the idea in mind.

Just a few months before that C++ talk I had given a talk myself, which you can see on YouTube, about a toy concurrent language I had built way back in the 1980s. That language was called Newsqueak. …

I gave that talk because there were ideas in Newsqueak that I missed in my work at Google and I had been thinking about them again. I was convinced they would make it easier to write server code and Google could really benefit from that.

I actually tried and failed to find a way to bring the ideas to C++. It was too difficult to couple the concurrent operations with C++’s control structures, and in turn that made it too hard to see the real advantages. Plus C++ just made it all seem too cumbersome, although I admit I was never truly facile in the language. So I abandoned the idea.

But the C++11 talk got me thinking again. …

We returned to our offices after the talk. I started another compilation, turned my chair around to face Robert, and started asking pointed questions. Before the compilation was done, we’d roped Ken in and had decided to do something. We did not want to be writing in C++ forever, and we—me especially—wanted to have concurrency at my fingertips when writing Google code. We also wanted to address the problem of “programming in the large” head on.

Ken Thompson Ken comes from US and C & Unix

Robert Griesemer Europe Smalltalk & Pascal

Studied under Niklaus Wirth

What should a modern, practical programming language look like?

CPU speeds decreasing @AyanGeorge

CPU Cores increasing @AyanGeorge

. 2008

Concurrent

Simple

Powerful

Fast

Easy to learn

Expressive

Readable

Type-safe

Dynamic feeling

Compiled

Compiled very quickly

We built from scratch, borrowing [from C] only minor things like operators and brace brackets and a few common keywords. And of course we also borrowed ideas from other languages we knew. … Rob Pike
Less is Exponentially More – 2012 5 min

Go’s Many Ancestors and Influences Alef Algol APL BCPL B C C++ CLU CSP Java Limbo Modula Newsqueak

Oberon Occam Pascal Python Simula Smalltalk

Rich knowledge of what came before from the designers which is very rare.. They knew just what to cherry pick.

C: Statement & expression syntax Pascal: Declaration syntax Modula 2 & Oberon 2: Packages CSP, Occam, Newsqueak, Limbo, Alef: Concurrency BCPL: Semicolon rule Smalltalk: Methods

Nearly 5 years of refinement 2007 2009 2012 Started at Google as a 20% project 2 years later, Go is open sourced After nearly 5 years of development, Go 1.0 is released and attention shifts to using Go Open Source Birth 1.0

Hello World (中国) package main

import “fmt”

func main() { fmt.Println(“Hello, 中国!") }

Go Adoption and Growth SECTION 3 7 min

Google Trends for “Golang”

https://trends.google.com/trends/explore?date=2012-05-27%202018-06-27&geo=US&q=golang

Github 2017 Octoverse https://octoverse.github.com/ Go #9 on GitHub usage & 1st fastest growing in 2017

Stack Overflow Survey 2017 & 2018 https://insights.stackoverflow.com/survey/2018#technology 4.6% 7.1%

Stack Overflow Survey 2018 - Loved & Wanted https://insights.stackoverflow.com/survey/2018#technology 5th Most Loved 3rd Most Wanted

JetBrains 2018 Developer Survey https://www.jetbrains.com/research/devecosystem-2018/

14 Conferences

140 Meetups

Over 50 Countries

Google Search for “Golang” https://trends.google.com/trends/explore?date=2012-05-27%202018-06-27&geo=US&q=golang

GopherChina in Shanghai

Go Users Worldwide 2018

1.5 – 2 Million

Go’s Journeys in Open source SECTION 4 10.5 min

4 of the 32 InfoWorld’s 2017 Technology of the Year Award winners were created in Go InfoWorld editors and reviewers pick the year’s best hardware, software, development tools, and cloud services

49

Why Docker Chose Go Michael Crosby – 2013 Static compilation Neutral language It has what we needed Full development environment Multi-arch build

Joe Beda
Kubernetes Creator — 2018 Kubernetes' success is in part due to Go and the Go community. Go hits the sweet spot for distributed systems software. It is low enough to do the job but high level enough to attract a thriving community https://twitter.com/jbeda/status/1012486879248371713

Why Kubernetes Chose Go Joe Beda – 2014 Great libraries Fast tools A “keep it simple” culture Gofmt Concurrency Garbage Collection Type Safety https://blog.gopheracademy.com/birthday-bash-2014/kubernetes-go-crazy-delicious/

15 min

Steve Francia Hugo Creator — 2018 Without Go, Hugo wouldn’t exist. Go strikes the perfect balance of developer productivity and performance. It has a rich ecosystem of libraries making it ideal for both web and systems programming. Me here and now.

Why Hugo Chose Go Steve Francia – 2013 Performance Code simplicity Rich library support Especially for web Single binary distribution Cross compilation Fast complication

Go’s Journeys in Production SECTION 5 18.5 min

TECH

GAMING/MEDIA

BANKING/COMMERCE Get rid of ‘go adoption’ on these slides

GENERAL

ENTERPRISE Cass notes - call “Global Production Use Cases”

Patrick Lee Software Engineer @ Dropbox — 2014 We decided to migrate our performance- critical backends from Python to Go to leverage better concurrency support and faster execution speed.

Go at Dropbox 2017 Over 500 million users Most Dropbox infrastructure is written in Go 1.3 million lines of Go Every Dropbox engineer hired goes through Go onboarding

Why Go at Dropbox? Tammy Bütow – 2017 Performance Easy to be productive Easy to both write & consume services in Go Standard Library is very good Debugging tools work well People are happy with Go at Dropbox

Switching to a new language is always a big step, especially when only one of your team members has prior experience with that language. Early this year, we switched Stream’s primary programming language from Python to Go.

As part of this switch they ran an experiment writing the same functionality in both languages.

Theirry Schellenbach Founder @ Stream — 2017 Developing the Python version of the ranking code took roughly 3 days.

Next, we’ve spent approximately 2 weeks optimizing the code.

https://getstream.io/blog/switched-python-go/

Theirry Schellenbach Founder @ Stream — 2017 In contrast, developing the Go version of this code took roughly 4 days. The performance didn’t require any further optimization. As an added benefit, the Go code performed roughly 40 times faster than our highly-optimized Python code. https://getstream.io/blog/switched-python-go/

The ranking code was my first project in Go The Go code was built after the Python code, so the use case was better understood

Your mileage will vary. Some other components of our system took substantially more time to build in Go compared to Python. As a general trend, we see that developing Go code takes slightly more effort. However, we spend much less time optimizing the code for performance.

Why Go at Stream? 2017 Great performance Developer productivity Concurrency & channels Fast compile time Easy to learn & easy to hire Go Mature, production ready libraries

25 min

https://eng.uber.com/go-geofence/

2015 move away from a monolithic architecture to a (micro)service-oriented architecture, Geofence functionality was scattered in lots of services, they chose to centralize this functionality into a single new microservice.

Kai Wei Uber — 2016 In early 2015 we built a microservice that does one thing (and does it really well), geofence lookups. One year later, this service is Uber’s highest queries per second (QPS) service out of the hundreds we run in production.

https://eng.uber.com/go-geofence/

Why Go at Uber? 2016 High throughput & low latency Able to handle: CPU intensive workload Concurrent workload High developer productivity Super reliable https://eng.uber.com/go-geofence/

Kai Wei Uber — 2016 Go typically takes just a few days for a C++, Java or Node.js developer to learn, and the code is easy to maintain. (Thanks to static typing, no more guessing and unpleasant surprises). https://eng.uber.com/go-geofence/

Contributor workshop in Denver 2017

Go’s Journey Forward SECTION 6 28 min

2

Nearly 5 years of refinement 2007 2009 2012 Started at Google as a 20% project 2 years later, Go is open sourced After nearly 5 years of development, Go 1.0 is released and attention shifts to using Go Open Source Birth 1.0 We spent 5 years developing Go

2012 After nearly 5 years of development, Go 1.0 is released and attention shifts to using Go 1.0 5 years of gaining experience & refining the implementation 2017 After nearly 5 years of development using Go, attention shifts to using making language changes again Go 2 begins We spent 5 years developing Go

2012 After nearly 5 years of development, Go 1.0 is released and attention shifts to using Go 1.0 Go 2 is a process with a series of releases starting with Go 1.11 2018 - ?? A series of releases starting with Go 1.11 in August 2018

Might just be Go 1.x for increasing x. 2 2017 After nearly 5 years of development using Go, attention shifts to using making language changes again Go 2 begins We spent 5 years developing Go

Dependency Management SECTION 6.1

go get (for Go 1) 2012 Done for Go 1.0

Example of go get

Import package using canonical address

go get

import “github.com/spf13/cobra” $ go get github.com/spf13/cobra

go get got a lot right No configuration outside source code (just import statement) Very easy to use URL-like import paths Eliminated central hub No diamond dependency problems Worked very well in practice

go get got some things wrong Only update operation was “upgrade everything to latest code” No explicit concept of versions at all. No good way to make a backwards-incompatible change Unreproducible builds

Lots of community experimentation Unfortunately these fragmented the user base

go get (for Go 2) Go upgrades Look for patterns that are problematic and fix it.

It’s a complex thing.. Only walking through one part of it.

Semantic import versioning

Russ Cox
“Import compatibility rule” — 2018 If an old package and a new package have the same import path, the new package must be backwards compatible with the old package. Been saying similar things since 2013.

Since 2013 the Go FAQ has encouraged package developers to provide their own users with similar expectations of compatibility. We call this the import compatibility rule: “If an old package and a new package have the same import path, the new package must be backwards compatible with the old package.”

By extension this means that if a new package is not backwards compatible than it must have a different import path

By extension this means that if a new package is not backwards compatible than it must have a different import path

Semantically versioned imports

Import package using canonical address

go build

import “github.com/spf13/cobra/v2” $ go build Boxes

Other big parts Modules (different from packages, repos) Minimum version selection Reproducible, verifiable, verified builds

Hardly any change in user workflow (no extra tool) Integrated into standard go commands

Go 1.11

(August 2018) Addresses major enterprise pain point (dependency management); substantially improves debugging and latency diagnosis; eliminates OOMs in prod with automatic GC tuning; adds experimental compiler support for WebAssembly (WASM).

Go 1.11 Dependency Management Improved debugging and latency diagnosis Improved GC automatic tuning Experimental support for WebAssembly (WASM).

Generics SECTION 6.2 37 min

Discussed from the earliest of days

• Asked for from the beginning.
• The details of generics involve making a lot of subtle type system decisions.
• Point of language was to do concurrency, not generics or type systems.
• Friends who worked on Java Generics told us to be very very careful. Told us to not do it.
• So we didn’t.
• Ian was very interested, floated proposals about once a year, all had serious problems (proposals now public).
• Still thinking about it.

Waiting for Good design No is temporary, Yes is forever

Joshua Bloch
Joshua Bloch: A conversation about design – 2002 When in doubt, leave it out The extreme programming proponents don’t advocate writing something that will barely work as fast as you can. They don’t advise you to forgo any design. They do advocate leaving out the bells, whistles, and features you don’t need and add them later, if a real need is demonstrated. And that’s incredibly important, because you can always add a feature, but you can never take it out. Once a feature is there, you can’t say, sorry, we screwed up, we want to take it out because other code now depends on it. People will scream. So, when in doubt, leave it out

Go’s Future Growth SECTION 6.3 39 min

JetBrains 2018 Developer Survey https://www.jetbrains.com/research/devecosystem-2018/

JetBrains 2018 Developer Survey https://www.jetbrains.com/research/devecosystem-2018/

19%

HackerRank 2018 Developer Survey https://research.hackerrank.com/developer-skills/2018/ 38% Go

HackerRank 2018 Developer Survey https://research.hackerrank.com/developer-skills/2018/ 6.9% of employers

Your Journey SECTION 7 41 min

spf@golang.org @spf13 Maybe you already use Go

Maybe you relate to these stories

Going crazy from the complexity of your project

Start your own journey today

Joshua Bloch Coders at Work — 2009

When you choose a language, you’re choosing more than a set of technical trade-offs, you’re choosing a community.

Download

golang.google.cn

Join our online community https://studygolang.com

Watch some videos youtube.com/golang

Take a free course to learn Go greatercommons.com/cwg

Read a book on Go go.wuhaolin.cn

Read a book on Go

Start your own journey today

Thank You 深圳市 China – July 7 2018 Steve Francia Google @spf13

Appendix SECTION ONE

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. Brian Kernighan

Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves. Alan Kay

Women Who Go 26 Chapters World Wide // www.womenwhogo.org

GoBridge Providing workshops for underrepresented groups // www.golangbridge.org

Learning together!

Contributor workshop in Denver 2017

Bill Kennedys workshop here just a few days ago

A meetup in SF

Mariano Gappa Movio — 2017 A feedback loop of one second on our daily struggle with computers has made us more productive and happy. https://twitter.com/YiGeNaNa/status/1012921086286225409

Kathryn Jonas Lead Engineer @ The Economist — 2018 As The Economist went through a massive transition in every element of our tech stack, Go was there to provide simple, efficient tools that enabled our team to build powerful, scalable applications and remain sane throughout the process, we even had fun! https://twitter.com/YiGeNaNa/status/1012921086286225409

XXX Software Engineer @ x — 2015 Go is not like python or ruby, which “allow” you to write understandable code. In Go you basically have no other choice. Why Go is doomed to succeed

Mariano Gappa Movio — 2017 It took some of us six months… to be able to get relatively comfortable with Scala. In contrast, we picked up Go in two weeks https://twitter.com/YiGeNaNa/status/1012921086286225409

The Impact of Go SECTION ONE

ROB PIKE Go’s purpose is to make its designers' programming lives better.

Designed through Collaboration

Go started as a collaboration among 3 people MAYBE THE EFFORTS GOING ON INTERNALLY IE: ANDY AND CASS TEAMS

All in one room talking to each other MAYBE THE EFFORTS GOING ON INTERNALLY IE: ANDY AND CASS TEAMS

NEED A QUOTE OF WHEN YOU LET PEOPLE BE PART OF THE PROCESS THEY FEEL THEY ARE PART OF A COMMUNITY

As the team grew we needed to establish better communication & processes MAYBE THE EFFORTS GOING ON INTERNALLY IE: ANDY AND CASS TEAMS

Go Design Process

https://github.com/golang/proposal

https://golang.org/s/proposal-review

https://github.com/golang/go/issues/23459

https://github.com/golang/go/issues/23459

Go started as a collaboration among 3 people MAYBE THE EFFORTS GOING ON INTERNALLY IE: ANDY AND CASS TEAMS

Now many hundreds of people collobrate MAYBE THE EFFORTS GOING ON INTERNALLY IE: ANDY AND CASS TEAMS

Conferences Across the World

Contributor workshop in Denver 2017

Bill Kennedys workshop here just a few days ago

A meetup in SF

2017 2