Getting Started with Go

This presentation was given as a Workshop at OSCON 2014.

Description

New to Go? This tutorial will give developers an introduction and practical experience in building applications with the Go language. Gopher Steve Francia, Author of Hugo, Cobra, and many other popular Go packages breaks it down step by step as you build your own full featured Go application.

Starting with an introduction to the Go language. He then reviews the fantastic go tools available. With our environment ready we will learn by doing. The rest of the time is dedicated building a working go web and cli application. Through our application development experience we will introduce key features, libraries and best practices of using Go.

This tutorial is designed with developers in mind. Prior experience with any of the following languages: ruby, perl, java, c#, javascript, php, node.js, or python is preferred. We will be using the MongoDB database as a backend for our application.

We will be using/learning a variety of libraries including:

• bytes and strings
• templates
• net/http
• io, fmt, errors
• cobra
• mgo
• Gin
• Go.Rice
• Cobra
• Viper

Slide Transcript:

1. Building your first Go App 1 Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
2. @spf13 •Author of Hugo, Cobra, Viper & More •Chief Developer Advocate for MongoDB •Gopher 2
3. My Go Story •I write a medium size blog •Frustrated with Wordpress performance & maintenance •Existing Static Site Generators had complicated installations & were very slow •Began writing a S.S.G. in Go called Hugo 1.5 years ago 4
4. hugo.spf13.com
5. Hugo •Hugo is 2nd fastest growing Static Site Generator (40-50 * a week on github) •Hugo is one of the fastest Static Site Generators. 1000x faster than Jekyll •Easiest Installation of any SSG. Download and run •2nd most contributors of any Go project (Docker 1st)6
6. Plan •Introduce Go Language •Introduce Tools & Libraries •Build our application •Tell the world how awesome we are 8
7. Ground Rules •Workshops are hard, everyone works at a different pace •We will move on when about 50% are ready •Slides are online, feel free to work ahead or catch up 9
8. http:// spf13.com/ presentation/ first-go-app/ 10
9. Installing Go •You should already have Go installed •If you don’t, do it NOW •Installation Guide 11
10. Installing MongoDB •You should already have MongoDB installed •If you don’t, do it NOW •Installation Guide 12
11. Git & Mercurial •Lastly we need Git & Mercurial •http://git-scm.com/downloads •http://mercurial.selenic.com/wiki/Download 13
12. Introduction to Go 14
13. Why Another Language? • Software is slow • Sofware is hard to write • Software doesn’t scale well 15
14. Go is Fast • Go execution speed is close to C • Go compile time rivals dynamic interpretation 16
15. Go is Friendly • Feels like a dynamic language in many ways • Very small core language, easy to remember all of it • Single binary installation, no dependencies • Extensive Tooling & StdLib 17
16. Go is Concurrent • Concurrency is part of the language • Any function can become a goroutine • Goroutines run concurrently, communicate through channels • Select waits for communication on any of a set of channels 18
17. Go’s Inspiration •C: statement and expression syntax •Pascal: declaration syntax •Modula 2, Oberon 2: packages •CSP, Occam, Newsqueak, Limbo, Alef: concurrency •BCPL: the semicolon rule •Smalltalk: methods •Newsqueak: <-, := •APL: iota 19 … AND lessons good and bad from all those plus: C++, C#, Java, JavaScript, LISP, Python, Scala, …
18. — txxxxd “Most of the appeal for me is not the features that Go has, but rather the features that have been intentionally left out.” 20
19. — Rob Pike “Why would you have a language that is not theoretically exciting? Because it’s very useful.” 21
20. package main ! import "fmt" ! func main() { fmt.Println(“Hello, 世界") } Hello World 22
21. Package •Not classes or objects •No subpackages •Fundamental building block in Go •Visibility is package-level, not type-level •Programs run “main” package 23 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
22. import •Provides access to a package •Package consists of types, functions, etc… •No circular dependencies 24 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
23. “fmt" •Package path is just a string •Standard library sits at the root (no path) •Core language is very small •Most functionality is in Stdlibs 25 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
24. func •Declaring a function •Same syntax for methods, anonymous functions & closures •main.main() is the function run when the program is executed 26 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
25. Main •No void •No return value •No function args (command line is in os package) 27 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
26. { } •Braces not spaces •Feels like C ( & most languages ) •Newline must be after brace (semicolons are inserted during compiliation) 28 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
27. fmt •Accessing the package imported above •Everything visible in the package will be accessible through the name (or alias if provided) 29 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
28. Println • Println, not println • Println, not print_ln •Capital for export •Variadic function using reflection 30 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
29. “Hello, 世界” •UTF-8 input source •Strings are immutable •Strings are UTF-8 encoded •String is a built-in type 31 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
30. Rob Pike did it better http://confreaks.com/videos/3419- gophercon2014-opening-day-keynote 32
31. Tons more… •types •constants •methods •interfaces •pointers •control flow •conditionals •tools •packages •concurrency 33
32. Effective Go 34
33. Lets write some code ! Introduce the go tools 35
34. package main ! import "fmt" ! func main() { fmt.Println("Hello,World") } Hello World 36 hello.go
35. ❯ go run hello.go Hello,World Go Run 37
36. ❯ go build hello.go ! ❯ls -lh 1.7M Jun 17 22:44 hello 72B Jun 17 22:43 hello.go ! ❯ ./hello Hello, World Go Build 38
37. Go Fmt •Automatically formats Go source code •Ends stylistic debates •Integration with editors (vim, emacs, others) •Can also refactor code (see http://spf13.com/post/go-fmt ) 39
38. go Test •Enables easy testing of application •Integrates with the testing package •Supports benchmark, functional & unit style testing •Combine with ‘looper’ to have realtime feedback 40
39. package main ! import "testing" ! func TestOne(t *testing.T) { one := false if !one { t.Errorf(“Test Failed”) } } Hello Test 41 hello_test.go
40. ❯ go test ./… — FAIL: TestOne (0.00 seconds) hello_test.go:8: Test failed FAIL FAIL_/Code/firstApp0.012s ! ! ❯ go test ./… ok _/Code/firstApp0.012s Go Test 42
41. Go Planet 43The planet logo is based on the Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
42. Why Planet? •CLI application •Web application •Good introduction •Right sized •Database (MongoDB) •Concurrency 45
43. Steps 0. Env Setup 1. Commands 2. Configuration 3. Working with Feeds 4. DB as Storage 5-6. Web Server 7-8. DB -> Templates 9. & 10. Add Polish 46
44. Step 0 Setting up our environment 47
45. Go PAth ! $GOPATH 48
46. Go path •The GOPATH environment variable specifies the location of your workspace (and dependencies) •It must not be the same path as your Go installation 49
47. ❯ mkdir $HOME/go ! ❯ export GOPATH=$HOME/go Setting up 50
48. export GOPATH=$USER/go export GOROOT=go env GOROOT PATH=$PATH:$GOPATH/bin Linux & Mac 51 Add this to your ~/.bashrc (or equivalent)
49. Windows •Control panel > System 52
50. Go PAth •/home/user/gocode/ ($GOPATH) • src/ (put your source code here) • bin/ (binaries installed here) • pkg/ (installed package objects) 53
51. Creating our project 54
52. Project Dir •Make a directory inside of $GOPATH/src •I like mine to be in $GOPATH/src/github.com/spf13/PROJECTNAME •It can not be a symlink •We will call this our working directory or $wd 55
53. package main ! import "fmt" ! func main() { fmt.Println(“My Project") } main.go 56
54. ❯ go run main.go My Project Run it 57
55. 58 I’m on the #gopath with @spf13 at #OSCON http://j.mp/onGoPath Tweet Break
56. Step 1 Creating the Command(s) 59
57. Defining our first command 60
58. Structs •Short for structure •Objectish … blend of data & methods, but no inheritance •Really cheap. Struct{} is free (0 bytes). 61
59. Functions •Can have multiple return values •No overloading •No optional parameters (but variadic) 62
60. First class Functions •Function literals are anonymous functions in Go •Can assign to a variable or field •Can use as function parameters or return values •Can be created as a goroutine 63
61. pointers •Function calls copy arguments into the function •Pointers reference a location in memory where a value is stored rather than the value itself •In Go a pointer is represented using the * •The & operator gives us the address of a variable •Go automatically dereferences pointers when using “.” 64
62. cobra •A CLI Commander •Easy management of commands & flags •Used by bitbucket, openshift & lots more 65 http://fav.me/d5gkby7
63. go get -u github.com/spf13/cobra Getting Cobra 66
64. ❯ mkdir $wd/commands Commands Dir 67
65. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" ! "github.com/spf13/cobra" ) planet.go (1) 68
66. Path = Url go get uses it to install 69
67. var RootCmd = &cobra.Command{ Use: "dagobah", Short: Dagobah is an awesome planet style RSS aggregator, Long: Dagobah provides planet style RSS aggregation. It is inspired by python planet. It has a simple YAML configuration and provides it's own webserver., Run: func(cmd *cobra.Command, args []string) { fmt.Println("Dagobah runs") }, } planet.go (2) 70
68. executing cobra’s command 71
69. Variable Assignment •‘:=‘ is the short variable declaration operator Declares, creates & assigns simultaneously while infering type •Assignment via ‘=‘ requires prior declaration •var foo int = 10 is the same as foo := 10 72
70. Error Handling •Errors are not exceptional. Should be part of the language •Typically one of the return values •No exceptions in Go fd, err := os.Open("test.go") if err != nil { log.Fatal(err) } 73
71. ( Don’t) Panic •Should rarely be used if ever •Performs a full stack trace dump for end users •Use only in the most dire circumstances (like package can’t find FileSystem) •Can use ‘recover’, but only inside deferred functions 74
72. func Execute() { err := RootCmd.Execute() if err != nil { fmt.Println(err) os.Exit(-1) } } planet.go (3) 75
73. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package main ! import "github.com/spf13/firstGoApp-Planet/commands" ! func main() { commands.Execute() } main.go 76 Make sure to use your path
74. ❯ go run main.go Dagobah runs ! ❯ go run main.go help Running it 77
75. Getting Help •https://godoc.org/github.com/spf13/cobra •https://github.com/spf13/firstGoApp-Planet/ tree/step1 78
76. Creating #CobraCommands with @spf13 at #OSCON http://j.mp/cobracmd Tweet Break
77. Step 2 Easy Configuration 80
78. Handing Config files 81
79. Package Identifiers •Export with Capital name •No automatic getters & setters •var field … func Field() … func SetField() are appropriate names 82
80. init() •Each source file can have one (or multiple) •Niladic function to setup state (function without return value) •Called after all variable declarations in the package 83
81. Scoping •Very granular scoping. •Variables declared inside { } only survive in that cotrol structure (for, if, etc) •The := trap if := is used to declare x and x already exists in an outer scope a new variable x will be created (shadowing) 84
82. package main ! import ( "errors" "fmt" ) ! var Foo int ! func Zero() (int, error) { return 0, errors.New("yo") } ! ! func main() { ! Foo = 10 ! fmt.Println("outer:", Foo) // 10 ! if Foo, err := Zero(); err != nil { fmt.Println("inner:", Foo) // 0 } ! fmt.Println("outer again:", Foo) // still 10 } := Trap 85http://play.golang.org/p/i4L9Ao1P65
83. Viper •A configuration manager •Easy loading of config files •Registry for application settings •Designed to work well with (or without) Cobra 86
84. go get -u github.com/spf13/viper Getting Viper 87
85. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" ! "github.com/spf13/cobra" "github.com/spf13/viper" ) planet.go (1) 88
86. var CfgFile string ! func init() { cobra.OnInitialize(initConfig) RootCmd.PersistentFlags().StringVar(&CfgFile, "config", "", "config file (default is $HOME/dagobah/config.yaml)") } ! func initConfig() { if CfgFile != "" { viper.SetConfigFile(CfgFile) } viper.SetConfigName("config") viper.AddConfigPath("/etc/dagobah/") viper.AddConfigPath("$HOME/.dagobah/") viper.ReadInConfig() } planet.go (2) 89
87. Creating our config file 90
88. appname: "Dagobah" feeds: - "http://spf13.com/index.xml" - "http://dave.cheney.net/feed" - "http://www.goinggo.net/feeds/posts/default" - "http://blog.labix.org/feed" - "http://blog.golang.org/feed.atom" ! $HOME/.dagobah/config.yaml 91
89. trying it out 92
90. var RootCmd = &cobra.Command{ Use: "…", Short: ..., Long: ..., Run: rootRun, } ! func rootRun(cmd *cobra.Command, args []string) { fmt.Println(viper.Get("feeds")) fmt.Println(viper.GetString("appname")) } planet.go (3) 93
91. ❯ go run main.go [http://spf13.com/index.xml http:// dave.cheney.net/feed http:// www.goinggo.net/feeds/posts/default http://blog.labix.org/feed http:// blog.golang.org/feed.atom] Dagobah Running it 94
92. I got bit by the #goviper at #OSCON http://j.mp/go-viper Tweet Break
93. https:// github.com/spf13/ firstGoApp- Planet/tree/ step2 96
94. Step 3 Working with Feeds 97
95. Finding Libraries 98
96. Go-Search.org 99
97. Go wiki 100
98. go-pkg-rss •Fetch Rss and Atom feeds from the internet •Supports RSS .92, 1.0, 2.0 & Atom 1.0 •Respects TTL, SkipDays, CacheTimeout, etc in the feeds 101
99. go get -u github.com/jteeuwen/go- pkg-rss Getting Go-PKG-RSS 102
100. boilerplate stuff 103
101. import ( "time" … ) ! func addCommands() { RootCmd.AddCommand(fetchCmd) } ! func Execute() { addCommands() … } planet.go 104
102. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" "time" ! rss "github.com/jteeuwen/go-pkg-rss" "github.com/spf13/cobra" "github.com/spf13/viper" ) Fetch.Go (1) 105
103. Creating our own type 106
104. Named Types •Can be any known type (struct, string, int, slice, a new type you’ve declared, etc) •Can have methods declared on it •Not an alias. Explicit type. 107
105. Slices & arrays •Array is fixed in length •Slice can be viewed as a dynamic array (can grow) •Slice is a view of an array •Slice is the core data structure in Go for lists of data 108
106. type Config struct { Feeds []string Port int } Fetch.Go (2) 109
107. Defining the fetch command 110
108. New( ) •new(T) Does 2 things at once: •Allocates new zeroed value of type T •Returns a pointer to the new value ( *T) •Used for all non-reference types 111
109. Make( ) •Creates slices, maps, and channels only •These types are special “reference” types •Makes a initialized (not zeroed) value of type T •Does not return a pointer 112
110. Composite Literals •An expression that creates a new value each time it is evaluated •We’ve already used these quite a bit •Can also be create values of arrays, slices, and maps •Pseudo Constructor •eg File{fd: fd, name: name} 113
111. Constructors •Not built into Go •Use Composite Literals when possible •If initialization is needed use a factory •Convention is to use New___() •… or New() when only one type is exported in the package 114
112. var fetchCmd = &cobra.Command{ Use: "fetch", Short: "Fetch feeds", Long: Dagobah will fetch all feeds listed in the config file., Run: fetchRun } ! func init() { fetchCmd.Flags().Int("rsstimeout", 5, "Timeout (in min) for RSS retrival") viper.BindPFlag("rsstimeout", fetchCmd.Flags().Lookup("rsstimeout")) } Fetch.go (3) 115
113. Fetching in a goroutine 116
114. GoRoutines •A function executing concurrently with other goroutines in the same address space •Lightweight •Not threads, coroutines, or processes 117
115. Channels •Channels are lightweight •Synchronize goroutines by communication rather than locking shared memory •Channel <- Sender —- Writes block when channel is full •<- Channel —- Reads blocks waiting for something to be read •Great talk on channels given at GopherCon http://confreaks.com/videos/3422-gophercon2014-a-channel-compendium 118
116. Marshaling into •We can’t return a T (no generics) •We can reflect on any value of any type •Marshal accepts the address of the value so it can manipulate the value directly 119
117. func fetchRun(cmd *cobra.Command, args []string) { Fetcher() ! sigChan := make(chan os.Signal, 1) signal.Notify(sigChan, os.Interrupt) <-sigChan } Fetch.go (4) 120
118. func Fetcher() { var config Config if err := viper.Marshal(&config); err != nil { fmt.Println(err) } ! for _, feed := range config.Feeds { go PollFeed(feed.Url) } } Fetch.go (5) 121
119. Building our fetcher 122
120. Learning about go-pkg-rss •How do we learn how to use the package? 123
121. Learning about go-pkg-rss •github.com/jteeuwen/go-pkg-rss •Readme is pretty lacking •godoc.org/github.com/jteeuwen/go-pkg-rss 124
122. godoc.org/github.com/jteeuwen/go-pkg-rss 125
123. Go Search 126
124. For •For, Foreach & While are all spelled ‘for’ in Go for init; condition; post { } // Like a C for for condition { } // Like a C while for { } // Like a C for(;;) 127
125. func PollFeed(uri string) { timeout := viper.GetInt("RSSTimeout") if timeout < 1 { timeout = 1 } feed := rss.New(timeout, true, chanHandler, itemHandler) ! for { if err := feed.Fetch(uri, nil); err != nil { fmt.Fprintf(os.Stderr, "[e] %s: %s", uri, err) return } ! fmt.Printf("Sleeping for %d seconds on %sn", feed.SecondsTillUpdate(), uri) time.Sleep(time.Duration(feed.SecondsTillUpdate() * 1e9)) } } Fetch.GO (6) 128
126. func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %sn", len(newchannels), feed.Url) } ! func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %sn", len(newitems), feed.Url) } Fetch.go (7) 129
127. ❯ go run dagobah.go fetch –rsstimeout=1 15 new item(s) in http://spf13.com/index.xml 1 new channel(s) in http://spf13.com/index.xml Sleeping for 300 seconds on http://spf13.com/index.xml 25 new item(s) in http://www.goinggo.net/feeds/posts/default 1 new channel(s) in http://www.goinggo.net/feeds/posts/default Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/ default 10 new item(s) in http://dave.cheney.net/feed 1 new channel(s) in http://dave.cheney.net/feed Sleeping for 299 seconds on http://dave.cheney.net/feed Running it 130
128. Tweet Break I wrote my first goroutine at #OSCON with @spf13
129. https:// github.com/spf13/ firstGoApp- Planet/tree/ step3 132
130. Step 4 Storing in the Database 133
131. Pretty •github.com/kr/pretty •pretty.Println(X,Y,Z) 134
132. MongoDB •Open Source •Document Database •Document == Struct or Document == Map •Works seamlessly with Go (and other languages) 135
133. MongoDB •Fast & Scalable •Used by Disney, IBM, Metlife, eBay, Forbes, Craigslist, Cisco, Stripe 136
134. Mgo •Developed by the open source community •Heavily used throughout Go community •"mgo is the best database driver I've ever used." — Patrick Crosby, Founder of StatHat •"mgo and Go are a pair made in heaven. — Brian Ketelsen, Author of GopherTimes.com137
135. go get -u gopkg.in/mgo.v2 Getting mgo 138
136. boilerplate stuff 139
137. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" ! "github.com/spf13/viper" "gopkg.in/mgo.v2" ) ! var mongodbSession *mgo.Session ! func init() { RootCmd.PersistentFlags().String("mongodb_uri", "localhost", "host where mongoDB is") viper.BindPFlag("mongodb_uri", RootCmd.PersistentFlags().Lookup(“mongodb_uri")) CreateUniqueIndexes() } MongoDB.go (1) 140
138. Setting up a DB session 141
139. func DBSession() *mgo.Session { if mongodbSession == nil { uri := os.Getenv("MONGODB_URI") if uri == "" { uri = viper.GetString("mongodb_uri") ! if uri == "" { log.Fatalln("No connection uri for MongoDB provided") } } ! var err error mongodbSession, err = mgo.Dial(uri) if mongodbSession == nil || err != nil { log.Fatalf("Can't connect to mongo, go error %vn", err) } ! mongodbSession.SetSafe(&mgo.Safe{}) } return mongodbSession } MongoDB.go (2) 142
140. setting up some shortcuts 143
141. ! func DB() *mgo.Database { return DBSession().DB(viper.GetString("dbname")) } ! func Items() *mgo.Collection { return DB().C("items") } ! func Channels() *mgo.Collection { return DB().C("channels") } MongoDB.go (3) 144
142. Setting up indexes 145
143. func CreateUniqueIndexes() { idx := mgo.Index{ Key: []string{"key"}, Unique: true, DropDups: true, Background: true, Sparse: true, } ! if err := Items().EnsureIndex(idx); err != nil { fmt.Println(err) } ! if err := Channels().EnsureIndex(idx); err != nil { fmt.Println(err) } } MongoDB.go (4) 146
144. Preparing the data 147
145. Data Prep •Feeds are dirty •Go doesn’t allow “patching” of structs (not how structs work) •MongoDB works with native types & named types 148
146. Tags (Annotations) •String literal following a field declaration •Provides additional information during reflection •Used by JSON, BSON, YAML, etc 149
147. for x,y := range •Provides a way to iterate over an array, slice, string, map, or channel. •Like Foreach or Each in other languages •x is key, y is value… can ignore by setting to _ 150
148. Append •Append is really awesome, feels like push, but does a lot more. •Append will grow a slice (both length & capacity) •… and copy the underlying array if needed 151
149. type Itm struct { Date time.Time Key string ChannelKey string Title string FullContent string Links []*rss.Link Description string Author rss.Author Categories []*rss.Category Comments string Enclosures []*rss.Enclosure Guid *string bson:",omitempty" Source *rss.Source PubDate string Id string bson:",omitempty" Generator *rss.Generator Contributors []string Content *rss.Content Extensions map[string]map[string][]rss.Extension } fetch.go 152
150. func itmify(o *rss.Item, ch *rss.Channel) Itm { var x Itm x.Title = o.Title x.Links = o.Links x.ChannelKey = ch.Key() x.Description = o.Description x.Author = o.Author x.Categories = o.Categories x.Comments = o.Comments x.Enclosures = o.Enclosures x.Guid = o.Guid x.PubDate = o.PubDate x.Id = o.Id x.Key = o.Key() x.Generator = o.Generator x.Contributors = o.Contributors x.Content = o.Content x.Extensions = o.Extensions x.Date, _ = o.ParsedPubDate() ! if o.Content != nil && o.Content.Text != "" { x.FullContent = o.Content.Text } else { x.FullContent = o.Description } ! return x } fetch.go 153
151. type Chnl struct { Key string Title string Links []rss.Link Description string Language string Copyright string ManagingEditor string WebMaster string PubDate string LastBuildDate string Docs string Categories []*rss.Category Generator rss.Generator TTL int Rating string SkipHours []int SkipDays []int Image rss.Image ItemKeys []string Cloud rss.Cloud TextInput rss.Input Extensions map[string]map[string] []rss.Extension Id string Rights string Author rss.Author SubTitle rss.SubTitle } fetch.go 154
152. func chnlify(o *rss.Channel) Chnl { var x Chnl x.Key = o.Key() x.Title = o.Title x.Links = o.Links x.Description = o.Description x.Language = o.Language x.Copyright = o.Copyright x.ManagingEditor = o.ManagingEditor x.WebMaster = o.WebMaster x.PubDate = o.PubDate x.LastBuildDate = o.LastBuildDate x.Docs = o.Docs x.Categories = o.Categories x.Generator = o.Generator x.TTL = o.TTL x.Rating = o.Rating x.SkipHours = o.SkipHours x.SkipDays = o.SkipDays x.Image = o.Image x.Cloud = o.Cloud x.TextInput = o.TextInput x.Extensions = o.Extensions x.Id = o.Id x.Rights = o.Rights x.Author = o.Author x.SubTitle = o.SubTitle ! … ! return x } fetch.go 155
153. … ! ! var keys []string for _, y := range o.Items { keys = append(keys, y.Key()) } x.ItemKeys = keys ! … fetch.go 156
154. Storing Feeds & Items 157
155. The Blank Identifier ‘_’ •Used to ignore return values from a function •Used with imports as an alias • Allows import of unused packages so init functions can be executed 158
156. func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %sn", len(newchannels), feed.Url) for _, ch := range newchannels { chnl := chnlify(ch) if err := Channels().Insert(chnl); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } } fetch.go 159
157. func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %sn", len(newitems), feed.Url) for _, item := range newitems { itm := itmify(item, ch) if err := Items().Insert(itm); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } } fetch.go 160
158. Running MongoDB 161
159. ❯ mongod MongoDB starting : pid=51054 port=27017 dbpath=/data/db Running MongoDB 162
160. ❯ mongo Mongo Shell 163
161. ❯ go run dagobah.go fetch –rsstimeout=1 15 new item(s) in http://spf13.com/index.xml 1 new channel(s) in http://spf13.com/index.xml Sleeping for 300 seconds on http://spf13.com/index.xml 25 new item(s) in http://www.goinggo.net/feeds/posts/default 1 new channel(s) in http://www.goinggo.net/feeds/posts/default Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/ default 10 new item(s) in http://dave.cheney.net/feed 1 new channel(s) in http://dave.cheney.net/feed Sleeping for 299 seconds on http://dave.cheney.net/feed Running planet 164
162. > use dagobah > db.items.findOne() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : "" Looking at the Data 165
163. _ID •MongoDB’s primary key •Default is to use an “object_id” •Object_id guaranteed to be unique across your entire cluster •MongoDB will create it for you on insert 166
164. Tweet Break Feeding mgo at #OSCON with @spf13 http://j.mp/mongomgo
165. https:// github.com/spf13/ firstGoApp- Planet/tree/ step4 168
166. Step 5 Building a web Server 169
167. Lots of choices •Beego High-performance web framework •Gin Martini like with better performance •Goji A minimalistic web framework •Gorilla A web toolkit •httprouter A high performance router •Mango modular web-app framework like Rack •Martini modular web apps & services •net/http Standard library web package •pat Sinatra style, by the author of Sinatra. •Revel A high-productivity web framework •tigertonic framework for JSON web services •traffic Sinatra inspired web framework for Go •web.go A simple framework to write webapps in Go. 170
168. Gin •Compatible with net/http •Uses Httprouter (really fast) •Familiar/Friendly Interface •Works well with JSON •Had to pick something 171
169. go get -u github.com/gin-gonic/gin Getting Gin 172
170. boilerplate stuff 173
171. // Copyright © 2014 Steve Francia. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" ) Server.Go (1) 174
172. Defining the Server command 175
173. var serverCmd = &cobra.Command{ Use: "server", Short: "Server for feeds", Long: Dagobah will serve all feeds listed in the config file., Run: serverRun, } ! func init() { serverCmd.Flags().Int("port", 1138, "Port to run Dagobah server on") viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) } server.go (2) 176
174. func addCommands() { RootCmd.AddCommand(fetchCmd) RootCmd.AddCommand(serverCmd) } planet.go 177
175. Defining our first route 178
176. func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() ! r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) ! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) } server.go (3) 179
177. ❯ go run dagobah.go server Running it 180
178. Check it out http://localhost:1138 181
179. Tweet Break I wrote a webserver in go using Gin with @spf13 at #OSCON http://j.mp/gin-go
180. https:// github.com/spf13/ firstGoApp- Planet/tree/ step5 183
181. Step 6 Building a Dynamic Server 184
182. Binary Problem? •Go ships as a single binary •Need to load files from within the binary •Don’t want to embed by hand 185
183. go.rice •Nicely allows local loads in dev, and embedded loads when executing binary •Stable, but not very mature yet •Best option I know of 186
184. go get -u github.com/GeertJohan/ go.rice Getting Go.Rice 187
185. Serving Static Files 188
186. Get your static & templates on c.spf13.com/OSCON/step6-static.zip 189
187. import ( "fmt" "html/template" "log" "net/http" ! "github.com/GeertJohan/go.rice" "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" ) server.go 190
188. func serverRun(cmd *cobra.Command, args []string) { … r.GET("/static/*filepath", staticServe) ! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) } server.go 191
189. func staticServe(c *gin.Context) { static, err := rice.FindBox("static") if err != nil { log.Fatal(err) } ! original := c.Request.URL.Path c.Request.URL.Path = c.Params.ByName("filepath") fmt.Println(c.Params.ByName("filepath")) http.FileServer(static.HTTPBox()).ServeHTTP(c.Writer, c.Request) c.Request.URL.Path = original } server.go 192
190. Loading & Serving Templates 193
191. func loadTemplates(list …string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } } return templates } server.go 194
192. func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() templates := loadTemplates("full.html") r.SetHTMLTemplate(templates) ! r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) ! r.GET("/", homeRoute) r.GET("/static/*filepath", staticServe) ! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) } server.go 195
193. func homeRoute(c *gin.Context) { obj := gin.H{"title": "Go Rules"} c.HTML(200, “full.html", obj) } server.go 196
194. ❯ go run dagobah.go server Running it 197
195. Check it out http://localhost:1138 198
196. Tweet Break No more mr rice guy with @spf13 at #OSCON http://j.mp/go-rice
197. https:// github.com/spf13/ firstGoApp- Planet/tree/ step6 200
198. Step 7 Connecting the Data to The templates 201
199. Go Templates •Really nice to work with •Very simple, yet very powerful •Safe •Go Template Primer 202
200. Understanding the data 203
201. > db.items.find().sort({ "date" : -1 }).limit(1).pretty() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : "" } Looking at the Schema 204
202. Listing POsts 205
203. passing data into the template •Go’s templates accept a variety of data types •Gin’s default is ‘H’ : map[string]interface{} •Feels natural •All (exported) methods bound to values in the map are accessible inside the template 206
204. func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", "posts": posts} c.HTML(200, "full.html", obj) } server.go 207
205. … <section id="latest-list" class="ui divided inbox selection list active tab" data-tab="recent"> {{ range $item := .posts }} {{ if $item.WorthShowing }} <a class="item" href=“#{{$item.Key | urlquery }}"> <div class="description">{{ $item.Title }}</div> <div class="right floated ui label small">{{ $item.Date.Format "Jan 2, 2006" }}</div> </a> {{ end }} {{ end }} </section> … Full.html 208
206. Adding Helpers to itm 209
207. func (i Itm) FirstLink() (link rss.Link) { if len(i.Links) == 0 || i.Links[0] == nil { return } return *i.Links[0] } ! func (i Itm) WorthShowing() bool { if len(i.FullContent) > 100 { return true } return false } fetch.go 210
208. displaying POsts 211
209. … <main> {{ with .message}}! <h1 class="ui header large"> {{.}} </h1>! {{ end }}! ! {{ range $item := .posts }}! {{ if $item.WorthShowing }}! <article data-key="{{$item.Key}}">! <header>! <a name="{{$item.Key}}">! <h1 class="ui header">{{$item.Title}}</h1>! <section class="meta-tags">! <a class="ui label large blue author" href="/channel/{{$item.ChannelKey}}">{{$item.Author.Name}}</a>! <span class="large ui label date">{{ $item.Date.Format "Jan 2, 2006" }}</span>! </section>! </header>! <section class="main-content">! {{$item.FullContent | html }}! </section>! <footer>! {{with $item.FirstLink}}! <a class="ui basic button" href="{{.Href}}">Source</a>! {{end}}! <div class="ui divider"></div>! </footer>! </article>! {{ end }}! {{ else }}! No Articles! {{ end }}! </main> … full.html 212
210. Adding Template Functions 213
211. func loadTemplates(list …string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } ! templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } } ! funcMap := template.FuncMap{ "html": ProperHtml, "title": func(a string) string { return strings.Title(a) }, } ! templates.Funcs(funcMap) ! return templates } server.go 214
212. func ProperHtml(text string) template.HTML { if strings.Contains(text, "content:encoded>") || strings.Contains(text, "content/:encoded>") { text = html.UnescapeString(text) } return template.HTML(html.UnescapeString(template.HTMLEscapeString(text))) } server.go 215
213. Looking Good http://localhost:1138 216
214. Tweet Break Conquering Go Templates with @spf13 at #OSCON
215. https:// github.com/spf13/ firstGoApp- Planet/tree/ step7 218
216. Step 8 More Routes 219
217. Adding Helpers 220
218. func (c Chnl) HomePage() string { if len(c.Links) == 0 { return "" } ! url, err := url.Parse(c.Links[0].Href) if err != nil { log.Println(err) } return url.Scheme + "://" + url.Host } fetch.go 221
219. ! func four04(c *gin.Context, message string) { c.HTML(404, "full.html", gin.H{"message": message, "title": message}) } server.go 222
220. func AllChannels() []Chnl { var channels []Chnl r := Channels().Find(bson.M{}).Sort("-lastbuilddate") r.All(&channels) return channels } MongoDB.go 223
221. Listing channels 224
222. func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", "posts": posts, "channels": AllChannels()} c.HTML(200, "full.html", obj) } server.go 225
223. … <section id="channel-list" class="ui large inverted vertical menu tab" data-tab=“channels"> <a class="item js-navigate" href="/" >! <i class="external home icon"></i>! <span class="ui name" href=""> All </span>! </a>! {{ range $channel := .channels }}! <div class="item js-navigate-channel" data-href="/channel/{{$channel.Key}}" data-key="{{$channel.Key}}" style="cursor:pointer;">! <a href="{{$channel.HomePage}}" class="ui float right"><i class="external url sign icon"></i></a>! <span class="ui name" href=""> {{$channel.Title}} </span>! <span class="ui label">{{ len $channel.Links }}</span>! </div>! {{end}}! </section> … full.html 226
224. displaying channels 227
225. func serverRun(cmd *cobra.Command, args []string) { … r.GET("/channel/*key", channelRoute) r.Run(":" + port) } server.go 228
226. func channelRoute(c *gin.Context) { key := c.Params.ByName("key") if len(key) < 2 { four04(c, "Channel Not Found") return } ! key = key[1:] ! fmt.Println(key) ! var posts []Itm results := Items().Find(bson.M{"channelkey": key}).Sort("-date").Limit(20) results.All(&posts) ! if len(posts) == 0 { four04(c, "No Articles") return } … server.go - pt 1 229
227. … var currentChannel Chnl err := Channels().Find(bson.M{"key": key}).One(¤tChannel) if err != nil { if string(err.Error()) == "not found" { four04(c, "Channel not found") return } else { fmt.Println(err) } } ! obj := gin.H{"title": currentChannel.Title, "header": currentChannel.Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) } server.go - pt 2 230
228. displaying A post 231
229. func serverRun(cmd *cobra.Command, args []string) { … r.GET("/post/*key", channelRoute) r.Run(":" + port) } server.go 232
230. func postRoute(c *gin.Context) { key := c.Params.ByName("key") ! if len(key) < 2 { four04(c, "Invalid Post") return } ! key = key[1:] ! var ps []Itm r := Items().Find(bson.M{"key": key}).Sort("-date").Limit(1) r.All(&ps) ! if len(ps) == 0 { four04(c, "Post not found") return } … server.go - Pt 1 233
231. … ! var posts []Itm results := Items().Find(bson.M{"date": bson.M{"$lte": ps[0].Date}}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": ps[0].Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) } server.go - Pt 2 234
232. Looking Good http://localhost:1138 235
233. Tweet Break Making my own planet in #go with @spf13 at #OSCON http://j.mp/go-planet
234. https:// github.com/spf13/ firstGoApp- Planet/tree/ step8 237
235. Step 9 Advanced Routes 238
236. Congrats •You’ve made it really far •Now the training wheels are off •Let’s try to add pagination & search 240
237. Hints •You will need a new index •We’re talking full text here •You will also need a new route 241
238. … r.GET("/", homeRoute) r.GET("/post/*key", postRoute) r.GET("/search/*query", searchRoute) r.GET("/static/*filepath", staticServe) r.GET("/channel/*key", channelRoute) … server.go 242
239. Step 10 ! Add Polish 243
240. What’s next ? •Add ‘truncate’ command to remove old posts and channels •Change root command to run fetcher & server •Add infinite scroll capabilities 244
241. What’s next ? •Make it so users can provide their own static files & templates •Documentation •Blog posts 245
242. In Conclusion 246 Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
243. Thank yOU 247
244. What have we done ? •Written our first lines of Go •Written our first Go package •Written our own web server •Written our first Go application •Learned a lot & had fun doing it 248
245. @spf13 •Author of Hugo, Cobra, Viper & More •Chief Developer Advocate for MongoDB •Gopher 249