Getting Started with Go

This presentation was given as a Workshop at OSCON 2014.

Description

New to Go? This tutorial will give developers an introduction and practical experience in building applications with the Go language. Gopher Steve Francia, Author of Hugo, Cobra, and many other popular Go packages breaks it down step by step as you build your own full featured Go application.

Starting with an introduction to the Go language. He then reviews the fantastic go tools available. With our environment ready we will learn by doing. The rest of the time is dedicated building a working go web and cli application. Through our application development experience we will introduce key features, libraries and best practices of using Go.

This tutorial is designed with developers in mind. Prior experience with any of the following languages: ruby, perl, java, c#, javascript, php, node.js, or python is preferred. We will be using the MongoDB database as a backend for our application.

We will be using/learning a variety of libraries including:

  • bytes and strings
  • templates
  • net/http
  • io, fmt, errors
  • cobra
  • mgo
  • Gin
  • Go.Rice
  • Cobra
  • Viper

Slide Transcript:

  1. Building your first Go App 1 Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
  2. @spf13 •Author of Hugo, Cobra, Viper & More •Chief Developer Advocate for MongoDB •Gopher 2
  3. My Go Story •I write a medium size blog •Frustrated with Wordpress performance & maintenance •Existing Static Site Generators had complicated installations & were very slow •Began writing a S.S.G. in Go called Hugo 1.5 years ago 4
  4. hugo.spf13.com
  5. Hugo •Hugo is 2nd fastest growing Static Site Generator (40-50 * a week on github) •Hugo is one of the fastest Static Site Generators. 1000x faster than Jekyll •Easiest Installation of any SSG. Download and run •2nd most contributors of any Go project (Docker 1st)6
  6. Plan •Introduce Go Language •Introduce Tools & Libraries •Build our application •Tell the world how awesome we are 8
  7. Ground Rules •Workshops are hard, everyone works at a different pace •We will move on when about 50% are ready •Slides are online, feel free to work ahead or catch up 9
  8. http:// spf13.com/ presentation/ first-go-app/ 10
  9. Installing Go •You should already have Go installed •If you don’t, do it NOW •Installation Guide 11
  10. Installing MongoDB •You should already have MongoDB installed •If you don’t, do it NOW •Installation Guide 12
  11. Git & Mercurial •Lastly we need Git & Mercurial •http://git-scm.com/downloads •http://mercurial.selenic.com/wiki/Download 13
  12. Introduction to Go 14
  13. Why Another Language? • Software is slow • Sofware is hard to write • Software doesn’t scale well 15
  14. Go is Fast • Go execution speed is close to C • Go compile time rivals dynamic interpretation 16
  15. Go is Friendly • Feels like a dynamic language in many ways • Very small core language, easy to remember all of it • Single binary installation, no dependencies • Extensive Tooling & StdLib 17
  16. Go is Concurrent • Concurrency is part of the language • Any function can become a goroutine • Goroutines run concurrently, communicate through channels • Select waits for communication on any of a set of channels 18
  17. Go’s Inspiration •C: statement and expression syntax •Pascal: declaration syntax •Modula 2, Oberon 2: packages •CSP, Occam, Newsqueak, Limbo, Alef: concurrency •BCPL: the semicolon rule •Smalltalk: methods •Newsqueak: <-, := •APL: iota 19 … AND lessons good and bad from all those plus: C++, C#, Java, JavaScript, LISP, Python, Scala, …
  18. — txxxxd “Most of the appeal for me is not the features that Go has, but rather the features that have been intentionally left out.” 20
  19. — Rob Pike “Why would you have a language that is not theoretically exciting? Because it’s very useful.” 21
  20. package main ! import "fmt" ! func main() { fmt.Println(“Hello, 世界") } Hello World 22
  21. Package •Not classes or objects •No subpackages •Fundamental building block in Go •Visibility is package-level, not type-level •Programs run “main” package 23 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  22. import •Provides access to a package •Package consists of types, functions, etc… •No circular dependencies 24 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  23. “fmt" •Package path is just a string •Standard library sits at the root (no path) •Core language is very small •Most functionality is in Stdlibs 25 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  24. func •Declaring a function •Same syntax for methods, anonymous functions & closures •main.main() is the function run when the program is executed 26 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  25. Main •No void •No return value •No function args (command line is in os package) 27 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  26. { } •Braces not spaces •Feels like C ( & most languages ) •Newline must be after brace (semicolons are inserted during compiliation) 28 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  27. fmt •Accessing the package imported above •Everything visible in the package will be accessible through the name (or alias if provided) 29 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  28. Println • Println, not println • Println, not print_ln •Capital for export •Variadic function using reflection 30 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  29. “Hello, 世界” •UTF-8 input source •Strings are immutable •Strings are UTF-8 encoded •String is a built-in type 31 package main ! import "fmt" ! func main() { fmt.Println("Hello, 世界") }
  30. Rob Pike did it better http://confreaks.com/videos/3419- gophercon2014-opening-day-keynote 32
  31. Tons more… •types •constants •methods •interfaces •pointers •control flow •conditionals •tools •packages •concurrency 33
  32. Effective Go 34
  33. Lets write some code ! Introduce the go tools 35
  34. package main ! import "fmt" ! func main() { fmt.Println("Hello,World") } Hello World 36 hello.go
  35. ❯ go run hello.go Hello,World Go Run 37
  36. ❯ go build hello.go ! ❯ls -lh 1.7M Jun 17 22:44 hello 72B Jun 17 22:43 hello.go ! ❯ ./hello Hello, World Go Build 38
  37. Go Fmt •Automatically formats Go source code •Ends stylistic debates •Integration with editors (vim, emacs, others) •Can also refactor code (see http://spf13.com/post/go-fmt ) 39
  38. go Test •Enables easy testing of application •Integrates with the testing package •Supports benchmark, functional & unit style testing •Combine with ‘looper’ to have realtime feedback 40
  39. package main ! import "testing" ! func TestOne(t *testing.T) { one := false if !one { t.Errorf(“Test Failed”) } } Hello Test 41 hello_test.go
  40. ❯ go test ./… — FAIL: TestOne (0.00 seconds) hello_test.go:8: Test failed FAIL FAIL_/Code/firstApp0.012s ! ! ❯ go test ./… ok _/Code/firstApp0.012s Go Test 42
  41. Go Planet 43The planet logo is based on the Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
  42. Why Planet? •CLI application •Web application •Good introduction •Right sized •Database (MongoDB) •Concurrency 45
  43. Steps 0. Env Setup 1. Commands 2. Configuration 3. Working with Feeds 4. DB as Storage 5-6. Web Server 7-8. DB -> Templates 9. & 10. Add Polish 46
  44. Step 0 Setting up our environment 47
  45. Go PAth ! $GOPATH 48
  46. Go path •The GOPATH environment variable specifies the location of your workspace (and dependencies) •It must not be the same path as your Go installation 49
  47. ❯ mkdir $HOME/go ! ❯ export GOPATH=$HOME/go Setting up 50
  48. export GOPATH=$USER/go export GOROOT=go env GOROOT PATH=$PATH:$GOPATH/bin Linux & Mac 51 Add this to your ~/.bashrc (or equivalent)
  49. Windows •Control panel > System 52
  50. Go PAth •/home/user/gocode/ ($GOPATH) • src/ (put your source code here) • bin/ (binaries installed here) • pkg/ (installed package objects) 53
  51. Creating our project 54
  52. Project Dir •Make a directory inside of $GOPATH/src •I like mine to be in $GOPATH/src/github.com/spf13/PROJECTNAME •It can not be a symlink •We will call this our working directory or $wd 55
  53. package main ! import "fmt" ! func main() { fmt.Println(“My Project") } main.go 56
  54. ❯ go run main.go My Project Run it 57
  55. 58 I’m on the #gopath with @spf13 at #OSCON http://j.mp/onGoPath Tweet Break
  56. Step 1 Creating the Command(s) 59
  57. Defining our first command 60
  58. Structs •Short for structure •Objectish … blend of data & methods, but no inheritance •Really cheap. Struct{} is free (0 bytes). 61
  59. Functions •Can have multiple return values •No overloading •No optional parameters (but variadic) 62
  60. First class Functions •Function literals are anonymous functions in Go •Can assign to a variable or field •Can use as function parameters or return values •Can be created as a goroutine 63
  61. pointers •Function calls copy arguments into the function •Pointers reference a location in memory where a value is stored rather than the value itself •In Go a pointer is represented using the * •The & operator gives us the address of a variable •Go automatically dereferences pointers when using “.” 64
  62. cobra •A CLI Commander •Easy management of commands & flags •Used by bitbucket, openshift & lots more 65 http://fav.me/d5gkby7
  63. go get -u github.com/spf13/cobra Getting Cobra 66
  64. ❯ mkdir $wd/commands Commands Dir 67
  65. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" ! "github.com/spf13/cobra" ) planet.go (1) 68
  66. Path = Url go get uses it to install 69
  67. var RootCmd = &cobra.Command{ Use: "dagobah", Short: Dagobah is an awesome planet style RSS aggregator, Long: Dagobah provides planet style RSS aggregation. It is inspired by python planet. It has a simple YAML configuration and provides it&#39;s own webserver., Run: func(cmd *cobra.Command, args []string) { fmt.Println("Dagobah runs") }, } planet.go (2) 70
  68. executing cobra’s command 71
  69. Variable Assignment •‘:=‘ is the short variable declaration operator Declares, creates & assigns simultaneously while infering type •Assignment via ‘=‘ requires prior declaration •var foo int = 10 is the same as foo := 10 72
  70. Error Handling •Errors are not exceptional. Should be part of the language •Typically one of the return values •No exceptions in Go fd, err := os.Open("test.go") if err != nil { log.Fatal(err) } 73
  71. ( Don’t) Panic •Should rarely be used if ever •Performs a full stack trace dump for end users •Use only in the most dire circumstances (like package can’t find FileSystem) •Can use ‘recover’, but only inside deferred functions 74
  72. func Execute() { err := RootCmd.Execute() if err != nil { fmt.Println(err) os.Exit(-1) } } planet.go (3) 75
  73. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package main ! import "github.com/spf13/firstGoApp-Planet/commands" ! func main() { commands.Execute() } main.go 76 Make sure to use your path
  74. ❯ go run main.go Dagobah runs ! ❯ go run main.go help Running it 77
  75. Getting Help •https://godoc.org/github.com/spf13/cobra •https://github.com/spf13/firstGoApp-Planet/ tree/step1 78
  76. Creating #CobraCommands with @spf13 at #OSCON http://j.mp/cobracmd Tweet Break
  77. Step 2 Easy Configuration 80
  78. Handing Config files 81
  79. Package Identifiers •Export with Capital name •No automatic getters & setters •var field … func Field() … func SetField() are appropriate names 82
  80. init() •Each source file can have one (or multiple) •Niladic function to setup state (function without return value) •Called after all variable declarations in the package 83
  81. Scoping •Very granular scoping. •Variables declared inside { } only survive in that cotrol structure (for, if, etc) •The := trap if := is used to declare x and x already exists in an outer scope a new variable x will be created (shadowing) 84
  82. package main ! import ( "errors" "fmt" ) ! var Foo int ! func Zero() (int, error) { return 0, errors.New("yo") } ! ! func main() { ! Foo = 10 ! fmt.Println("outer:", Foo) // 10 ! if Foo, err := Zero(); err != nil { fmt.Println("inner:", Foo) // 0 } ! fmt.Println("outer again:", Foo) // still 10 } := Trap 85http://play.golang.org/p/i4L9Ao1P65
  83. Viper •A configuration manager •Easy loading of config files •Registry for application settings •Designed to work well with (or without) Cobra 86
  84. go get -u github.com/spf13/viper Getting Viper 87
  85. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" ! "github.com/spf13/cobra" "github.com/spf13/viper" ) planet.go (1) 88
  86. var CfgFile string ! func init() { cobra.OnInitialize(initConfig) RootCmd.PersistentFlags().StringVar(&CfgFile, "config", "", "config file (default is $HOME/dagobah/config.yaml)") } ! func initConfig() { if CfgFile != "" { viper.SetConfigFile(CfgFile) } viper.SetConfigName("config") viper.AddConfigPath("/etc/dagobah/") viper.AddConfigPath("$HOME/.dagobah/") viper.ReadInConfig() } planet.go (2) 89
  87. Creating our config file 90
  88. appname: "Dagobah" feeds: - "http://spf13.com/index.xml" - "http://dave.cheney.net/feed" - "http://www.goinggo.net/feeds/posts/default" - "http://blog.labix.org/feed" - "http://blog.golang.org/feed.atom" ! $HOME/.dagobah/config.yaml 91
  89. trying it out 92
  90. var RootCmd = &cobra.Command{ Use: "…", Short: ..., Long: ..., Run: rootRun, } ! func rootRun(cmd *cobra.Command, args []string) { fmt.Println(viper.Get("feeds")) fmt.Println(viper.GetString("appname")) } planet.go (3) 93
  91. ❯ go run main.go [http://spf13.com/index.xml http:// dave.cheney.net/feed http:// www.goinggo.net/feeds/posts/default http://blog.labix.org/feed http:// blog.golang.org/feed.atom] Dagobah Running it 94
  92. I got bit by the #goviper at #OSCON http://j.mp/go-viper Tweet Break
  93. https:// github.com/spf13/ firstGoApp- Planet/tree/ step2 96
  94. Step 3 Working with Feeds 97
  95. Finding Libraries 98
  96. Go-Search.org 99
  97. Go wiki 100
  98. go-pkg-rss •Fetch Rss and Atom feeds from the internet •Supports RSS .92, 1.0, 2.0 & Atom 1.0 •Respects TTL, SkipDays, CacheTimeout, etc in the feeds 101
  99. go get -u github.com/jteeuwen/go- pkg-rss Getting Go-PKG-RSS 102
  100. boilerplate stuff 103
  101. import ( "time" … ) ! func addCommands() { RootCmd.AddCommand(fetchCmd) } ! func Execute() { addCommands() … } planet.go 104
  102. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" "time" ! rss "github.com/jteeuwen/go-pkg-rss" "github.com/spf13/cobra" "github.com/spf13/viper" ) Fetch.Go (1) 105
  103. Creating our own type 106
  104. Named Types •Can be any known type (struct, string, int, slice, a new type you’ve declared, etc) •Can have methods declared on it •Not an alias. Explicit type. 107
  105. Slices & arrays •Array is fixed in length •Slice can be viewed as a dynamic array (can grow) •Slice is a view of an array •Slice is the core data structure in Go for lists of data 108
  106. type Config struct { Feeds []string Port int } Fetch.Go (2) 109
  107. Defining the fetch command 110
  108. New( ) •new(T) Does 2 things at once: •Allocates new zeroed value of type T •Returns a pointer to the new value ( *T) •Used for all non-reference types 111
  109. Make( ) •Creates slices, maps, and channels only •These types are special “reference” types •Makes a initialized (not zeroed) value of type T •Does not return a pointer 112
  110. Composite Literals •An expression that creates a new value each time it is evaluated •We’ve already used these quite a bit •Can also be create values of arrays, slices, and maps •Pseudo Constructor •eg File{fd: fd, name: name} 113
  111. Constructors •Not built into Go •Use Composite Literals when possible •If initialization is needed use a factory •Convention is to use New___() •… or New() when only one type is exported in the package 114
  112. var fetchCmd = &cobra.Command{ Use: "fetch", Short: "Fetch feeds", Long: Dagobah will fetch all feeds listed in the config file., Run: fetchRun } ! func init() { fetchCmd.Flags().Int("rsstimeout", 5, "Timeout (in min) for RSS retrival") viper.BindPFlag("rsstimeout", fetchCmd.Flags().Lookup("rsstimeout")) } Fetch.go (3) 115
  113. Fetching in a goroutine 116
  114. GoRoutines •A function executing concurrently with other goroutines in the same address space •Lightweight •Not threads, coroutines, or processes 117
  115. Channels •Channels are lightweight •Synchronize goroutines by communication rather than locking shared memory •Channel <- Sender —- Writes block when channel is full •<- Channel —- Reads blocks waiting for something to be read •Great talk on channels given at GopherCon http://confreaks.com/videos/3422-gophercon2014-a-channel-compendium 118
  116. Marshaling into •We can’t return a T (no generics) •We can reflect on any value of any type •Marshal accepts the address of the value so it can manipulate the value directly 119
  117. func fetchRun(cmd *cobra.Command, args []string) { Fetcher() ! sigChan := make(chan os.Signal, 1) signal.Notify(sigChan, os.Interrupt) <-sigChan } Fetch.go (4) 120
  118. func Fetcher() { var config Config if err := viper.Marshal(&config); err != nil { fmt.Println(err) } ! for _, feed := range config.Feeds { go PollFeed(feed.Url) } } Fetch.go (5) 121
  119. Building our fetcher 122
  120. Learning about go-pkg-rss •How do we learn how to use the package? 123
  121. Learning about go-pkg-rss •github.com/jteeuwen/go-pkg-rss •Readme is pretty lacking •godoc.org/github.com/jteeuwen/go-pkg-rss 124
  122. godoc.org/github.com/jteeuwen/go-pkg-rss 125
  123. Go Search 126
  124. For •For, Foreach & While are all spelled ‘for’ in Go for init; condition; post { } // Like a C for for condition { } // Like a C while for { } // Like a C for(;;) 127
  125. func PollFeed(uri string) { timeout := viper.GetInt("RSSTimeout") if timeout < 1 { timeout = 1 } feed := rss.New(timeout, true, chanHandler, itemHandler) ! for { if err := feed.Fetch(uri, nil); err != nil { fmt.Fprintf(os.Stderr, "[e] %s: %s", uri, err) return } ! fmt.Printf("Sleeping for %d seconds on %sn", feed.SecondsTillUpdate(), uri) time.Sleep(time.Duration(feed.SecondsTillUpdate() * 1e9)) } } Fetch.GO (6) 128
  126. func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %sn", len(newchannels), feed.Url) } ! func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %sn", len(newitems), feed.Url) } Fetch.go (7) 129
  127. ❯ go run dagobah.go fetch –rsstimeout=1 15 new item(s) in http://spf13.com/index.xml 1 new channel(s) in http://spf13.com/index.xml Sleeping for 300 seconds on http://spf13.com/index.xml 25 new item(s) in http://www.goinggo.net/feeds/posts/default 1 new channel(s) in http://www.goinggo.net/feeds/posts/default Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/ default 10 new item(s) in http://dave.cheney.net/feed 1 new channel(s) in http://dave.cheney.net/feed Sleeping for 299 seconds on http://dave.cheney.net/feed Running it 130
  128. Tweet Break I wrote my first goroutine at #OSCON with @spf13
  129. https:// github.com/spf13/ firstGoApp- Planet/tree/ step3 132
  130. Step 4 Storing in the Database 133
  131. Pretty •github.com/kr/pretty •pretty.Println(X,Y,Z) 134
  132. MongoDB •Open Source •Document Database •Document == Struct or Document == Map •Works seamlessly with Go (and other languages) 135
  133. MongoDB •Fast & Scalable •Used by Disney, IBM, Metlife, eBay, Forbes, Craigslist, Cisco, Stripe 136
  134. Mgo •Developed by the open source community •Heavily used throughout Go community •"mgo is the best database driver I've ever used." — Patrick Crosby, Founder of StatHat •"mgo and Go are a pair made in heaven. — Brian Ketelsen, Author of GopherTimes.com137
  135. go get -u gopkg.in/mgo.v2 Getting mgo 138
  136. boilerplate stuff 139
  137. // Copyright © 2014 Steve Francia <spf@spf13.com>. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "fmt" "os" ! "github.com/spf13/viper" "gopkg.in/mgo.v2" ) ! var mongodbSession *mgo.Session ! func init() { RootCmd.PersistentFlags().String("mongodb_uri", "localhost", "host where mongoDB is") viper.BindPFlag("mongodb_uri", RootCmd.PersistentFlags().Lookup(“mongodb_uri")) CreateUniqueIndexes() } MongoDB.go (1) 140
  138. Setting up a DB session 141
  139. func DBSession() *mgo.Session { if mongodbSession == nil { uri := os.Getenv("MONGODB_URI") if uri == "" { uri = viper.GetString("mongodb_uri") ! if uri == "" { log.Fatalln("No connection uri for MongoDB provided") } } ! var err error mongodbSession, err = mgo.Dial(uri) if mongodbSession == nil || err != nil { log.Fatalf("Can't connect to mongo, go error %vn", err) } ! mongodbSession.SetSafe(&mgo.Safe{}) } return mongodbSession } MongoDB.go (2) 142
  140. setting up some shortcuts 143
  141. ! func DB() *mgo.Database { return DBSession().DB(viper.GetString("dbname")) } ! func Items() *mgo.Collection { return DB().C("items") } ! func Channels() *mgo.Collection { return DB().C("channels") } MongoDB.go (3) 144
  142. Setting up indexes 145
  143. func CreateUniqueIndexes() { idx := mgo.Index{ Key: []string{"key"}, Unique: true, DropDups: true, Background: true, Sparse: true, } ! if err := Items().EnsureIndex(idx); err != nil { fmt.Println(err) } ! if err := Channels().EnsureIndex(idx); err != nil { fmt.Println(err) } } MongoDB.go (4) 146
  144. Preparing the data 147
  145. Data Prep •Feeds are dirty •Go doesn’t allow “patching” of structs (not how structs work) •MongoDB works with native types & named types 148
  146. Tags (Annotations) •String literal following a field declaration •Provides additional information during reflection •Used by JSON, BSON, YAML, etc 149
  147. for x,y := range •Provides a way to iterate over an array, slice, string, map, or channel. •Like Foreach or Each in other languages •x is key, y is value… can ignore by setting to _ 150
  148. Append •Append is really awesome, feels like push, but does a lot more. •Append will grow a slice (both length & capacity) •… and copy the underlying array if needed 151
  149. type Itm struct { Date time.Time Key string ChannelKey string Title string FullContent string Links []*rss.Link Description string Author rss.Author Categories []*rss.Category Comments string Enclosures []*rss.Enclosure Guid *string bson:&quot;,omitempty&quot; Source *rss.Source PubDate string Id string bson:&quot;,omitempty&quot; Generator *rss.Generator Contributors []string Content *rss.Content Extensions map[string]map[string][]rss.Extension } fetch.go 152
  150. func itmify(o *rss.Item, ch *rss.Channel) Itm { var x Itm x.Title = o.Title x.Links = o.Links x.ChannelKey = ch.Key() x.Description = o.Description x.Author = o.Author x.Categories = o.Categories x.Comments = o.Comments x.Enclosures = o.Enclosures x.Guid = o.Guid x.PubDate = o.PubDate x.Id = o.Id x.Key = o.Key() x.Generator = o.Generator x.Contributors = o.Contributors x.Content = o.Content x.Extensions = o.Extensions x.Date, _ = o.ParsedPubDate() ! if o.Content != nil && o.Content.Text != "" { x.FullContent = o.Content.Text } else { x.FullContent = o.Description } ! return x } fetch.go 153
  151. type Chnl struct { Key string Title string Links []rss.Link Description string Language string Copyright string ManagingEditor string WebMaster string PubDate string LastBuildDate string Docs string Categories []*rss.Category Generator rss.Generator TTL int Rating string SkipHours []int SkipDays []int Image rss.Image ItemKeys []string Cloud rss.Cloud TextInput rss.Input Extensions map[string]map[string] []rss.Extension Id string Rights string Author rss.Author SubTitle rss.SubTitle } fetch.go 154
  152. func chnlify(o *rss.Channel) Chnl { var x Chnl x.Key = o.Key() x.Title = o.Title x.Links = o.Links x.Description = o.Description x.Language = o.Language x.Copyright = o.Copyright x.ManagingEditor = o.ManagingEditor x.WebMaster = o.WebMaster x.PubDate = o.PubDate x.LastBuildDate = o.LastBuildDate x.Docs = o.Docs x.Categories = o.Categories x.Generator = o.Generator x.TTL = o.TTL x.Rating = o.Rating x.SkipHours = o.SkipHours x.SkipDays = o.SkipDays x.Image = o.Image x.Cloud = o.Cloud x.TextInput = o.TextInput x.Extensions = o.Extensions x.Id = o.Id x.Rights = o.Rights x.Author = o.Author x.SubTitle = o.SubTitle ! … ! return x } fetch.go 155
  153. … ! ! var keys []string for _, y := range o.Items { keys = append(keys, y.Key()) } x.ItemKeys = keys ! … fetch.go 156
  154. Storing Feeds & Items 157
  155. The Blank Identifier ‘_’ •Used to ignore return values from a function •Used with imports as an alias • Allows import of unused packages so init functions can be executed 158
  156. func chanHandler(feed *rss.Feed, newchannels []*rss.Channel) { fmt.Printf("%d new channel(s) in %sn", len(newchannels), feed.Url) for _, ch := range newchannels { chnl := chnlify(ch) if err := Channels().Insert(chnl); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } } fetch.go 159
  157. func itemHandler(feed *rss.Feed, ch *rss.Channel, newitems []*rss.Item) { fmt.Printf("%d new item(s) in %sn", len(newitems), feed.Url) for _, item := range newitems { itm := itmify(item, ch) if err := Items().Insert(itm); err != nil { if !strings.Contains(err.Error(), "E11000") { fmt.Printf("Database error. Err: %v", err) } } } } fetch.go 160
  158. Running MongoDB 161
  159. ❯ mongod MongoDB starting : pid=51054 port=27017 dbpath=/data/db Running MongoDB 162
  160. ❯ mongo Mongo Shell 163
  161. ❯ go run dagobah.go fetch –rsstimeout=1 15 new item(s) in http://spf13.com/index.xml 1 new channel(s) in http://spf13.com/index.xml Sleeping for 300 seconds on http://spf13.com/index.xml 25 new item(s) in http://www.goinggo.net/feeds/posts/default 1 new channel(s) in http://www.goinggo.net/feeds/posts/default Sleeping for 300 seconds on http://www.goinggo.net/feeds/posts/ default 10 new item(s) in http://dave.cheney.net/feed 1 new channel(s) in http://dave.cheney.net/feed Sleeping for 299 seconds on http://dave.cheney.net/feed Running planet 164
  162. > use dagobah > db.items.findOne() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : "" Looking at the Data 165
  163. _ID •MongoDB’s primary key •Default is to use an “object_id” •Object_id guaranteed to be unique across your entire cluster •MongoDB will create it for you on insert 166
  164. Tweet Break Feeding mgo at #OSCON with @spf13 http://j.mp/mongomgo
  165. https:// github.com/spf13/ firstGoApp- Planet/tree/ step4 168
  166. Step 5 Building a web Server 169
  167. Lots of choices •Beego High-performance web framework •Gin Martini like with better performance •Goji A minimalistic web framework •Gorilla A web toolkit •httprouter A high performance router •Mango modular web-app framework like Rack •Martini modular web apps & services •net/http Standard library web package •pat Sinatra style, by the author of Sinatra. •Revel A high-productivity web framework •tigertonic framework for JSON web services •traffic Sinatra inspired web framework for Go •web.go A simple framework to write webapps in Go. 170
  168. Gin •Compatible with net/http •Uses Httprouter (really fast) •Familiar/Friendly Interface •Works well with JSON •Had to pick something 171
  169. go get -u github.com/gin-gonic/gin Getting Gin 172
  170. boilerplate stuff 173
  171. // Copyright © 2014 Steve Francia. // // Use of this source code is governed by an Apache2 // license that can be found in the LICENSE file. ! package commands ! import ( "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" ) Server.Go (1) 174
  172. Defining the Server command 175
  173. var serverCmd = &cobra.Command{ Use: "server", Short: "Server for feeds", Long: Dagobah will serve all feeds listed in the config file., Run: serverRun, } ! func init() { serverCmd.Flags().Int("port", 1138, "Port to run Dagobah server on") viper.BindPFlag("port", serverCmd.Flags().Lookup("port")) } server.go (2) 176
  174. func addCommands() { RootCmd.AddCommand(fetchCmd) RootCmd.AddCommand(serverCmd) } planet.go 177
  175. Defining our first route 178
  176. func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() ! r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) ! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) } server.go (3) 179
  177. ❯ go run dagobah.go server Running it 180
  178. Check it out http://localhost:1138 181
  179. Tweet Break I wrote a webserver in go using Gin with @spf13 at #OSCON http://j.mp/gin-go
  180. https:// github.com/spf13/ firstGoApp- Planet/tree/ step5 183
  181. Step 6 Building a Dynamic Server 184
  182. Binary Problem? •Go ships as a single binary •Need to load files from within the binary •Don’t want to embed by hand 185
  183. go.rice •Nicely allows local loads in dev, and embedded loads when executing binary •Stable, but not very mature yet •Best option I know of 186
  184. go get -u github.com/GeertJohan/ go.rice Getting Go.Rice 187
  185. Serving Static Files 188
  186. Get your static & templates on c.spf13.com/OSCON/step6-static.zip 189
  187. import ( "fmt" "html/template" "log" "net/http" ! "github.com/GeertJohan/go.rice" "github.com/gin-gonic/gin" "github.com/spf13/cobra" "github.com/spf13/viper" ) server.go 190
  188. func serverRun(cmd *cobra.Command, args []string) { … r.GET("/static/*filepath", staticServe) ! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) } server.go 191
  189. func staticServe(c *gin.Context) { static, err := rice.FindBox("static") if err != nil { log.Fatal(err) } ! original := c.Request.URL.Path c.Request.URL.Path = c.Params.ByName("filepath") fmt.Println(c.Params.ByName("filepath")) http.FileServer(static.HTTPBox()).ServeHTTP(c.Writer, c.Request) c.Request.URL.Path = original } server.go 192
  190. Loading & Serving Templates 193
  191. func loadTemplates(list …string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } } return templates } server.go 194
  192. func serverRun(cmd *cobra.Command, args []string) { r := gin.Default() templates := loadTemplates("full.html") r.SetHTMLTemplate(templates) ! r.GET("/ping", func(c *gin.Context) { c.String(200, "pong") }) ! r.GET("/", homeRoute) r.GET("/static/*filepath", staticServe) ! port := viper.GetString("port") fmt.Println("Running on port:", port) r.Run(":" + port) } server.go 195
  193. func homeRoute(c *gin.Context) { obj := gin.H{"title": "Go Rules"} c.HTML(200, “full.html", obj) } server.go 196
  194. ❯ go run dagobah.go server Running it 197
  195. Check it out http://localhost:1138 198
  196. Tweet Break No more mr rice guy with @spf13 at #OSCON http://j.mp/go-rice
  197. https:// github.com/spf13/ firstGoApp- Planet/tree/ step6 200
  198. Step 7 Connecting the Data to The templates 201
  199. Go Templates •Really nice to work with •Very simple, yet very powerful •Safe •Go Template Primer 202
  200. Understanding the data 203
  201. > db.items.find().sort({ "date" : -1 }).limit(1).pretty() { "_id" : ObjectId("53b4c08bddbc460a933cf3ed"), "date" : ISODate("2014-07-01T00:00:00Z"), "key" : "http://spf13.com/post/go-pointers-vs-references", "channelkey" : "Recent Content on Hacking Management", "title" : "Pointers vs References", "links" : [ { "href" : "http://spf13.com/post/go-pointers-vs-references", "rel" : "", "type" : "", "hreflang" : "" } Looking at the Schema 204
  202. Listing POsts 205
  203. passing data into the template •Go’s templates accept a variety of data types •Gin’s default is ‘H’ : map[string]interface{} •Feels natural •All (exported) methods bound to values in the map are accessible inside the template 206
  204. func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", "posts": posts} c.HTML(200, "full.html", obj) } server.go 207
  205. … <section id="latest-list" class="ui divided inbox selection list active tab" data-tab="recent"> {{ range $item := .posts }} {{ if $item.WorthShowing }} <a class="item" href=“#{{$item.Key | urlquery }}"> <div class="description">{{ $item.Title }}</div> <div class="right floated ui label small">{{ $item.Date.Format "Jan 2, 2006" }}</div> </a> {{ end }} {{ end }} </section> … Full.html 208
  206. Adding Helpers to itm 209
  207. func (i Itm) FirstLink() (link rss.Link) { if len(i.Links) == 0 || i.Links[0] == nil { return } return *i.Links[0] } ! func (i Itm) WorthShowing() bool { if len(i.FullContent) > 100 { return true } return false } fetch.go 210
  208. displaying POsts 211
  209. … <main> {{ with .message}}! <h1 class="ui header large"> {{.}} </h1>! {{ end }}! ! {{ range $item := .posts }}! {{ if $item.WorthShowing }}! <article data-key="{{$item.Key}}">! <header>! <a name="{{$item.Key}}">! <h1 class="ui header">{{$item.Title}}</h1>! <section class="meta-tags">! <a class="ui label large blue author" href="/channel/{{$item.ChannelKey}}">{{$item.Author.Name}}</a>! <span class="large ui label date">{{ $item.Date.Format "Jan 2, 2006" }}</span>! </section>! </header>! <section class="main-content">! {{$item.FullContent | html }}! </section>! <footer>! {{with $item.FirstLink}}! <a class="ui basic button" href="{{.Href}}">Source</a>! {{end}}! <div class="ui divider"></div>! </footer>! </article>! {{ end }}! {{ else }}! No Articles! {{ end }}! </main> … full.html 212
  210. Adding Template Functions 213
  211. func loadTemplates(list …string) *template.Template { templateBox, err := rice.FindBox("templates") if err != nil { log.Fatal(err) } ! templates := template.New("") ! for _, x := range list { templateString, err := templateBox.String(x) if err != nil { log.Fatal(err) } ! // get file contents as string _, err = templates.New(x).Parse(templateString) if err != nil { log.Fatal(err) } } ! funcMap := template.FuncMap{ "html": ProperHtml, "title": func(a string) string { return strings.Title(a) }, } ! templates.Funcs(funcMap) ! return templates } server.go 214
  212. func ProperHtml(text string) template.HTML { if strings.Contains(text, "content:encoded>") || strings.Contains(text, "content/:encoded>") { text = html.UnescapeString(text) } return template.HTML(html.UnescapeString(template.HTMLEscapeString(text))) } server.go 215
  213. Looking Good http://localhost:1138 216
  214. Tweet Break Conquering Go Templates with @spf13 at #OSCON
  215. https:// github.com/spf13/ firstGoApp- Planet/tree/ step7 218
  216. Step 8 More Routes 219
  217. Adding Helpers 220
  218. func (c Chnl) HomePage() string { if len(c.Links) == 0 { return "" } ! url, err := url.Parse(c.Links[0].Href) if err != nil { log.Println(err) } return url.Scheme + "://" + url.Host } fetch.go 221
  219. ! func four04(c *gin.Context, message string) { c.HTML(404, "full.html", gin.H{"message": message, "title": message}) } server.go 222
  220. func AllChannels() []Chnl { var channels []Chnl r := Channels().Find(bson.M{}).Sort("-lastbuilddate") r.All(&channels) return channels } MongoDB.go 223
  221. Listing channels 224
  222. func homeRoute(c *gin.Context) { var posts []Itm results := Items().Find(bson.M{}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": "Go Rules", "posts": posts, "channels": AllChannels()} c.HTML(200, "full.html", obj) } server.go 225
  223. … <section id="channel-list" class="ui large inverted vertical menu tab" data-tab=“channels"> <a class="item js-navigate" href="/" >! <i class="external home icon"></i>! <span class="ui name" href=""> All </span>! </a>! {{ range $channel := .channels }}! <div class="item js-navigate-channel" data-href="/channel/{{$channel.Key}}" data-key="{{$channel.Key}}" style="cursor:pointer;">! <a href="{{$channel.HomePage}}" class="ui float right"><i class="external url sign icon"></i></a>! <span class="ui name" href=""> {{$channel.Title}} </span>! <span class="ui label">{{ len $channel.Links }}</span>! </div>! {{end}}! </section> … full.html 226
  224. displaying channels 227
  225. func serverRun(cmd *cobra.Command, args []string) { … r.GET("/channel/*key", channelRoute) r.Run(":" + port) } server.go 228
  226. func channelRoute(c *gin.Context) { key := c.Params.ByName("key") if len(key) < 2 { four04(c, "Channel Not Found") return } ! key = key[1:] ! fmt.Println(key) ! var posts []Itm results := Items().Find(bson.M{"channelkey": key}).Sort("-date").Limit(20) results.All(&posts) ! if len(posts) == 0 { four04(c, "No Articles") return } … server.go - pt 1 229
  227. … var currentChannel Chnl err := Channels().Find(bson.M{"key": key}).One(¤tChannel) if err != nil { if string(err.Error()) == "not found" { four04(c, "Channel not found") return } else { fmt.Println(err) } } ! obj := gin.H{"title": currentChannel.Title, "header": currentChannel.Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) } server.go - pt 2 230
  228. displaying A post 231
  229. func serverRun(cmd *cobra.Command, args []string) { … r.GET("/post/*key", channelRoute) r.Run(":" + port) } server.go 232
  230. func postRoute(c *gin.Context) { key := c.Params.ByName("key") ! if len(key) < 2 { four04(c, "Invalid Post") return } ! key = key[1:] ! var ps []Itm r := Items().Find(bson.M{"key": key}).Sort("-date").Limit(1) r.All(&ps) ! if len(ps) == 0 { four04(c, "Post not found") return } … server.go - Pt 1 233
  231. … ! var posts []Itm results := Items().Find(bson.M{"date": bson.M{"$lte": ps[0].Date}}).Sort("-date").Limit(20) results.All(&posts) ! obj := gin.H{"title": ps[0].Title, "posts": posts, "channels": AllChannels()} ! c.HTML(200, "full.html", obj) } server.go - Pt 2 234
  232. Looking Good http://localhost:1138 235
  233. Tweet Break Making my own planet in #go with @spf13 at #OSCON http://j.mp/go-planet
  234. https:// github.com/spf13/ firstGoApp- Planet/tree/ step8 237
  235. Step 9 Advanced Routes 238
  236. Congrats •You’ve made it really far •Now the training wheels are off •Let’s try to add pagination & search 240
  237. Hints •You will need a new index •We’re talking full text here •You will also need a new route 241
  238. … r.GET("/", homeRoute) r.GET("/post/*key", postRoute) r.GET("/search/*query", searchRoute) r.GET("/static/*filepath", staticServe) r.GET("/channel/*key", channelRoute) … server.go 242
  239. Step 10 ! Add Polish 243
  240. What’s next ? •Add ‘truncate’ command to remove old posts and channels •Change root command to run fetcher & server •Add infinite scroll capabilities 244
  241. What’s next ? •Make it so users can provide their own static files & templates •Documentation •Blog posts 245
  242. In Conclusion 246 Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.
  243. Thank yOU 247
  244. What have we done ? •Written our first lines of Go •Written our first Go package •Written our own web server •Written our first Go application •Learned a lot & had fun doing it 248
  245. @spf13 •Author of Hugo, Cobra, Viper & More •Chief Developer Advocate for MongoDB •Gopher 249