Master Go (Exersizes) - part 2

NEW: Time To Practice: channels

Time To Practice: Channels

Task 1: What do these two main() functions print?

Here are two quite similar main() functions. Can you predict the output of each of them?

package main

import "fmt"

func main() {
    c := make(chan int)
    c <- 1024
    fmt.Println(<- c)
}
package main

import "fmt"

func main() {
    c := make(chan int, 1)
    c <- 1024
    fmt.Println(<- c)
}

(There is no solution file for this task. Simply run the two code snippets and compare the result. If you want, you can use the Go playground for this test.)

Task 2: Waiting for a goroutine

Remember channel axiom #4:

A receive from a closed channel returns the zero value immediately

In other words, once the sender closes the channel, the channel starts delivering zero values to the reader.

Can you think of a way to use this behavior for making the main goroutine wait for a goroutine to finish work?

Let’s start from this code template:

package main

import (
    "fmt"
    "time"
)

func worker(/* TODO: Maybe receive a channel here? */) {
    // Do something for some time
    for i := 0; i < 1000; i++ {
        fmt.Print(".")
    }
    fmt.Println()
    // TODO: How to tell main() that work is done?
}

func main() {

    // start the goroutine.
    go worker(/* TODO: Maybe pass a channel here? */)

    fmt.Println("Waiting for the goroutine")
    // TODO: Add code to wait for the goroutine.
    // time.Sleep() doesn't count.
    fmt.Println("Done")
}

This technique is also discussed in the lecture about managing goroutines.

Task 3: Sharing one channel between more than two goroutines

Channels are variables, and therefore can be passed around like variables. What if we pass a channel to multiple functions? Can two or more goroutines send to or receive from the same channel at the same time?

Write a program that starts three goroutines and passes a channel (the same channel!) to each one.

Let one goroutine send a number of consecutive integers (1,2,3,…), and let each of the other two continuously read elements from the channel.

Will both goroutines receive the same data?

Will the sent data get split up and reach both receivers, or will the receivers

(This task comes with no code template.)

Time To Practice: Reflection

Time To Practice: Reflection

The standard formatting of structs is not very nice to read. Remember our CelestialBody, Planet, and Star structs? Filling them with data and printing them with

fmt.Printf("%v\n", ....)

yealds this uninspiring output:

{{Sun 1988000000 1391400 587h0m0s} 0 4.83 -9223372036854775808 0xc4200a2000}
{{Mercury 330 4879 1407h0m0s} 3.7 false true [] 0xc4200a2060 <nil>}
{{Venus 4870 12104 5833h0m0s} 8.9 true false [] 0xc4200a20c0 0xc4200a2000}
{{Earth 5970 12756 24h0m0s} 9.8 true true [Moon] 0xc4200a2120 0xc4200a2060}
{{Mars 642 6792 24h37m0s} 3.7 true false [Phobos Deimos] <nil> 0xc4200a20c0}

I think we can do better.

Your tasks

Task #1: Pretty-print a struct

Write a library package pretty with a function

Print(i interface{}) 

that takes an interface{} parameter and prints this parameter according to the following rules:

  • If the type is a struct , then print
  • the struct’s type
  • an opening brace
  • with indent: all fields, including field name, type, and value
  • a closing brace
  • Else just print the type and the value of the parameter.

Consider that the passed-in value may be invalid. The reflect package has a function that tests the validity of a value.

Obviously, we need the reflect package here, as all this information (types, struct field names, unexported struct fields) are only accessible through reflection.

The provided code includes a file main.go with the sun and a few planets already being set up for use. There is also a package file “pretty.go” prepared for you to implement.

Task #2 (optional): Follow pointers

Expand your solution of task #1 to follow pointers.

That is, if the current value is a pointer,

  • print “*”
  • print the name of the struct field IF the pointer is a struct field
  • print the value that the pointer points to (with indenting)

Consider these edge cases:

  • The pointer may be nil
  • The data may contain circular references. Address this by doing either or both of:
  • Choosing a maximum depth for follwing pointers (say, 10)
  • Maintaining a list of already visited nodes, and only follow pointers that do not point to an already visited node.

A hint: In my sample solution, I used a helper function “print(v reflect.Value,…)” with some more parameters that I call recursively when evaluating fields and following pointers. The additional parameters control the indent level and pass field names down one level, so that they can be printed with the proper indent level.

I also made this print() function a method of a struct, and in this struct I keep a list of already visited pointers.

However, the provided solution is only one way of implementing the tasks. Feel free to find a different, maybe better, solution.

The file prettySolution.go.txt implements tasks #1 and #2. Its output looks like this:

Output:

main.Star: {
    main.CelestialBody: {
        Name (string): Sun
        Mass (int64): 1988000000
        Diameter (int64): 1391400
        RotationPeriod (time.Duration): 587h0m0s
    }
    Distance (float64): 0
    Magnitude (float64): 4.83
    Discovery (int64): -9223372036854775808
    *FirstPlanet (main.Planet):
        main.Planet: {
            main.CelestialBody: {
                Name (string): Mercury
                Mass (int64): 330
                Diameter (int64): 4879
                RotationPeriod (time.Duration): 1407h0m0s
            }
            Gravity (float64): 3.7
            HasAtmosphere (bool): false
            HasMagneticField (bool): true
            Satellites ([]string): []
            *next (main.Planet):
                main.Planet: {
                    main.CelestialBody: {
                        Name (string): Venus
                        Mass (int64): 4870
                        Diameter (int64): 12104
                        RotationPeriod (time.Duration): 20998800000000000
                    }
                    Gravity (float64): 8.9
                    HasAtmosphere (bool): true
                    HasMagneticField (bool): false
                    Satellites ([]string): []
                    *next (main.Planet):
                        main.Planet: {
                            main.CelestialBody: {
                                Name (string): Earth
                                Mass (int64): 5970
                                Diameter (int64): 12756
                                RotationPeriod (time.Duration): 86400000000000
                            }
                            Gravity (float64): 9.8
                            HasAtmosphere (bool): true
                            HasMagneticField (bool): true
                            Satellites ([]string): [Moon]
                            *next (main.Planet):
                                main.Planet: {
                                    main.CelestialBody: {
                                        Name (string): Mars
                                        Mass (int64): 642
                                        Diameter (int64): 6792
                                        RotationPeriod (time.Duration): 88620000000000
                                    }
                                    Gravity (float64): 3.7
                                    HasAtmosphere (bool): true
                                    HasMagneticField (bool): false
                                    Satellites ([]string): [Phobos Deimos]
                                    next (*main.Planet): <nil>
                                     previous (!*main.Planet ALREADY VISITED)
                                }
                             previous (!*main.Planet ALREADY VISITED)
                        }
                     previous (!*main.Planet ALREADY VISITED)
                }
            previous (*main.Planet): <nil>
        }
}

Task #3 (optional): Print struct tags

Also print struct field tags after the field values.

Happy coding!

Welcome To The Modules Lectures Draft

The new Go Modules concept

The following lectures cover the Go Modules concept that has been released for production use with Go 1.14.

Tight schedules have prevented me from having full-featured video lectures ready for roll out so far, and I apologize for that. I provide the text version of the lectures here as it is important that you get familiar with the modules system. I encourage you to read through the lectures and follow the examples. Go modules are the new way of managing libraries, and once you recognize the advantages over plain old GOPATH, you will not want to go back.

Right now these lectures are located at the end of the course. This is because so many lectures in the course refer to GOPATH, and placing the lectures in the middle of the course will cause confusion as the new and existing lectures would not fit together.

The next steps now consist of:

  • Adjusting all lectures that refer to GOPATH in some way to refer to the respective Go Modules feature instead (or simply to not mention GOPATH anymore, depending on the context).

  • Recording and editing audio for all new and changed lectures

  • Creating screen recordings

  • Cutting audio and screen recordings to form a lecture

  • Adding animations and overlays

  • Re-arranging the course curriculum to make the modules section fit naturally into the curriculum

So please bear with me while I move this transformation forward. In the meantime, enjoy the text lectures and get proficient in creating and using Go modules.

Using Third-Party Packages

Transcript

Up to this point, we have solely used go run to run our Go programs, and imported packages only from the standard library. Now let’s have a look at using third-party packages. In contrast to the packages of the standard library, these packages reside on some remote server, inside the repository of a version control system like Git, Mercurial, or Bazaar.

Including a third-party package in your project is ridiculously easy; it requires only two simple steps.

1. Add an import directive

First step: you add an import directive to your code, just like the one you already used for importing packages from the standard library. This time, however, the import string describes the remote location of the package. It is in fact a URL without the https:// prefix.

Let’s look at an example.

package main
import "github.com/appliedgo/proverbs"
func main() {
    fmt.Println(proverbs.Random())
}

Here, we want to use a package that resides in a repository named “proverbs” owned by “appliedgo” on “github.com”. Note that the last part of the path is also the name of the package. This is not enforced by the language specification but rather is a convention. You may find exceptions to this rule; most often, you will see repositories that start with “go-” to distinguish them from another repository on that server with the same name but written in a different language. The package name itself, however, does not have the “go-” prefix as there is no ambiguity inside the Go code. After all, Go code can only import Go packages.

When importing a package, you can also choose to use an alias for the package name; for this simply put the alias in front of the import path:

import pv "github.com/appliedgo/proverbs"

The obvious use case for this is to avoid name clashes of packages that have the same name. Besides that, an alias can also come handy if the package name is long and bulky and has to be used in many places in your code. A short alias helps making the code easier to read.

2. Build the binary

Now that we have defined our input path, let’s go ahead and run the binary.

$ go run main.go

Now the go command starts looking for the remote repository and downloading and extracting the package into a local cache on your file system. All this happens automatically; all you had to do is specifying the import path in the code. This level of convenience is really hard to beat, and is one of the big benefits of a really well-crafted toolchain.

For the curious, the local cache resides inside a directory that is determined by the environment variable GOPATH. If this environment variable is not set for your shell, it defaults to

$HOME/go

on Unix-like systems, or

%USERPROFILE%go

on Windows.

You can get the current value of GOPATH by typing

$ go env GOPATH

which returns the value of the environment variable, or the default path if the environment variable does not exist.

Not covered in the video

Version control systems supported by the Go toolchain

To import a library, Go supports a couple of different version control systems for downloading remote packages. These are (in alphabetical order):

Bazaar      .bzr
Fossil      .fossil
Git         .git
Mercurial   .hg
Subversion  .svn

Creating Custom Packages

Transcript

Now that we know how to import custom packages, let’s create our own custom library package.

In contrast to a main package that defines a binary, a library package can contain functions, variables, and types, but no main function.

Library packages serve two main purposes.

  1. They help dividing a large project into separate, manageable pieces.
  2. They allow creating collections of reusable code.

How to write a package

Writing a library package is almost as easy as writing a main package.

As an example, we will create a library for rolling dice.

Step 1: Choose a location

Library packages should reside in a directory with the same name as the package name. This is just a convention and not enforced by the compiler, as you learned in the chapter about using third-party packages. However, I recommend sticking to this rule, as this makes importing and using your packages more straightforward.

You can put the directory containing your package anywhere you want. We will see later how to prepare your library so that clients can discover, install, and use it.

For our example library, let’s name that directory simply “dice”.

The file names you use for your .go files inside the directory are not following any particular convention. It is the package declaration inside the file that defines them as part of the package, as we will see in a minute.

For now you need only one file; let’s call it roll.go .

Step 2: Write the code

2.1 Declare the package

Create the file and open it in an editor. Now you can declare the package as:

package dice

As mentioned before, the name used here should be the same as the name of the package directory.

2.2 Add data and functions

Now you can import other packages and declare constants and variables and functions as usual. Except that we now have to specify which of these objects shall be visible to a client of our library.

2.3 Define an exported function

Go has a simple visibility rule for packages.

If the name of a function, variable, or type starts with an uppercase letter, it is visible to the clients of the package

All lowercase names are only visible within the package itself.

So in order to define an exported function Roll() for rolling a die, we just need to spell the function with an uppercase R, and we are done.

func Roll(sides int) int {
    return rand.Intn(sides) + 1
}

As we use the rand package here, we need to add an import statement.

import "math/rand"

Go-aware editors do this automatically, as you surely have noticed in the videos.

The init function

A package might require initialization at startup. Go provides a special function called init for this.

We can use it here to set a seed value for the random number generator.

func init() {
    rand.Seed(time.Now().UnixNano())
}

A suitably random seed value is the current time that we can get by calling time.Now() . This call returns a value of type time.Time that we turn into an int64 value using UnixNano() , because this is what rand.Seed() expects.

The init() function runs once when the program starts, and can do any required initialization, like, as in our example, seed the random number generator with a sufficiently arbitrary value.

I have to point out that many Go developers frown upon the user of the init() function, as its main purpose is to initialize package-level variables. Having a global package state is almost always a bad idea. On top of this, if there are multiple ‘init()’ functions in your code, they are called in no defined sequence, which can lead to subtle bugs that the compiler cannot catch.

For these reasons, it is almost always better to avoid init functions and put any global state into a custom data type, and all initialization into a constructor method for that type.

You will learn more about custom data types in section 3.

In our case, we have no global state inside our package anyway, but only a random number generator that needs some initialization, so we can simply define a public Seed function instead that clients can call at will.

func Seed() {
    rand.Seed(time.Now().UnixNano())
}

The advantage of this becomes apparent when we add an int64 parameter to our Seed function. This way, a unit test can set the random number generator to a defined start value to produce replicable output. This would not have been possible when only using an init() function.

func Seed(n int64) {
    if n == 0 {
        n = time.Now().UnixNano()
    }
    rand.Seed(n)
}

Now our dice library is complete and ready for use.

Summary

  • The name of the package directory should be the name of the package.
  • A package can be spread across multiple files.
  • Start each package file with a package directive: package <name> .
  • Identifiers starting with an uppercase letter are exported.
  • Identifiers starting with a lowercase letter are internal.
  • Avoid global package state and the init function. Use a custom type and a constructor instead.

Not covered in the video

Tips

Some tips for writing library packages.

Use short, clear, descriptive names.

Avoid long names.
Avoid cryptic names.
Avoid ambiguous names.

Avoid stutter.

Use workflow.New() instead of workflow.NewWorkflow() .
Use workflow.Component rather than workflow.WorkflowComponent .

Your package should provide something, not just contain something.

Group your packages by purpose.
Do not create packages named like “utility” that are filled with a random collection of arbitrary data structures and functions. Rather, group your packages by specific purposes and choose names that reveal those purposes.
Think carefully about the API you want to present to clients.

Init function details

  • The init function can be defined multiple times within the same package, and even within the same file.
  • All init functions are executed sequentially, one package at a time. However, an init function can spawn a goroutine to run code concurrently with the initialization sequence.
  • The init function cannot be called or otherwise referenced from within a program. Only the Go runtime can invoke an init function.
  • Neither the Go compiler nor the Go runtime impose a specific order on running the init files. Build systems therefore should pass files belonging to the same package in lexical order, to achieve a reproducible initialization behavior. (The better option of course is to avoid using init() altogether. Only in rare cases you will need an init() function.)

Publish Your Package

Transcript

The dice package we just created resides somewhere on the local file system. Client code can import it using a relative path. For example, I might create a main package in another folder,

cd ..
mkdir diceclient
cd diceclient
code main.go


And I can then import the dice package like so:

package main
import (
    "fmt"
    "../dice" / ./ or ../ indicate a local import path
)
func main() {
    fmt.Println(dice.Roll(6))
}

However, this is not particularly useful.

  • First, if, at some point, I move the package to another folder for any reason, the import directive breaks.
  • Second, in almost all real-life situations you want to have a package in some central location, so that you and others can access this package in a uniform way, using an import path that does not change over time.

So what we want to do as the very next step is to publish our package somewhere.

Fortunately, the tools and infrastructure are readily available. We only need a common version control system like Git or Mercurial, and a remote repository server like GitHub, GitLab, or BitBucket. Of course you can use any repository server; especially if the package is meant to be used inside your company only, you can use your company’s internal repository servers.

In this course I will use git as the version control system and GitHub as the remote repository server. When working on lectures or exercises, you can of course use any of the other SCM’s that Go supports, as well as your favorite remote server.

For this course, I am assuming that you are familiar with using a version control system, so I will not go into the details of git or version control in general. Learning version control systems would fill a course of its own, so if you never heard of Git or the concept of version control before, this would be a good occasion to pause this course and work through at least an introductory tutorial on Git.

Step 1: Add the code to source control

So let’s set up source control now.

As a first step, I will initialize a local repository and add our dice code to that repository.

With git, this is done in three steps:

git init

for initializing the repository metadata,

git add -A

for adding all files inside and beneath the current directory to the repository, and

git commit -m "Initial commit"

for committing our changes to the repository.

Step 2: Set up a remote location and push the code there

Now we need to set up the remote repository. While I could do this in a Web browser, I will use a command line tool instead.

With

hub create -d "dice: a Go package for rolling dice" appliedgocourses/dice

I create a new, empty repository named “dice” in the organization named “AppliedGoCourses” on GitHub. You would of course have to change this to a GitHub user or organization that you have write access to.

The -d flag sets the repository’s tagline, which is not strictly required but helpful for other people who happen to come across our repository on GitHub. I always recommend setting a tagline.

Now I can add the remote location and push the local repository to the remote repository. The default remote location of a repository is usually called “origin”, and I have it refer to the GitHub repository I created in the previous step.

git remote add origin git@github.com:appliedgocourses/dice
git push --set-upstream origin master

(And when calling push for the first time, I also need to set the upstream of this repository, and here I set it to the master branch of the remote named “origin”.)

Result: a stable, globally available import path

As a result, the package is now available for download to wherever the client code resides, using an import path that never changes.

And if you remember the lecture about using third-party libraries, you already know how the import path for our dice package looks like:

import "github.com/appliedgocourses/dice"

The path, by the way, is not case-sensitive, so you can as well use all-lowercase here.

Summary

Not covered in the video

Vanity import paths

Import paths do not need to directly point to a code repository. The go get command also supports so-called “vanity import paths” . In a nutshell, a vanity import path represents a URL that is under the control of the package author. When go get queries this URL, the server redirects go get to the actual repository.

As an example, the package appliedgo.net/proverbs is hosted on github com, and go get appliedgo.net/proverbs redirects to github.com/appliedgo/proverbs .

Technically, go get calls https://appliedgo.net/proverbs?go-get=1 , and the server at appliedgo.net responds with an HTML document that contains a meta tag providing the path to the repository.

You can try this out right away. Simply call

go get -u -v appliedgo.net/proverbs

and inspect the output:

get "appliedgo.net/proverbs": found meta tag get.metaImport{Prefix:"appliedgo.net/proverbs", VCS:"git", RepoRoot:"https://github.com/appliedgo/proverbs"} at //appliedgo.net/proverbs?go-get=1
appliedgo.net/proverbs (download)

The obvious advantage for package authors is that the import path is decoupled from the repository hosting service. If at one fine day I decide to move the proverbs package to another repo hosting service, the import path would stay the same.

Also, package authors can choose a nice, short, easy-to-remember import path, no matter how long the URL to the repository is.

The disadvantage for package users is that they now depend on two separate things for accessing a package. As before, the public repository must exist and must be accessible. But on top of this, the users must rely on the package author to keep maintaining the custom domain (in the above example, appliedgo.net) as well as the server that does the redirect. So for the package user, there is an extra link in the chain that may break.

What is my take on this situation? Well, I think that being decoupled from the actual repository hoster is a good thing. If a code author decides (or even needs) to change the hosting service, no client libaries would be affected by the change. No existing import paths will break.

And if a package author decides to pull the plug on his project and shuts down the server at the custom domain, this surely will break all existing import paths; however, most likely this author then also sunsets the repository itself, and all users would have to move over to a fork of the package or a different package altogether. Which means they would have to adjust their import paths anyway.

So as you can see, I am in favor of vanity import paths. Make it easy for your clients to remember your packages!

Modules

Transcript

So far we have seen how to create and publish packages but not how to use them. The reason is that we first need to examine another concept in Go called modules.

The case for modules

One aspect of packages that we have not yet talked about is dependency management and versioning. The problem in a nutshell is: To keep your code stable, you will want to specify a particular version, or a range of allowed versions, of a package to use in your code. However, a package does not carry any version information.

So a package client cannot specify a particular version or version range when importing a package.

To address this, Go provides a concept called “modules”. A Go module is the base unit of versioning and dependency management. You can assign a specific version to a module, and you can control the required versions or version ranges of those modules that your module depends on.

From a project point of view, modules reside between packages and repositories.

A module can include one or more packages. A repository can contain one or more modules.

Typically, you will want to start with a single module per repository. Sub-modules can become a bit tricky to handle, so you should first become familiar with the module concept before starting to use sub-modules.

Next steps

In the following lectures, you will learn how to create and use modules. We will cover the basic steps that will help you working through the lectures, practices and exercises of the following lectures.

Later, in the lectures on the Go toolchain, we will have a closer look into dependency management, semantic versioning, and other aspects of the modules concept.

Summary

  • Go manages code dependencies on the basis of modules
  • A module groups one or more packages into a single unit that has
  • a specific version, and
  • a well-defined set of dependencies on other packages
  • A version control repository can contain one or more modules
  • For easy maintenance, use a single module definition per repository.

Links

The Modules article in the golang/go Wiki

Modules Support in the go command documentation

Creating a Module

Transcript

The first steps into using Go modules turn out to be quite simple. Let me demonstrate this by wrapping our dice and diceprinter packages into a module.

Initialization

To initialize a module, type

go mod init <module path>

at the root directory of the repository containing the packages.
In our case, the module path is

go mod init github.com/appliedgocourses/dice

The init command replies with,

go: creating new go.mod: module github.com/appliedgocourses/dice

Apparently, go mod init creates a file called go.mod. Let’s examine that file.

module github.com/appliedgocourses/dice
go 1.14

The first line of a go.mod file always contains the module directive that declares the module’s import path. This path points to the remote repository of the module, so every time some client code imports our module, it reaches out to the remote repository server using the given path as a URL.

Now you might ask, what if I do local development on that module? Do I always have to push any changes to the remote repository, in order to enable test code to access the module?

The answer is, there is a way to tweak the import to use a local copy of the module instead. We will discuss this in detail in a later lecture.

Back to our go.mod file. Note that there is only one module import path, even though we have two packages with two different import paths inside. All packages inside a module share the same base import path, and they also share the same version information.

Especially, you will not want to run go mod init for the diceprinter subpackage. The diceprinter subpackage is automatically recognized as part of the github.com/appliedgocourses/dice module.

Summary

  • Create a module by calling go mod init followed by the path to the remote repository of the package.
  • go mod init creates a file called go.mod that specifies the module’s import path.
  • go.mod does not treat sub-packages separately. The base import path, as well as the version information, are the same for all packages included in a module.
  • Do not run go mod init for subpackages. They are maintained by the same module as the base package.

Using Modules

Transcript

At this point, we have dice and diceprinter packages, and we have a module wrapped around them, and now it is time to use our new module.

For this, we first need to generate a go.mod file for the main package. Even though the main package is not a library but rather an executable, the go.mod file is also required here; and this time, we use it to specify the modules that the executable needs.

So let’s create a new go.mod like before, by calling

go mod init main

in the directory where our dice client code resides. This time, instead of a module path, we use just the name “main” here, to indicate that this is an executable and not another library. At this point, the go.mod file contains nothing particularly interesting:

module main
go 1.14

So far, the file lists the module name, and the Go version that was used for creating the file.

When we now call

go run main.go

the code compiles and runs as expected. The go.mod file, however, has changed. It now contains a new directive:

module main
go 1.14
require github.com/appliedgocourses/dice

The require directive specifies any module that the current module depends on. This information seems redundant, as there is already an import path inside the code that points to the same location. However, we will see the purpose of this line when adding version information.

Summary

  • A main package also uses a go.mod file.
  • Here, go.mod is used for listing the modules that the main module depends on.

Not covered in the video

Why does go.mod contain the Go version used for creating the file?

Different Go versions have different language features. Usually, new versions add new features; very rarely, it happens that a feature changes or gets deprecated. To ensure reproducible builds, Go therefore must know the language being used for developing the project. If your project requires a newer language feature or maybe just a bug fix, or if you simply do not want to test your project with a previous version anymore, you can increase the version number in go.mod to reflect this fact.

Links

Initialize new module in current directory (Go command documentation)

The go.mod file (Go command documentation)

Add Version Information

Transcript

In the previous lecture, we created a library module without any version information. However, one of the main purposes of Go modules is to manage versioning, so where do we specify the version of our module?

Module clients must be able to access the current version as well as any previous version of a module, and this makes using the version control system a very natural choice. Again I will use Git here, but the concept of tagging is quite similar across version control systems.

Git lets you add a tag to a repository. A new tag always marks the latest status of that repository. While the repository changes and evolves, tags enable the developer to refer back to a previous version of the repository.

Go uses exactly this mechanism to track module versions. Let’s tag our dice repository to see how it works.

By calling git tag we can add a new tag to a repository.

git tag -a v0.1.0 -m "Initial version"

The -a flag specifies the tag’s name, and the -m flag allows you to add an optional comment.

The tag name must follow a convention called Semantic Versioning, or SemVer in short. This is important, because this convention allows to unambiguously specify the major version, the minor version, and the patch level of a module. The name always starts with a lowercase v , followed by major version, minor version, and patch level, separated by a full stop.

Semantic versioning is an established concept by now, so I will not go into details at this point. If you are new to semantic versioning, there is an extra SemVer lecture for you that provides a brief introduction to the concept.

Now having added a tag to the repository, we can push that tag to our remote repository as usual.

git push

The command

git tag

lists all the tags that we have defined in the repository so far, and

git log

shows the commits that have been tagged.

When we now run the client code again,

go run main.go

the go.mod file now contains the SemVer tag right at the end of the require directive.

module main
go 1.14
require github.com/appliedgocourses/dice v0.1.1

How to request a specific minimum module version

With the SemVer tag in place, client code can now easily request to use a specific minimum version of our module.

So when the author of the dice library publishes a new version, say, v.0.2.0, we can decide to upgrade to that version by changing the dependency inside the go.mod file.

The go.mod file can be edited directly, but for added syntax checking, I recommend using the go tool for that purpose.

So to update our requirement from v0.1.1 to v0.2.0, we can type

go mod edit -require github.com/appliedgocourses/dice@v0.2.0

and the go.mod file gets changed accordingly.

I bet it did not escape your attention that the command expects an “@” sign between module path and version number, unlike the go.mod file itself, which expects a space between the two. The @ sign saves us from having to deal with forgotten or mismatched quotes, which is something that tends to happen too easily on the command line - at least for me.

Summary

  • A module version is simply a tag in the source control system.
  • Version numbers must comply to the rules of Semantic Versioning.
  • The go.mod file documents the required version for each dependency.
  • The go subcommand go mod edit allows to update requirements without having to edit go.mod manually.

Not covered in the video

Useful go mod edit directives

go mod edit allows you to edit about all aspects of the go.mod file. Here are some useful directives:

  • -fmt : reformat go.mod . Note: other directives also reformat the file.
  • -module : change the path in the module line.
  • -require=path@version : reqire a specific version of a dependency.
  • -exclude=path@version : exclude the given dependency path and version.
  • -replace=oldpath[@version]=newpath[@version] : replace a dependency by another. Note: if you omit the new version, newpath should point to a local module root directory. This gives you a super-easy way of modifying modules locally and testing them as if they were located on the import path.
  • -droprequire , -dropexclude , and -dropreplace drop a requirement, exclusion, or replacement, respectively, on the given module path and version.

For more info, open a shell and type

go help mod edit

Links

The go.mod file - The Go Programming Language

Edit go.mod files from tools or scripts - The Go Programming Language

How Go Selects a Module Version

Transcript

The problem of finding a non-conflicting set of module versions

From your project’s perspective, versioning is easy. Determine the module version you need, document this in go.mod , and you are done.

However, the modules your project uses may also have dependencies on other modules. And more often than not it happens that two modules depend on a third module, but each of the two modules need a different version of the third one.

In a dependency graph, this scenario shows as the so-called “diamond-shaped dependency”.

Diamond-shaped dependencies

Here module A from your project requires modules B and C. Both B and C require module D; however, B needs version 1.0 and C needs version 1.3 of module D.

Most languages, including Go, do not allow compiling two versions of the same module into a binary. If B and C are third-party modules that are outside your control, you cannot simply adjust the requirements of B or C to the same version.

How can this version conflict be resolved?

Traditional solvers

This problem has a problem – a meta-problem if you want. The version selection problem is NP-complete. So – loosely speaking – when the number of dependencies gets sufficiently large, determining whether there is a set of module versions that fulfills all requirements and does not contain any version conflicts would take an enormous amount of time.

For this reason, practical dependency solvers usually use heuristic algorithms to find a solution fast, at the risk of not finding a solution although one exists. However, the problem with heuristics based solvers is that it is hard to reason about why a specific set of versions has been chosen.

Go does it differently

The dependency version solver in Go takes a different approach. There is only one central rule for resolving dependencies. This rule says,

Always choose the minimal version that fulfills a set of version requirements.

This approach is called Minimum Version Selection , or short, MVS.

In our example, the MVS solver would choose version 1.3, because this one is the minimal version that fulfills both version requirements of module B and C.

Why does this work? The solver implicitly assumes that all modules strictly follow the principles of semantic versioning. One of the rules of semantic versioning is that only major version upgrades may introduce breaking changes. Updates to minor versions or to the patch level must be backward-compatible to earlier versions. Based on this rule, the solver can safely choose version 1.3 of module D, knowing that it satisfies the requirements of both module B and module C.

Version conflicts

Now you may ask, what if version 1.3 of module D is actually not backward-compatible to earlier versions? Module B then would not be able to use version 1.3 of module D, while module C requires at least version 1.3.

Version conflict

This is definitely a conflict, and ultimately, this conflict must be resolved by fixing version 1.3 to be backward-compatible. Until this happens, go.mod lets us explicitly exclude that version from being considered.

Consequences from using Minimum Version Selection

The Minimum Version Selection algorithm has some important consequences to consider.

1. No guaranteed builds

MVS always finds a solution. It does not guarantee, however, that this solution results in a correct binary, or even compiles at all. This question is simply outside the focus of Go’s dependency management system. It is the developer’s task to determine if a particular version of a particular module is compatible with the developer’s code and has no bugs that get in the way.

2. A new dependency uses the latest version

When a new dependency is added to go.mod without manually specifying a version, the latest version is used by default. In most cases, this should match your actual requirements, unless you have a specific need to develop against an older version of a module.

3. Dependencies do not automatically upgrade to newer versions

When a dependency ships a newer minor or patch version, it is the developer’s responsibility to update go.mod accordingly. The go mod tool does not automatically update all previously newest versions by the now newest version, even though the latest version should be backward compatible.

This rule is important, as it guarantees reproducible builds.

However, there is a shortcut available for upgrading all dependencies to the latest versions.
You can upgrade all dependencies of your module to the latest minor or patch release by running

go get -u ./...

from the module’s root directory.

go get -u=patch ./...

does the same but only for newer patch releases.

Of course, you should then test your code thoroughly to ensure nothing breaks.

Summary

  • Two modules may depend on different versions of a third module.
  • The Go dependency solver always choses the minimal version that matches all requests.
  • For this, the solver assumes that all modules follow the backwards compatibility promise of semantic versioning.
  • Because of this, the solver always finds a solution.
  • However, if some version of a module contains a bug, the solver cannot catch this.

Links

research!rsc: Version SAT

research!rsc: Minimal Version Selection (Go & Versioning, Part 4)

Thank you!

Congratulations!

You reached the end of this course. A lot of lectures are now behind you, and I hope you enjoyed your journey and learned a lot along the way.

At this point, I want to say thank you for taking the course! You are awesome. Learning a new language is never easy, even in the case of Go. But you did it!

Maybe your head is spinning now - don’t worry, this is the classic “after-course-hangover” :slight_smile: Your brain is busy processing all the new information.

First, take a short rest. You deserved it.

But then don’t wait too long to put your new knowledge into practice. It will pay off. Try some small toy projects first until you start feeling confident with writing Go code from scratch; then try bigger ones. Read code. There are a lot of Go projects on GitHub and elsewhere (but mostly on GitHub). Soon you will get a feeling for the language, backed by the profound Go knowledge you acquired through this course. And I am sure you will enjoy Go’s conceptual simplicity all along the way.

Happy coding!