Colin Tester

Helping to build a more robust and inclusive web

Three to go!

Published:
By:

Multiple Go apps from one Digital Ocean droplet with their own domain/sub-domain.

In planning a project with a need to abstract out some of its operations to separate, self-contained services, it seems sensible to utilise the benefits of containerisation. Sometimes, however, this approach might appear overly complex, particularly for a project that begins its life with fewer demands on technical resources.

With that in mind, I was interested to understand what was involved in setting multiple web-based Go apps running on one server; each operating from their own domain/sub-domain.

My server provider for this exercise is Digital Ocean who provide plenty of content to help with setting up numerous server configurations, and it is from some of their content that I found a solution.

Introduction

In this post, I assume that a Digital Ocean account and droplet (Linux) has been created along with a non-root user for connecting into the server using a secure shell (ssh). Plus, nginx is installed on the droplet and you have your own domain name which can be pointed to the Digital Ocean droplet.

Please see the following Digital Ocean articles on how to prepare for the preceding assumptions.

I intend to run three separate Go apps and make them accessible from their own domain; one from the main domain, and the remaining two from sub-domains:

  1. First app – example.com
  2. Second app – second.example.com
  3. Third app – third.example.com

Local development environment

Let’s set up the local working space. In a terminal window navigate to your working space and create a new project folder.

$ mkdir three-to-go && cd three-to-go

Let’s create the First simple Go app file within its own folder.

$ mkdir -p src/first && touch src/first/main.go

Open the main.go file in a code editor and save into it the following code:

package main

import (
  "fmt"
  "net/http"
)

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

    // http response text:
    fmt.Fprintf(w, "Hello from the [ First ] Go app.")
  })
  
  http.ListenAndServe(":9991", nil)
}

Each of the three Go apps will listen to http requests on a unique port number. The First Go app will listen on port number: 9991. The Second and Third Go apps will listen on port numbers 9992 and 9993 respectively.

Now, repeat the above steps to create a main.go file and content for the two other Go apps, each within their own folder.

$ mkdir -p src/second && touch src/second/main.go
$ mkdir -p src/third && touch src/third/main.go

Open each main.go file and save to it the same code as in the First Go app, changing the http response text and http port number.

For the Second Go app:

package main

import (
  "fmt"
  "net/http"
)

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

    // http response text:
    fmt.Fprintf(w, "Hello from the [ Second ] Go app.")
  })
  
  http.ListenAndServe(":9992", nil)
}

For the Third Go app:

package main

import (
  "fmt"
  "net/http"
)

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

    // http response text:
    fmt.Fprintf(w, "Hello from the [ Third ] Go app.")
  })
  
  http.ListenAndServe(":9993", nil)
}

App binary build and deployment

The plan is to deploy the Go apps to the server as executable binaries, so let’s build those first, locally.

Go’s compiler facilitates building a binary for many different operating systems and architecture and we can specify those options with the go build command.

To build a executable to run on Ubuntu we need to select Linux as the operating system and amd64 as the architecture. That is done by setting environment variables: GOOS=linux GOARCH=amd64.

$ GOOS=linux GOARCH=amd64 go build -o bin/first/main src/first/main.go

Using the -o flag, sets the output file path and filename of the compiled executable file main into the local bin folder.

Repeat for the other two apps (Second and Third):

$ GOOS=linux GOARCH=amd64 go build -o bin/second/main src/second/main.go
$ GOOS=linux GOARCH=amd64 go build -o bin/third/main src/third/main.go

The plan is to store the Go apps within the home folder of the server’s non-root user, in a sub-folder labelled: ‘go’.

Open a new terminal window and connect to your server via a secure shell (ssh).

$ ssh non-root-user@server-ip-address

Substitute your own non-root user name for the non-root-user part of the ssh string, and substitute your server’s IP address for server-ip-address.

When connected, create the folder where the executable binaries will be stored.

$ mkdir go

We can now copy the binaries securely to the Digital Ocean server. There are two ways that can be done, using either the scp or rsync command line utility.

Back in the terminal window of the working project folder. Again, substitute your own non-root user name and your server’s IP address for those in the following command strings.

$ scp -r bin/ non-root-user@server-ip-address:/home/non-root-user/go

Using the -r flag to recursively copy folders and files from the local bin folder.

Alternatively, use rsync to synchronise the local files with the server – better and faster, particularly if there is a need to push updates to the server later.

$ rsync -avz -e ssh bin/ non-root-user@server-ip-address:/home/non-root-user/go

Using flags as -avz:

  • -a Archive mode, recursively: preserving permissions, modification times, symlinks, etc
  • -v Increase verbosity
  • -z Compress file data

Plus -e specify the remote shell to use.

The rsync command will synchronise all folders and files in the local bin folder over to the go folder we created earlier in the non-root user’s home folder on the server.

The executable binaries should now be in the go folder on the server. You can check this by listing, recursively, the contents of the go folder.

Via the terminal window with the secure shell connection (ssh) to your server:

$ ls -R go
go:
first  second  third

go/first:
main

go/second:
main

go/third:
main

Domain name setup

If you have followed the steps in the article How To Set Up a Host Name with DigitalOcean, you should have your own domain name set up on your Digital Ocean account. You will also need to point your domain and sub-domains to the droplet you are using for your Go apps.

Please see how to manage DNS records in the Digital Ocean docs section, where you can set the WILL DIRECT TO option to point your domain/sub-domain to your droplet instance.

Setup Nginx to serve multiple Go apps

As stated in the Introduction, I assume that nginx has been installed onto your droplet. You can check if it is running by visiting the server’s IP address in your browser.

nginx welcome content.

On the server, nginx is to be set up as a reverse-proxy to which a client (browser) will connect. The proxy represents the three apps’ individual http server, connecting to them via port-forwarding.

Remember, that each of the three apps, when running, will listen for http requests on a unique port number. So, we need to create an nginx configuration file for each of the apps, defining the domain name and the location of the end server (a Go app) which the proxy represents.

Nginx retains site configuration files in its sites-available folder in which the configuration file for the First Go app will be created.

The Vim text editor will be used to create and edit the configuration file, as it will be installed already for your droplet.

Within the terminal window with the ssh connection to your server, type:

$ sudo vi /etc/nginx/sites/sites-available/first-app

As we are logged in as a non-root user, we need to prefix the vi command with the special sudo (super user do) unix command to allow our non-root user to have elevated privileges. The command vi opens Vim, creating the file first-app at the path to sites-available.

In Vim, press the ‘i’ key to enter -- INSERT -- mode and add the following to the file.

server {
  server_name example.com www.example.com
  
  location / {
    proxy_pass http://localhost:9991;
  }
}

In the configuration file, we are specifying a server definition for the domain names we wish to catch, and the location of the end server to which the proxy will pass the requests.

Note: the proxy_pass value includes the port number (9991), that being the port number the First Go app is listening on.

Substitute your own domain name for example.com. Note that we have included the www sub-domain in the server_name value, defining that either of the two domain variants are to be captured.

Exit Vim’s -- INSERT -- mode by pressing the ESC key, then save the configuration file by typing :wq – the Vim command to write (w) changes to the file and then quit (q).

Now that we have a configuration file for the First Go app, we can create one each – using Vim – for the Second and Third Go apps.

For the Second Go app:

$ sudo vi /etc/nginx/sites/sites-available/second-app

Enter the following in Vim’s -- INSERT -- mode (i) …

server {
  server_name second.example.com
  
  location / {
    proxy_pass http://localhost:9992;
  }
}

…then save the file ESC, :wq

Note: that the server_name value is now set with the sub-domain name and the proxy_pass value points to a location using the port number the Second Go app is listening on (9992).

For the Third go app:

$ sudo vi /etc/nginx/sites/sites-available/third-app

Enter the following… then save the file.

server {
  server_name third.example.com
  
  location / {
    proxy_pass http://localhost:9993;
  }
}

Now that we have configuration files for each of the three Go apps in nginx’s sites-available, we now need to enable each individual site. That is done by creating a symbolic link in nginx’s sites-enabled folder, linking to each Go app configuration file created in sites-available.

Create a symlink for each of the three Go apps:

$ sudo ln -s /etc/nginx/sites-available/first-app \
/etc/nginx/sites-enabled/first-app

$ sudo ln -s /etc/nginx/sites-available/second-app \
/etc/nginx/sites-enabled/second-app

$ sudo ln -s /etc/nginx/sites-available/third-app \
/etc/nginx/sites-enabled/third-app

Check that the symlinks have been created as expected by using the unix list command:

$ sudo ls -l /etc/nginx/sites-enabled
...
... first-app -> /etc/nginx/sites-available/first-app
... second-app -> /etc/nginx/sites-available/second-app
... third-app -> /etc/nginx/sites-available/third-app

The -l flag will display in long listing format and show how the symlinks are connected, e.g. link_file -> source_file.

To get nginx to read the configuration files, we need to instruct it to reload.

$ sudo nginx -s reload

The -s reload flag sends a signal to the master process running nginx; telling it to ‘reload’.

As a test, let’s run the First Go app executable and see what loads in a browser for the main domain name (example.com).

Run the First Go app (from the server session terminal window).

$ ./go/first/main

With the First Go app running, point a browser to the app’s domain name, e.g. http://example.com – substitute your own domain name.

In the browser window, the following should be loaded: Hello from the [ First ] Go app.

Ok, that should be working ok, so let’s stop the First Go app by returning to the server terminal window and pressing the CTRL + C keys. Then, let’s also test if the Second and Third apps configurations are running as expected.

Run the Second Go app.

$ ./go/second/main

Visit http://second.example.com – substitute your own domain name – in a browser, we should see:
Hello from the [ Second ] Go app.

In the server session terminal window, press CTRL + C to stop the Second Go app, then repeat a test for the Third Go app.

$ ./go/third/main

Visit http://third.example.com we should see: Hello from the [ Third ] Go app.

Press CTRL + C to stop the Third Go app.

Automatically run our apps when the server boots

We want to run multiple (three) Go apps and, crucially, we need the apps to run automatically without the need for manual intervention. For that to work we will use the Linux systemd facility.

The systemd suite comprises basic building blocks of the Linux system and facilitates a system and service manager from which we will use .service files to define system services, and the command utility systemctl to manage those services.

We are going to create three .service files, one for each of the three Go apps.

Back in the server session terminal window, use Vim to create the First .service file.

$ sudo vi /lib/systemd/system/first-app.service

Enter the following into the .service file as the minimum to define a service:

[Unit]
Description=Go app running under example.com

[Service]
Type=simple
Restart=always
RestartSec=5s
ExecStart=/home/non-root-user/go/first/main
WorkingDirectory=/home/non-root-user/go/first

[Install]
WantedBy=multi-user.target

In the .service file, the service specifics are defined under the [Service] section. Crucial options are:

  • Restart – service shall be restarted after it exits or is killed.
  • ExecStart – The command to run, points to a Go app.
  • WorkingDirectory – The absolute path from where the ExecStart command is to run.

Remember to substitute your own domain name in the Description and non-root user name in the ExecStart and WorkingDirectory paths.

Save the file and repeat for the other two Go apps.

For the Second Go app service…

$ sudo vi /lib/systemd/system/second-app.service
[Unit]
Description=Go app running under second.example.com

[Service]
Type=simple
Restart=always
RestartSec=5s
ExecStart=/home/non-root-user/go/second/main
WorkingDirectory=/home/non-root-user/go/second

[Install]
WantedBy=multi-user.target

… and the Third Go app service…

$ sudo vi /lib/systemd/system/third-app.service
[Unit]
Description=Go app running under third.example.com

[Service]
Type=simple
Restart=always
RestartSec=5s
ExecStart=/home/non-root-user/go/third/main
WorkingDirectory=/home/non-root-user/go/third

[Install]
WantedBy=multi-user.target

Now, we can set each of our three services to start automatically at server boot by enabling them. We use the systemd systemctl command to do that.

$ sudo systemctl enable first-app.service
$ sudo systemctl enable second-app.service
$ sudo systemctl enable third-app.service

If we want to, we can also get our services, and subsequently our apps, to run now with systemctl:

$ sudo systemctl start first-app.service
$ sudo systemctl start second-app.service
$ sudo systemctl start third-app.service

The final test: let’s see if the services we’ve set up, now start automatically after a server reboot:

$ sudo shutdown -r now

The above shutdown command will instruct the server to shutdown and restart -r.

After a few moments, reload the three apps via their domain variants in your browser – they should all be running!

Summary

We have managed to:

  1. Create three Go apps.
  2. Deploy executable binaries of each app to a Digital Ocean server.
  3. Set up an nginx reverse-proxy to route domain specific traffic to each Go app.
  4. Set up system services to make sure the apps start automatically when the server is rebooted.

In the next post I will detail how to serve the Go apps using TLS.

Links to reference articles and tutorials: