BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles Build, Test, and Deploy Scalable REST APIs in Go

Build, Test, and Deploy Scalable REST APIs in Go

Bookmarks

Key Takeaways

  • Go is a statically typed open-source language that can be used for server-side programming, cloud-based development, command-line tool development, and game development.
  • Continuous integration and deployment (CI/CD) enables organisations to release updates on a regular basis while maintaining quality.
  • It is easier to test smaller changes using CI/CD before they are merged into the main application. This, as a result, encourages testability.
  • Failures are easier to detect and resolve with CI/CD. 
  • Testing helps find error points in your application.

Introduction

In this article, we'll look at how to use the gin framework to create a simple Golang application. We will also learn how to use CircleCI, a continuous deployment tool, to automate testing and deployment.

Go is a statically typed open-source programming language created by Google engineers with the sole purpose of simplifying complex software development processes and architectures. Its key features include high-performance networking, concurrency and  ease of use. Goroutines are also extensively used in Go. A Goroutine is a function that runs alongside other goroutines in a program. Goroutines are useful when you need to do several things at once. Google, Cloudflare, MongoDB, Netflix, and Uber are a few companies that use Go.

Gin is a Go-based high-performance HTTP web framework that can be used to build microservices and web applications. The main advantage of Gin is that it allows developers to create scalable and efficient applications without having to write a lot of boilerplate code. It produces code that is easy to read and concise. It includes built-in routing, middleware for handling functionality, a logger, and a web server.

Building a simple CRUD API

We'll create a simple API for a student management tool for a departmental club. The club president will have the ability to add new students and retrieve all students. To fully follow this tutorial, we will also require the following:

  1. Go installed, as well as a working knowledge of the language.
  2. Knowledge of tests and how to write them.
  3. An account on GitHub.
  4. An account on CircleCI.
  5. An Heroku account.

Please keep in mind that if you wanted to try this on a free tier account on Heroku, Heroku will be discontinuing its free tier plan soon. However, the procedures described here can be easily applied to the majority of other cloud hosting platforms.

Building the CRUD Application

Our simple departmental club API will only have two functionalities: adding students as members and viewing all members; nothing complicated! These will be POST and GET requests. We will not connect to any database such as MongoDB or MySQL. We will, however, use local storage and create a default student in the database. This student is automatically added whenever we restart the server.

Let's get this party started. First, we'll make a folder for our project. Let's call it stud-api. Within that folder, we will initialise our golang program and install all of the dependencies that we will need.

mkdir stud-api
cd stud-api

Next we will initialise our go.mod file and install all the needed dependencies.

go mod init stud-api
cd stud-api
go get -u github.com/gin-gonic/gin github.com/rs/xid github.com/stretchr/testify 

Github.com/rs/xid is a library for creating unique identifiers. We will use it  in this project to automatically generate IDs for each new student. The github.com/stretchr/testify package will be used to test our various endpoints.

Let's get started on the API. For the sake of simplicity, we will only create one file called main.go. This file will contain our struct, as well as our API controllers, services, and routes. We will create three endpoints:

  1. A welcome function that sends a greeting message.
  2. A CreateStudent() function for adding a student to the database.
  3. A GetStudents() function that returns all of the database's registered students.

We will import three packages into our newly created main.go file: an HTTP package, the xID package, and the gin package. Next, we'll write a main() function that will contain all of our API routes. Then we'll make another function, which we'll call WelcomeMessage(). It will contain a simple message to be printed when we call the associated route.

package main
import (
	"net/http"
	"github.com/gin-gonic/gin"
	"github.com/rs/xid"
)

func main() {
	//setting up routes
	router := gin.Default()
	router.GET("/", WelcomeMessage)
	router.Run()
}
//Welcome Message
func WelcomeMessage(c *gin.Context) {
	c.JSON(http.StatusOK, gin.H{"message": "Hey boss!"})
}

To see what we have done so far, we are going to spin up our server using the command below:

go run main.go

When it runs successfully, the CLI will display the message "Hey boss!".  Now that we've created that simple function, let's move on to our database and struct.

We'll build a simple Student struct that takes three parameters: the student's name, department, and level, and then generates an ID for the user when it's successfully added to the database.

//Setting up student struct
type Student struct {
	ID         string `json:"id"`
	Name       string `json:"name"`
	Department string `json:"department"`
	Level      string `json:"level"`
}

Let us now create our local database, that will be expecting the three values we are passing to the server as well as the generated ID. Our database will be called Students and will contain default data for one student, while any new students we create will simply be added to it.

//Students database
var students = []Student{
	{
		ID:         "10000xbcd3",
		Name:       "Alicia Winds",
		Department: "Political Science",
		Level:      "Year 3",
	},
}

Now that this is in place, let's write the CreateStudent() function and the route that will interact with it.

//Create a new student account
func CreateStudent() gin.HandlerFunc {
	return func(c *gin.Context) {
		var newStudent Student
		if err := c.BindJSON(&newStudent); err != nil {
			c.JSON(http.StatusBadRequest, gin.H{
				"Status":  http.StatusBadRequest,
				"Message": "error",
				"Data":    map[string]interface{}{"data": err.Error()}})
			return
		}
		//Generate a student ID
		newStudent.ID = xid.New().String()
		students = append(students, newStudent)
		c.JSON(http.StatusCreated, newStudent)
	}

}

Now we will add the route needed to communicate with the function to our main() function.

func main() {
	-------------
	router.POST("/createStudent", CreateStudent())
	-------------
}

To test what we have done so far, if we spin up our server, and test our endpoint (localhost:8080/createStudent) in Postman or any other environment,  passing the name, department, and level in the body, a new user will be generated automatically with a unique ID. Please note that this is a non-persistent database.  

We now need to create our last function which we will use to get all the students in the club database. This request is a simple GET function, which will search the students database and return all of its content.

func GetStudents() gin.HandlerFunc {
	return func(c *gin.Context) {
		//Fetch all students in the DB
		c.JSON(http.StatusOK, students)
	}
}

Finally, we'll create the route that will interact with our newly created function. It will be added to our main function, along with the other routes.

func main() {
	------------------
	router.GET("/students", GetStudents())
	router.Run()
}

Let's also test this one on postman! To accomplish this, we will need to launch our server and access this endpoint: localhost:8080/students. All we need to do is use the HTTP verb GET. There is no need to include any body or query parameters. It will return all of the students in the database after a successful run. And with that, our simple CRUD API is finished!

Writing Simple Local Tests

In this section, we will run unit tests on the endpoints we have created. The goal is to ensure that each function works as expected. To test these functions, we will use the testify package. In addition, we must create a new file called new_test.go. This file will contain the various tests that we will write. We'll need to import a few packages after we've created the new file in our root in our main directory.

func main() {
	------------------
	router.GET("/students", GetStudents())
	router.Run()
}

Testify makes it easy to perform simple assertions and mocking. The testing.T object is passed as the first argument to the assert function in go. The assert function then returns a bool indicating whether the assertion was successful or not. The testify mock package offers a method for quickly creating mock objects that can be substituted for actual objects when writing test code.

Now we will set up a router and write a simple test for our welcome message. The assert function in the simple welcome message test below will use the equality variant to determine whether the test argument matches the mock response.

func SetRouter() *gin.Engine {
	router := gin.Default()
	return router
}

func TestWelcomeMessage(t *testing.T) {
	mockResponse := `{"message":"Hey boss!"}`
	r := SetRouter()
	r.GET("/", WelcomeMessage)
	req, _ := http.NewRequest("GET", "/", nil)
	w := httptest.NewRecorder()
	r.ServeHTTP(w, req)
	responseData, _ := ioutil.ReadAll(w.Body)
	assert.Equal(t, mockResponse, string(responseData))
	assert.Equal(t, http.StatusOK, w.Code)
}

Next, we'll write a simple test for the createStudent() function using mock data. We will continue to use the xID package to generate the Student ID, and we will receive a bool indicating whether or not the test was successful.

func TestCreateStudent(t *testing.T) {
	r := SetRouter()
	r.POST("/createStudent", CreateStudent())
	studentId := xid.New().String()
	student := Student{
		ID:         studentId,
		Name:       "Greg Winds",
		Department: "Political Science",
		Level:      "Year 4",}
	jsonValue, _ := json.Marshal(student)
	req, _ := http.NewRequest("POST", "/createStudent", bytes.NewBuffer(jsonValue))
	w := httptest.NewRecorder()
	r.ServeHTTP(w, req)
	assert.Equal(t, http.StatusCreated, w.Code)}

 Finally, we'll write our final test for the GetStudents() function. 

func TestGetStudents(t *testing.T) {
	r := SetRouter()
	r.GET("/students", GetStudents())
	req, _ := http.NewRequest("GET", "/students", nil)
	w := httptest.NewRecorder()
	r.ServeHTTP(w, req)
	var students []Student
	json.Unmarshal(w.Body.Bytes(), &students)
	assert.Equal(t, http.StatusOK, w.Code)
	assert.NotEmpty(t, students)
}

We have completed all of the tests and can now run them locally. To do so, simply run the following command:

GIN_MODE=release go test -v

And here’s our final result:

Automating Tests with Continuous Development

CircleCI is a platform for continuous integration and delivery that can be used to integrate DevOps practices. We will use this CI/CD tool in this article to automate our tests and deploy our code to our server but let's start by automating tests with CircleCI first.

Make sure you have a CircleCI account, as specified in the prerequisites, and that you have successfully pushed the code to GitHub. Check your CircleCI dashboard to ensure that the project repository is visible.

Now, in the project directory, we need to create a .circleci folder and a config.yml file, which will contain the commands needed to automate the tests.

Setting up the config.yaml

This file contains all of the configurations required to automate Heroku deployment and testing. For the time being, we won't be focusing on the Heroku section because we're more interested in the code that helps automate our tests. The file contains the Go orb and Jobs that checkout and run the test. We'll need to re-push to GitHub after adding the code below to our config file.

workflows:
  heroku_deploy:
    jobs:
      - build
      - heroku/deploy-via-git:  
          requires:
            - build
          filters:
            branches:
              only: main
jobs:
  build:
    working_directory: ~/repo
    docker:
      - image: cimg/go:1.17.10
    steps:
      - checkout
      - restore_cache:
          keys:
            - go-mod-v4-{{ checksum "go.sum" }}
      - run:
          name: Install Dependencies
          command: go get ./...
      - save_cache:
          key: go-mod-v4-{{ checksum "go.sum" }}
          paths:
            - "/go/pkg/mod"
      - run:
          name: Run tests
          command: go test -v

After this step, we can return to our CircleCI dashboard and choose our project. Then we need to click the Setup button next to it and select the branch we're working on. When we click the Setup button, our program will begin to run. A successful build would look like the image below (when we scroll down to the run tests section)

That's it! We were able to successfully build a simple API, create local tests, and automate the testing process. The entire automation process here means that the pipeline attempts to run the test every time a push is made to that branch on the github repository.

Automating Deployment to Heroku using circleCI

First, we must configure Heroku. If you do not already have a Heroku account, you will need to create one and connect your GitHub profile to your Heroku account for easy deployment and automation. Once that is completed, we will need to create a Procfile (yeah, with no extensions) within our project folder. Within the Procfile we will add the following:

web: app

After we do that we will make a push to GitHub. Now let’s take a quick look at the config.yaml file we created earlier. We can now analyse the first section. We can see that we imported the Heroku orb and we have a workflow that contains a job that builds and deploy the code in the main repository. 

Returning to our Heroku dashboard, we must first create a project on Heroku and obtain our API keys, which can be found in the account settings. We Will need to add this key to our CircleCI project. To do this, navigate to our existing project on CircleCI and select Project Settings. Then go to the environment variables section and add two things:

  1. HEROKU_APP_NAME,  with the value stud-api (the name of our application)
  2. HEROKU_API_KEY with the key we just obtained from Heroku.

We have successfully configured our circleci project for continuous deployment to Heroku. If this was successful, we should see a successful build message on our CircleCI dashboard. Something like this:

To see what we've done, we can return to our Heroku dashboard and retrieve our project URL. This is the URL in this case: https://stud-app-api.herokuapp.com/. All of the functionalities can be tested by appending the routes you want to test to the end of the url. For example testing the endpoint that fetches all students:

Conclusion

Continuous development enables developers to create better and faster products. Continuous integration and development tools have simplified the entire process by automating operations that would otherwise have required more time or expertise. CI/CD tools have gradually helped to improve product quality by automating everything from testing to quick application deployment.

About the Author

Rate this Article

Adoption
Style

BT