Building a Backend – API (Part 2 of 3)

In this part of the series I am going to build on what we learned in part 1 of the series by creating a Web API. After this tutorial, we will have endpoints that allow us to create and read data from our database.

In this tutorial we are going to be using Node.js to set up a web server. In addition, we’re going to use the Express.js framework to help us quickly layout our routes. We will also be configuring our project to use Typescript. I will also help you set up your project for unit tests using jest.

Install Node.js

If you haven’t done so already, make sure to install Node.js. Node.js is a runtime environment that we’ll be using to create a web server. You can download and install Node.js from this link.

Node.js will come with a package manager called Node Package Manager, or npm for short. You’ll use npm to create projects and install project dependencies.

Creating a Project

To create a project, simply navigate to the directory of your choice. I recommend creating a new one for this project. After doing so, run the following command to initialize our project:

$ npm init

Npm will now ask you to provide details regarding the package you are creating. They’ll ask you for a package name, version, description, entry point, etc. I provided a name for my project and my name as the author. For everything else I used the default value by pressing enter when it asked.

You should now have a package.json file with the information you provided. This file holds critical information about your project such as the information you just provided. In addition, it contains the information to what packages your project is dependent on. In other words, if you ever switch workspaces or hand your project off to someone else to work on, the package.json file should have enough information to help them set up their dependencies so that they can run the project.

Installing Dependencies

Now that we have our package.json file, we can proceed and install our dependencies. This will add our dependencies to the package.json file and to a folder called “node_modules”. When you’re sharing your code with a team via git there is no reason to include the “node_modules” file since anyone should be able to install the required dependencies from the package.json file.

Installing Typescript

Typescript is a superset of the JavaScript language. The main advantage of using Typescript over JavaScript is being able to use static types.

Run the following commands in the same directory as your package.json file to install Typescript:

$ npm install typescript --save-dev
$ npm install tslint --save-dev

The --save-dev flag saves the module we’re installing to our dev dependencies, since we don’t really need these outside of development.

Installing Express

Run the following commands in the same directory as your package.json file to install express:

$ npm install express --save
$ npm install @types/express --save-dev

On the first command we used --save instead of --save-dev. That’s because we are expecting to use the express module outside of our development environment.

Installing pg-promise

pg-promise is a package that helps us interface with our PostgreSQL database.

Run the following commands in the same directory as your package.json file to install pg-promise:

$ npm install pg-promise --save-dev

Install http-status-codes

http-status-codes makes it easy to access http status codes (200, 400, etc.)

Run the following commands in the same directory as your package.json file to install http-status-codes:

$ npm install http-status-codes --save-dev

Install nodemon & ts-node

When we save changes to our code, the changes are not instantly reflected. nodemon is a package that allows us to monitor changes to our source code and restart our server automatically as opposed to having us stop node and starting it again.

Since we are using Typescript, we are going to also use ts-node to run our code without transpiling it to Javascript.

To install both of these packages, run the following commands:

$ npm install nodemon --save-dev
$ npm install ts-node --save-dev

Installing jest

jest is a testing framework that has more than what we need to write our unit tests.

Run the following commands in the same directory as your package.json file to install jest:

$ npm install jest --save-dev
$ npm install @types/jest --save-dev
$ npm install ts-jest --save-dev

Configuring the Project

In this section we are going to configure various elements of our project such as the typescript configuration and behavior when we attempt to start and test our project.

Create tsconfig.json

Create a file called tsconfig.json in the root of our project with the following contents:

{
    "compilerOptions": {
        "module": "commonjs",
        "esModuleInterop": true,
        "target": "es6",
        "moduleResolution": "node",
        "sourceMap": true,
        "outDir": "dist"
    },
    "lib": [
        "es2015"
    ]
}

In the configuration object above we are telling Typescript things like what version of ECMAScript we are targeting.

Configuring Our package.json File

Open our package.json file which is located in the root of our project directory. Then modify “scripts” so it looks like this:

"scripts": {
  "start": "nodemon app.ts",
  "test": "jest"
}

The value in for the “start” key launches nodemon, the utility that will restart our server as soon as we make changes to our code. nodemon will run app.ts, which we will soon create to handle our server’s startup.

Now add the following code after dependencies and don’t forget to add a comma:

"jest": {
  "preset": "ts-jest"
}

The code above defines the preset we want to use with jest, our testing framework. This way jest can work with our Typescript code.

Your package.json file should now look somewhat like this:

{
  "name": "api",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "nodemon app.ts",
    "test": "jest"
  },
  "author": "Andre Yonadam",
  "license": "ISC",
  "devDependencies": {
    "@types/express": "^4.17.6",
    "http-status-codes": "^1.4.0",
    "jest": "^25.4.0",
    "nodemon": "^2.0.3",
    "pg-promise": "^10.5.2",
    "ts-node": "^8.9.1",
    "tslint": "^6.1.2",
    "typescript": "^3.8.3"
  },
  "dependencies": {
    "express": "^4.17.1"
  },
  "jest": {
    "preset": "ts-jest"
  }
}

Creating Your First Endpoint

In this part of the tutorial we’ll go over creating our first endpoint and making sure it works. I will give some insight into how I layout my project structure.

Create app.ts File

As I mentioned when we were configuring our package.json file, nodemon will look for our app.ts file which helps initialize our web service. Our project currently doesn’t actually have an app.ts file so we will need to create it and have some initial code to help us get started. I initialized the app.ts with the following code:

import express from 'express';

const app = express();
const port = 3000;

var http = require('http');
var server = http.createServer(app);

app.use(express.json());
app.use(express.urlencoded({ extended: false }));

server.listen(port);

On the first line we are importing the express framework which we installed earlier in this tutorial. We are also telling express to listen to port 3000.

Initializing pg-promise

Before being able to use pg-promise we need to tell it our database’s information in order for it successfully connect to it.

Creating a new directory in our project, “support”, I created a typescript file inside of that newly created directory called db.ts. This way we can use this module everywhere in our application by importing it as opposed to redefining our pgPromise object and repeatedly telling it our database information.

This is how the db.ts file looks:

import pgPromise from 'pg-promise';
const pgp = pgPromise({ /* Initialization Options */ });
const db = pgp('postgres://user:@localhost:5432/database')

export default db

As you can see, we used “user” and “database” which are the username and database name respectively that we defined in part 1 of this series.

Creating Our First Endpoint

Before jumping into actually writing the code to define an endpoint I’d like to quickly brief how I layout my project for our different endpoints.

Routes are how you access your endpoint. In other words, they appear inside the url you use when making your REST request. For a single route, you can have multiple endpoints. For example, let’s say you have a route defined as “user”. This will show up in your URL as example.com/user. This route can have multiple endpoints. For example, it could have a POST for creating a user, a DEL for deleting a user, a PUT for updating a user, and a GET endpoint for getting a user’s information.

I usually have a directory in my project for my routes. Within that directory, I’ll have a file that defines a module for each route.

To get started, create a directory called “routes” in your project’s root directory.

Within “routes” let’s create a file that will have the code to handle our first endpoint.

I’m going to name this file “hello-world.ts” since I am going to create an endpoint with route “hello-world”. Inside this Typescript file I have the following code:

import HttpStatus from 'http-status-codes';
import express from 'express';
import db from '../support/db';

var helloWorld = express.Router();

/* Create hello world message. */
helloWorld.post('/', function (req, res) {
  let message = req.query.message

  db.one('INSERT INTO hello_world(message) VALUES($1) RETURNING id', [message])
    .then(data => {
      res.status(HttpStatus.CREATED).send({ "id": data.id });
    })
    .catch(error => {
      res.status(HttpStatus.BAD_REQUEST).send();
    });
});

/* Delete hello world message. */
helloWorld.delete('/', function (req, res) {
  let id = req.query.id

  db.result('DELETE FROM hello_world WHERE id = $1', id)
    .then(result => {
      res.status(HttpStatus.NO_CONTENT).send();
    })
    .catch(error => {
      res.status(HttpStatus.BAD_REQUEST).send();
    });
});

/* Update hello world message. */
helloWorld.put('/', function (req, res) {
  let id = req.query.id
  let message = req.query.message

  db.result('UPDATE hello_world SET message = $2 WHERE id = $1', [id, message])
    .then(result => {
      res.status(HttpStatus.NO_CONTENT).send();
    })
    .catch(error => {
      res.status(HttpStatus.INTERNAL_SERVER_ERROR).send();
    });
});

/* Get all hello world messages. */
helloWorld.get('/', function (req, res) {
  db.any('SELECT * FROM hello_world')
    .then(function (data) {
      res.status(HttpStatus.OK).send(data);
    })
    .catch(function (error) {
      res.status(HttpStatus.BAD_REQUEST).send();
    });
});

export default helloWorld

At the top of our file we import three modules. As you can see from the responses within our endpoints, we use HttpStatus to provide a status message back to the client.

We also use express to define the different endpoints on our route. Note that it looks like our route is “/” but we haven’t registered our route in our app.ts file yet. In our app.ts we will import this module we just created and register the path for the route. The examples above demonstrate how to define POST, DEL, PUT and GET endpoints using express.

Last but not least we import our db module that we created to be able to query our database when the user calls one of our endpoints.

To register this route we need to go back to our app.ts file and import the module we created. Then we need to register the module as a route with express. Our new app.ts should look like the following:

import express from 'express';
import helloWorld from './routes/hello-world';

const app = express();
const port = 3000;

var http = require('http');
var server = http.createServer(app);

app.use(express.json());
app.use(express.urlencoded({ extended: false }));

server.listen(port);

app.use('/hello-world', helloWorld);

Running the Project

Our API project is dependent on our database server. Therefore, before starting our API server, we need to make sure our database server is running. To do this run our database server from part 1 of this series.

To check the containers on our system run the following command:

$ docker ps -a

The status column will tell you if your container is running. If it is it should say “Up x hours”. If you don’t specify the -a flag then it will only display the running containers and we won’t be able to get the name of the container we want to start.

If your database container isn’t running and has a status of “Exited”, make note of its name from the Names column (likely “database” if you haven’t changed anything from the previous tutorial) and start it with the following command:

$ docker start CONTAINER_NAME

Now that our backend server is running, navigate to our api project’s root directory and run the following command:

$ npm start

[nodemon] 2.0.3
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: ts,json
[nodemon] starting `ts-node app.ts`

If you see an output like above that means you have successfully started your web server. Nodemon is now watching all your typescript and json files for changes and will restart your server automatically when you make changes to your code so you don’t have to.

Using Our Endpoints

To quickly use our endpoints and see if they’re working we can use a tool like Insomnia or Postman. In this specific tutorial I’m going to use Insomnia.

Unit Testing

Adding Unit tests to our project is good practice and allows us to perform things like test driven-driven development. There might not be many small blocks of code in our project to test since we’re mostly defining endpoints. However, we will still probably create helper functions for things that we do inside the endpoints. We can create unit tests to test the expected output of these helper functions. With test-driven development you’d usually start off by creating the tests and then writing the function to meet the requirements of your test but in this case I’m going to start off with the actual function and then the unit test for purposes of this tutorial.

Let’s say we have the following file, message-guard.ts, in our support directory that sets a default string if the user hasn’t provided one. We will use this function inside our handlers that we provided express for our route wherever we are setting the value for the message column in our database. This way, there will be a default value if the user didn’t pass a value in for the message paramater. Our message-guard.ts is inside our support directory and is the following:

let defaultString = "Default Message"

export function messageGuard(messageGuard: String): String {
    if (messageGuard == null || messageGuard == "") {
        return defaultString
    }
    else {
        return messageGuard
    }
}

Let’s utilize our new function inside our handlers that set the message value inside hello-world.ts. We have to add import { messageGuard } from '../support/message-guard'; to the top and our lines that look like let message = req.query.message should now look like let message = messageGuard(req.query.message as String).

Writing Unit Tests

Create a directory inside of our project called tests. In this directory create a file called message-guard.test.ts.

Add the following code to the file containing two tests:

import { messageGuard } from '../support/message-guard';

describe("Message Guard", () => {
    test("should return the default message", () => {
        let defaultMessage = "Default Message"
        expect(messageGuard(null)).toEqual(defaultMessage);
        expect(messageGuard("")).toEqual(defaultMessage);
    });
});

describe("Message Guard", () => {
    test("should return the same message provided", () => {
        let providedMessage = "My message"
        expect(messageGuard(providedMessage)).toEqual(providedMessage);
    });
});

Running Unit Tests

To run our tests all we have to do is run the command npm test in our project directory. Jest will automatically find out test files and will run our tests.

Dockerizing

Create a named Dockerfile with the following contents:

FROM node:10

WORKDIR /usr/src/app

COPY package*.json ./

RUN npm install

COPY . .

EXPOSE 3000
CMD [ "npm", "start" ]

In the code above we’re choosing the official node image as our base image. We’re also specifying to use version 10. Then we create our working directory. We then copy our package.json file and install our dependencies. Note that the current configuration is going to assume a development environment, so all the developer dependencies will be also installed. Then we copy all of our files and expose port 3000. Then we run our app.ts file which initializes our web service.

Before we’re able build and run the image, we want it to be able to talk to the container that’s running our database. We were able to connect directly to our database from our host machine since we set port 5432 to be mapped and exposed to port 5432 on our host machine. However, when using one of our endpoints in our Docker container, it won’t be able to connect to port 5432 because it isn’t available on its localhost. To solve this issue we must create a Docker network. We are going to give our database a static ip of 172.18.0.11. So go ahead and change localhost inside our db.ts file to that IP address. We can now build our docker image:

$ docker build -t api .

Let’s create a network so both of our containers can communicate with each other.

$ docker network create --subnet=172.18.0.0/16 api-database

We can now see our newly created network when we run the command:

$ docker network list

Go ahead and connect our database container to our network and assign it an IP address value with the following command:

$ docker network connect api-database --ip 172.18.0.11 database

Then we can finally run our image for our api using the following command which connects it to the network and exposes port 3000 to our localhost.

$ docker run --name api -d -p 3000:3000 --network api-database api

Conclusion

I’m glad you’ve made it this far in the series. You now know how to create an API that has tests and runs within a networked Docker container. The code is available in this GitHub repo. One thing to note is that the database address was changed from localhost to 172.18.0.11 as documented in our “Dockerizing” process. This being said you would need to change it back to localhost if you were running the api project directly using npm start on the host machine and the database not being on a Docker network. You can still connect to 172.18.0.11 from your host machine, so you can run the api project locally without docker and have the database running on the Docker network.

Stay tuned for the third and last part of this series!