Creating a REST API in Rust with Persistence: Rust, Rocket and Diesel

Photo by John Baker on Unsplash

Introduction

We will be creating a REST API in Rust, which serves resources (posts) with persistence in a postgres database.

Rocket framework will be used to setup the API and Diesel to handle persistence with a Postgres database.

We will test our Rust API and a Postgres database by running them as separate docker containers which communicate with each other to expose CRUD functionalities from our API and implement persistence with a Postgres database.

I’ve provided the source code in the rocket-diesel branch of my Github repo.

genekuo/rust-microservices

Dependencies: Cargo.toml

First, we need to setup the following dependencies for our REST API project:

[dependencies]
rocket = "0.4.5"
rocket_codegen = "0.4.5"
diesel = { version = "1.4.4", features = ["postgres"] }
dotenv = "0.15.0"
r2d2-diesel = "1.0.0"
r2d2 = "0.8.9"
serde = "1.0.116"
serde_derive = "1.0.116"
serde_json = "1.0.58"
env_logger = "0.5.12"
log = "0.4.6"
[dependencies.rocket_contrib]
version = "*"
default-features = false
features = ["json"]

Let’s focus on several important crates in the following.

rocket crate: this crate provides Rocket as a web framework for Rust with a focus on ease-of-use, expressibility, and speed. Rocket requires a nightly version of Rust as it makes heavy use of syntax extensions.

rocket_codegen: this crate allows us to use the implementation of code generation of Rocket, which includes custom derives, custom attributes, and procedural macros.

diesel: this crate provides a safe, extensible ORM and Query Builder for Rust. We specify postgres feature to include only the postgres module in the Diesel crate. If different or multiple databases are needed in our project, we can just specify them in the features list.

r2d2 and r2d2-diesel: r2d2 is a generic connection pool for Rust. r2d2-diesel provides r2d2 support to allow connection pooling with Diesel. r2d2 and r2d2-diesel are included for using the diesel crate with r2d2 pools.

serde, serde_derive, serde_json: these crates helps us to make structs serializable and deserializable with different formats, such as JSON, CBOR, MessagePack, and BSON.

Diesel

Diesel crate helps generating Rust types that representing tables and records in a SQL database and creating a domain-specific language to query data from a database.

At the time of writing this article, the only databases that Diesel supports are Postgres, MySql and Sqlite.

Generating and applying migrations: up.sql and down.sql

We will need diesel_cli to create migrations and apply them. For that, we need to use the following command to install diesel_cli.

cargo install diesel_cli --no-default-features --features "postgres"

We can then prepare the necessary migrations.

diesel setup

Since we are going to model posts which can be inserted, retrieved, updated and deleted from the database, we can create a migration directory containing two files: up.sql and down.sql.

diesel migration generate create_posts

The above command creates two new files: up.sql and down.sql in a single folder in the migrations directory. We can then modify up.sql and down.sql with the following SQL statements for upgrading and downgrading migrations in the database if necessary.

up.sql

CREATE TABLE posts (
  id SERIAL PRIMARY KEY,
  title VARCHAR NOT NULL,
  body TEXT NOT NULL,
  published BOOLEAN NOT NULL DEFAULT 'f'
)

down.sql

DROP TABLE posts

We can apply all the upgrading and downgrading migrations with the following command to create the database and the posts table.

DATABASE_URL=postgres://username:password@localhost/postgres diesel migration run

Or, with the .env file specify DATABASE_URL environment variable, we can just run the following instead.

diesel migration run

schema.rs

We can use the following command to generate a Rust schema file which uses table! macro to implement the database mappings for us.

diesel print-schema > src/schema.rs

schema.rs

table! {
    posts (id) {
        id -> Int4,
        title -> Varchar,
        body -> Text,
        published -> Bool,
    }
}

Object-relational Mapping (ORM): model.rs

After we have a posts table in a Postgres database, we can start mapping the table to structs in Rust.

#[derive(Queryable, AsChangeset, Serialize, Deserialize, Debug)]
#[table_name = "posts"]
pub struct Post {
    pub id: i32,
    pub title: String,
    pub body: String,
    pub published: bool,
}
#[derive(Insertable, Serialize, Deserialize)]
#[table_name="posts"]
pub struct NewPost {
    pub title: String,
    pub body: String,
}

#[table_name = “posts”] on Post struct maps the struct to the table posts from the database. Therefore, an instance of the Post struct represents each record in the posts table.

The Queryable, AsChangeset attributes on Post struct instruct Diesel to generate code for retrieving posts and executing updates on posts table from the database.

The Serialize and Deserialize use serde crate to transform between instances of Post struct and JSON format between REST API requests and responses.

We also define a NewPost struct without the id field to serve the purpose of inserting a post record into the database. The id of the record will be generated automatically when inserted, and we could specify exactly the fields that can be filled during insertion.

The Insertable attribute on NewPost is used by diesel crate to generate code for inserting a new record.

Connection Pool: connection.rs

Before wen can make use of Diesel ORM to persist Rust struct instances in the database, we have to create a module for the connection pool and the connection request guard. The connection pool will be configured with Rocket router (router.rs) as a Rocket managed state.

DBConn connection request guard can set a policy to retrieve a PgConnection from the managed pool and then pass it to handlers (handler.rs), and subsequently pass it to repository (repository.rs) for the database querying and updating executions.

type Pool = r2d2::Pool<ConnectionManager<PgConnection>>;
pub fn init_pool() -> Pool {
  let manager = ConnectionManager::<PgConnection>::new(database_url());
    Pool::new(manager).expect("db pool")
}
fn database_url() -> String {
    env::var("DATABASE_URL").expect("DATABASE_URL must be set")
}
pub struct DbConn(pub
  r2d2::PooledConnection<ConnectionManager<PgConnection>>);
impl<'a, 'r> FromRequest<'a, 'r> for DbConn {
    type Error = ();
  fn from_request(request: &'a Request<'r>) ->   request::Outcome<DbConn, Self::Error> {
        let pool = request.guard::<State<Pool>>()?;
        match pool.get() {
            Ok(conn) => Outcome::Success(DbConn(conn)),
            Err(_) => Outcome::Failure((Status::ServiceUnavailable, ())),
        }
    }
}
impl Deref for DbConn {
  type Target = PgConnection;
  fn deref(&self) -> &Self::Target {
        &self.0
    }
}

Here we define an alias Pool to the type for a pool of Diesel Postgres connection.

The init_pool function will initialize the connection pool by using r2d2 and r2d2_diesel crates.

We will later configure the pooled connections in router.rs as managed state in Rocket framework.

We then define the connection request guard type DbConn which is a wrapper around an r2d2 pooled connection. According to Rocket documentation: https://api.rocket.rs/v0.4/rocket/request/trait.FromRequest.html, a request guard is a type that represents an arbitrary validation policy which is implemented through FromRequest trait. Every type that implements FromRequest is a request guard.

Therefore, we can implements FromRequest for DbConn to define a policy for the attempts to retrieve a single connection from the managed database pool.

We can define an arbitrary number of request guards to use as arguments in a route handler (handler.rs). Rocket will automatically invoke the FromRequest implementation for request guards before calling the handler. And, Rocket only dispatches requests to a handler when all its guards pass. In our case, an available connection from the pool is retrieved.

We also implements Deref trait for DbConn for the convenience of using an &*PgConnection when we want to get the actual connection.

Repository: repository.rs

With the table created and structs mapped to it in the previous sections, we can now implement CRUD functionalities against the database in the repository module.

#![allow(proc_macro_derive_resolution_fallback)]
use diesel;
use diesel::prelude::*;
use crate::sample::model::Post;
use crate::sample::model::NewPost;
use crate::schema::posts;
use crate::schema::posts::dsl::*;
pub fn create_post(new_post: NewPost, conn: &PgConnection) -> QueryResult<Post> {
    diesel::insert_into(posts::table)
        .values(&new_post)
        .get_result(conn)
}
pub fn show_posts(connection: &PgConnection) -> QueryResult<Vec<Post>>  {
    //posts.filter(published.eq(true))
    posts.limit(5)
        .load::<Post>(&*connection)
}
pub fn get_post(post_id: i32, connection: &PgConnection) -> QueryResult<Post> {
    posts::table.find(post_id).get_result::<Post>(connection)
}
pub fn update_post(post_id: i32, post: Post, connection: &PgConnection) -> QueryResult<Post> {
    diesel::update(posts::table.find(post_id))
        .set(&post)
        .get_result(connection)
}
pub fn delete_post(post_id: i32, connection: &PgConnection) -> QueryResult<usize> {
    diesel::delete(posts::table.find(post_id))
        .execute(connection)
}

The diesel module allow us to use various functions such as insert_into, update, and delete.

diesel::prelude::* provides access to a range of modules and structs such as PgConnection and QueryResult .

schema::posts and schema::posts::dsl::* are for access to the posts table from within Rust and for bringing in all the available Diesel DSL methods, so we can interact with the database tables.

For example, get_post allow us to access the posts table via posts::table. We can then use find function with the specified post_id and call get_result to execute the query with the provided connection.

Another examples show different usages: create_post , update_post, which we can compose diesel::insert_into or diesel::update functions with posts::table and/or its functions to execute inserting and updating a record.

QueryResult is returned from all functions we define here. It is an alias type for Result<T, Error> . Therefore, returning QueryResult allows us to indicate what happens if the query fails in whatever way is suitable for where the function is used.

Rocket

Rocket is a web framework that uses nightly compiler’s features that converts a set of Rust functions called handlers into a complete web services.

The following lines of code import macros from the crates we need to use the features of connection pooling with Diesel, data serialization and deserialization for requests and responses, handlers declaration and logging.

#![feature(decl_macro, proc_macro_hygiene)]
#[macro_use]
extern crate diesel;
extern crate dotenv;
extern crate r2d2;
extern crate r2d2_diesel;
#[macro_use]
extern crate rocket;
extern crate rocket_contrib;
#[macro_use]
extern crate serde_derive;

The two features from the nightly release: decl_macro, proc_macro_hygiene, allow us to declare handlers to handle requests from clients.

Handlers: handler.rs

A handler in Rocket is a function that takes parameter bindings, handle the mapped request and return the result in JSON format.

Each function has an attribute that specifies what REST verb of GET, POST, PUT and DELETE it accepts and the path needed to get to the function.

Each handler delegates to the repository module (repository.rs) we added here, with the database connection: PgConnection to perform queries
and updates on the Postgres database.

For example, the function update_post accepts an input parameter id , binding the </id> from the request, then delegating to repository::update_post function to update a specific post with the same id in the database.

The </id> represents the path variable id specified in the request URL. The data = “<post>” represents the request body that maps to post inside the function.

The contents of post is retrieved by calling post.into_inner() and pass to repository::update_post function.

If the operation is successful, it will wrap the updated post as an Json instance, which is wrapped into the Result type.

If the operation is failed, the error will be transformed into Status and wrapped into Result.

The Rocker framework will handle the result from this function to respond to the client with either successful records or a specific status code when failure.

#[get("/")]
pub fn all_posts(connection: DbConn) -> Result<Json<Vec<Post>>, Status> {
    sample::repository::show_posts(&connection)
        .map(|post| Json(post))
        .map_err(|error| error_status(error))
}
#[post("/", format ="application/json", data = "<new_post>")]
pub fn create_post(new_post: Json<NewPost>, connection: DbConn) ->  Result<status::Created<Json<Post>>, Status> {
    println!("here 0 {}",&new_post.title);
    sample::repository::create_post(new_post.into_inner(), &connection)
        .map(|post| post_created(post))
        .map_err(|error| error_status(error))
}
#[get("/<id>")]
pub fn get_post(id: i32, connection: DbConn) -> Result<Json<Post>, Status> {
    sample::repository::get_post(id, &connection)
        .map(|post| Json(post))
        .map_err(|error| error_status(error))
}
#[put("/<id>", format = "application/json", data = "<post>")]
pub fn update_post(id: i32, post: Json<Post>, connection: DbConn) -> Result<Json<Post>, Status> {
    sample::repository::update_post(id, post.into_inner(), &connection)
        .map(|post| Json(post))
        .map_err(|error| error_status(error))
}
#[delete("/<id>")]
pub fn delete_post(id: i32, connection: DbConn) -> Result<status::NoContent, Status> {
    sample::repository::delete_post(id, &connection)
        .map(|_| status::NoContent)
        .map_err(|error| error_status(error))
}

Router: router.rs

After we setup handlers to accept requests to the server, we can then map the routes to the different functions in the handler module.

use rocket;
use crate::connection;
use crate::sample;
pub fn create_routes() {
    rocket::ignite()
        .manage(connection::init_pool())
        .mount("/posts",
               routes![
                    sample::handler::all_posts,
                    sample::handler::create_post,
                    sample::handler::get_post,
                    sample::handler::update_post,
                    sample::handler::delete_post
                    ],
        ).launch();
}

We create a Rocket instance with the ignite method. Before we launch it, we have to add state which is Pool to be managed by this instance of Rocket.

Managed state can be retrieved by any request handler via the State request guard, which is via from_request function of the DbConn .

In particular, if a value of type T is managed by Rocket, adding State<T> , in our case DbConn , to the list of arguments in a request handler instructs Rocket to retrieve the managed value.

Finally, we specify all of the handler functions inside of routes!macro and then start our REST server by calling launch.

Environment variables and main function: main.rs

In main.rs, we simply load environment variables and call router::create_routes function to start the REST server.

The environment variables we will use are:

RUST_BACKTRACE=1
RUST_LOG=debug
ROCKET_ADDRESS=0.0.0.0
ROCKET_PORT=8000
DATABASE_URL=postgres://username:password@db/postgres

Environment variables include Rust, Rocket, and database related configuration variables as the above. They can be specified in .env file or Rocket.toml file in the project’s root directory if needed.

Since we will containerize our REST services and Postgres database, we will make use of docker-compose file to specify these environment variables and inject them into containers, shown in the following section.

Building images and Testing

When we are in the middle of developing our REST services, we can use the following commands to run and test the services.

First, we will run a Postgres database container.

docker run --rm --detach --name postgres --env POSTGRES_USER=username --env POSTGRES_PASSWORD=password --publish 127.0.0.1:5432:5432 postgres

We will then install diesel-cli and run the following Diesel-related commands mentioned previously.

diesel setup
diesel migration generate create_posts
// modify up.sql and down.sql as needed
diesel migration run

After coding ORM, repository, handlers and routers, we run cargo run to start the services and use curl to test them.

cargo run
curl -d '{"title": "Rust microservices", "body": "Using Diesel to implement persistence"}' -H "Content-Type: application/json" -X POST http://localhost:8000/posts
curl http://localhost:8000/posts

We also provide a Dockerfile in my GitHub repo for two-stage containers build to create a build image (for compilation) and a runtime image to reduce the size of the runtime image.

Also, we use a docker-compose.yml to run our REST services from the runtime image and Postgres database from postgres:latest image, capture the environment configurations and link them to simulate a simple microservices deployment in our local development.

Finally, we can confirm the functionalities of our REST services and persistence using the following commands.

docker-compose up -d
diesel migration run
curl -d '{"title": "Rust microservices", "body": "Using Diesel to implement persistence"}' -H "Content-Type: application/json" -X POST http://localhost:8000/posts
curl -d '{"title": "Rust microservices", "body": "Using Rocket to build a API"}' -H "Content-Type: application/json" -X POST http://localhost:8000/posts
curl http://localhost:8000/posts/1
curl -d '{"id":1, "title": "Rust microservices", "body": "Using Diesel and Rocket to build microservices", "published": true}' -H "Content-Type: application/json" -X PUT http://localhost:8000/posts/1
curl -X DELETE http://localhost:8000/posts/2
curl http://localhost:8000/posts
docker-compose down

Conclusions

Combining Rocket web framework and Diesel ORM in Rust can be a powerful way to create a REST API with persistence on a SQL database.

The Rocket framework allows us to write request handlers in an intuitive and clear style with the help of the nightly Rust compiler.

Interaction to databases with Diesel object-relation mapping features, its generated domain-specific language and composable abstractions simplify our Rust code for complex queries.

We can further explore their features and usages to create more production-grade applications or APIs, such API security, complex queries with multiple tables or schemas, applying more microservices patterns and so on.

Thanks for reading.