Adapting RethinkDB For The Evented Web With Pusher

RethinkDB recently released version 2.0, and here at Pusher we’re all very excited about how creating real-time apps can now be even easier. Changefeeds, a feature introduced by RethinkDB a few versions ago, allows your system to listen to changes in your database. This new version has significantly improved, opening up interesting possibilities for real-time applications.

While RethinkDB covers listening to events on your server, there is still the issue of publishing these changes to your client, who can build anything from news feeds to data visualizations.

This is where a hosted message broker such as Pusher comes into play. By exposing RethinkDB changefeed changes as Pusher events, you can quickly achieve scalable last-mile delivery that instantly pushes database updates to the client. Not only that, but Pusher’s evented publish-subscribe approach fits the logic of real-time applications:

• Channels identify the data, whether that’s a table in a database or, in the case of RethinkDB, a changefeed.
• Events represent what’s happening to the data: new data available, existing data being updated or data being deleted.

As an added bonus, real-time features can be rolled into production fast, in the knowledge that Pusher will scale to millions of concurrent devices and connections, removing the pain of managing your own real-time infrastructure. To show you how this can be done, this post will guide you through how to create the type of activity streams found on the RethinkDB website. By creating a small Sinatra app, we’ll quickly build the JSON feed and high-scores list you can see in our demo. Note that while we are using Ruby and Sinatra, one of the great things about RethinkDB’s adapters is how similar they are across all languages. In other words, what we do here can easily be applied to the stack of your choice.

You can play with the demo here. If you get stuck at any point, feel free to check out the source code.

Step 1: Setting Up

First, if you don’t already have one, sign up for a free Pusher account. Keep your application credentials close by.

If you do not have RethinkDB installed, you can install it on a Mac using Homebrew:

$ brew install rethinkdb

Installation for other operating systems can be found in the documentation.

To start your RethinkDB server, type into the terminal:

$ rethinkdb

Now browse to https://localhost:8080/#tables, which leads to a Web UI where you can create your database. Create a database called game with a table called players.

In your project directory, add the Pusher and RethinkDB gems to your Gemfile, along with Sinatra for our web application:

gem 'pusher'
gem 'rethinkdb'
gem 'sinatra'

Bundle install your gems, and create app.rb for the route-handlers, and rethinkdb.rb for the database configuration.

In rethinkdb.rb, let’s connect to the database. Setup the Pusher instance we’ll need for later using your app credentials. You can get these on your dashboard.

require 'rethinkdb'
include RethinkDB::Shortcuts

require 'pusher'

pusher = Pusher::Client.new({
  app_id: ,
  key: 
  secret: })

$conn = r.connect(
  host: "localhost",
  port: 28015, # the default RethinkDB port
  db: 'game',
)

In app.rb, setup the bare-bones of a Sinatra application:

require 'sinatra'
require './rethinkdb'

get '/' do
  erb :index
end

Step 2: Creating Players

As you can see in the demo, whenever a player enters their name, a game of Snake starts. In the meantime, we want to create a player instance from the name the user has provided.

This demonstration will not go heavily into the HTML and jQuery behind the app. The snake game is based on borrowed code (that you can read up on here) and will detract from the purpose of the tutorial: last mile delivery in a few lines of code. But if you want to dig into it, feel free to check out the source code.

In the case of creating users, we simply want to send an AJAX POST to /players with {name: "the user's name"} to the Sinatra server. At this endpoint, the app runs a simple insert query into the players table:

post '/players' do
  name = params[:name]
  r.table("players").insert(name: name).run($conn) # pass in the connection to `run`
  "Player created!"
end

It’s as simple as that! If you run the app and browse to https://localhost:8080/#dataexplorer, running r.table("game").table("players"), you should see your brand new player document.

While this successfully creates a new player, we’ll probably want our server to remember them for subsequent requests, like submitting scores. We can just amend this endpoint to store the player’s ID in a session. Conveniently, a RethinkDB query response returns the instance’s ID in a "generated_keys" field.

post '/players' do
  name = params[:name]
  response = r.table("players").insert(name: name).run($conn)
  session[:id] = response["generated_keys"][0]
  "Player created!"
end

Step 3: Submitting Players’ Scores

For the purpose of making the demo more fun and interactive, I’ve added two jQuery events to the snake code. One to trigger the start of the game once the player has been created and a second to listen for the end of the game and retrieve the user’s score.

When the game ends, the score is passed to the jQuery event listener. Using this score, make a simple POST to /players/score with the params {score: score}. The route handler gets the player by their session ID and updates their score and high-score, accordingly:

post '/players/score' do
  id = session[:id]
  score = params[:score]

  player = r.table("players").get(id).run($conn) # get the player

  score_update = {score: score} # our update parameters

  if !player["score"] || score > player["high_score"]
    # if the player doesn't have a score yet
    # or if the score is higher than their highest score
    score_update[:high_score] = score
    # add the high-score to the query
  end

  r.table("player").get(id).update(score_update).run($conn) # e.g. .update(score: 94, high_score: 94)
  {success:200}.to_json
end

Now that we have a high_score key for players in our database, we can start rendering a static view of the leaderboard, before we make it realtime. In rethinkdb.rb, let’s build our leaderboard query:

LEADERBOARD = r.table("players").order_by({index: r.desc("high_score")}).limit(5)

In order for this to work, make sure you have created an index called high_score through which to order players. You can do this in your RethinkDB data explorer by running r.db("game").table("players").indexCreate("high_score").

The app needs a /leaderboard endpoint so that renders leaders to the DOM:

get '/leaderboard' do
  leaders = LEADERBOARD.run($conn)
  leaders.to_a.to_json
end

Using jQuery or your preferred Javascript framework, show a static list of the players with the highest scores:

$.get('/leaderboard', function(leaders){
  showLeaderboard(leaders); // showLeaderboard can render leaders in the DOM.
})

Step 4: Make Your Database Real-time

As you can see from the demo, we have two real-time streams involved: a raw JSON feed of live scores, and a live leaderboard. In order to start listening to these changes, we’ll use the RethinkDB gem’s adapter for EventMachine. To the top of rethinkdb.rb, add require 'eventmachine'. Sinatra lists EventMachine as a dependency, so it should already be available within the context of your bundle.

Seeing as we’ve already built our LEADERBOARD query above, let’s dive into how we can listen for changes regarding that query. All that is necessary is to call the changes method on the query, and instead of calling run, call em_run. Once we have the change, all we need is one line of Pusher code to trigger the event to the client.

EventMachine.next_tick do # usually `run` would be sufficient - `next_tick` is to avoid clashes with Sinatra's EM loop
  LEADERBOARD.changes.em_run($conn) do |err, change|
    updated_player = change["new_val"]
   pusher.trigger(“scores”, "new_high_score", update_player)
  end
end

An awesome thing about RethinkDB’s changefeed is that it passes the delta whenever there is a change concerning a query, so you get the old value and the new value. An example of this query, showing the previous and updated instance of the player who has achieved a high score, would be as follows:

{
  "new_val": {
    "high_score": 6 ,
    "id":  "476e4332-68f1-4ae9-b71f-05071b9560a3" ,
    "name":  "thibaut courtois" ,
    "score": 6
  },
  "old_val": {
    "high_score": 2 ,
    "id":  "476e4332-68f1-4ae9-b71f-05071b9560a3" ,
    "name":  "thibaut courtois" ,
    "score": 1
  }
}

In this instance, we just take the new_val of the change – that is, the most recently achieved high-score – and trigger a Pusher event called new_high_score on a channel called scores with that update. You can test that it works by changing a player’s high score, either in your app or the RethinkDB data explorer, then heading to your debug console on https://app.pusher.com to view the newly-created event.

The raw JSON feed of scores shown in our demo is also simple to implement. Let’s just build the query and place it in the same EventMachine block:

LIVE_SCORES = r.table("players").has_fields("score")
# all the players for whom `score` isn't `nil`

EventMachine.next_tick do

  ...

  LIVE_SCORES.changes.em_run($conn) do |err, change|
    updated_player = change["new_val"]
    pusher.trigger(“scores”, "new_score", updated_player)
  end
end

And now, in the browser debug console, whenever somebody’s score has changed, you should see an event like this:

{
  "high_score": 1,
  "id": "97925e44-3e8f-49cd-a34c-90f023a3a8f7",
  "name": "nacer chadli",
  "score": 1
}

Step 5: From DB To DOM

Now that there are Pusher events firing whenever the value of the queries change, we can bind to these events on the client and mutate the DOM accordingly. We simply create a new Pusher instance with our app key, subscribe to the scores channel, and bind callbacks to the events on that channel:

var pusher = new Pusher("YOUR_APP_KEY");

var channel = pusher.subscribe("scores");

channel.bind("new_score", function(player){
  // append `player` to the live JSON feed of scores
});

channel.bind("new_high_score", function(player){
  // append `player` to the leaderboard
});

And there you have it: a simple and efficient way to update clients in real-time whenever a change happens in the database!

Going Forward

Hopefully, we’ve given you insight into RethinkDB’s nice querying language and awesome real-time capabilities. We’ve also shown how Pusher can easily be integrated to work as ‘last-mile delivery’ from changes in your database to your client. Aside from setting up the app itself, there is not much code involved in getting changefeeds up and running.

The demo I have shown you is a fairly small and straightforward. Large, production apps are quite possibly where the benefits of integrating Pusher with RethinkDB are greater felt. RethinkDB allows you to easily scale your database, and Pusher handles the scalability of real-time messaging for you. Combining the two allows developers to build powerful and reliable applications that scale.

Frequently Asked Questions (FAQs) about Adapting RethinkDB for the Evented Web with Pusher