Building a Rails dashboard app with the DoneDone API

Last week, I talked about my experiences using DoneDone to manage household issues. To accompany my post, I wrote a simple Rails application to query data from the DoneDone API and display it on a publicly-accessible dashboard. Today, I'd like to walk through the process of building that application from the ground up.

If you'd like to see the end result, you can see the source code on GitHub. Otherwise, follow along to recreate the demo step-by-step.

Setting up your environment

First, make sure you have Ruby, RubyGems, and Rails installed. You might be interested in using RailsInstaller or RVM to simplify this process on your system.

For this demo, I'm using Ruby 2.1.1 and Rails 4.2.0.

Creating a new Rails project

Using your Terminal or Command Prompt, navigate to the directory where you'd like to store your project, then run:

rails new donedone-public-dashboard -T

Here, we're indicating that we don't want Rails to install a test framework inside our app with the -T modifier. This is simply to keep our directory a bit cleaner, as we won't be writing tests for this short demo.

Next, switch into your new donedone-public-dashboard directory for the remainder of the demo.

Creating models

Although we'll be retrieving all of our data from the DoneDone API, we don't want to do this every time a user visits our dashboard. Instead, we want to download all our data from DoneDone behind the scenes and store it in our local database. Our application will then display the local data, saving us from making unnecessary requests from DoneDone.

But first, we'll need to create models to hold all this data. Our models will closely follow the structure of the data that's returned by the DoneDone API methods. Enter this in your console:

rails generate model Project project_id:integer title:string

This command creates a database table named projects with two columns: project_id (which stores integers) and title (which stores strings). This also creates an ActiveRecord class named Project, which you can find in /app/models/project.rb. This class allows you to interact with the database records very easily using Ruby code.

Note: Why didn't we simply call the first column id? Well, Rails automatically gives every database table a column named id, which it uses to internally lookup records. But, since the projects we retrieve from DoneDone will have their own ID values, we need another column to keep track of these. You can think of the id column as the local ID, and the project_id column as the DoneDone ID.

Now let's create a model to store issues:

rails g model Issue project:references order_number:integer title description created_on:datetime last_updated_on:datetime last_updater tester fixer creator priority status due_date:date

Here, we've used a few shortcuts to make the command less wordy. The generate keyword can be shortened to g, and any columns that don't specify a data type will automatically be created as string types.

Notice that the project column uses a references type - this will create a foreign key relationship between projects (the parent table) and issues (the child table).

Finally, let's generate a model definition for issue history data, so we can view how each issue has changed over time.

rails g model History issue:references history_id:integer created_on:datetime action description creator

To recap, each Project can have multiple Issues, and each Issue can have multiple Histories.

Now that we've finished our model generation, we need to actually create these tables in our database. Each time we've used the rails generate model command, Rails has created a small piece of database migration code, which you can view in the /db/migrate directory. We need to run all these migration methods to synchronize our schema to our database:

rake db:migrate

Improving relationships

Now that our models have been created (both in code and in our database), we need to give Rails a bit more information about how our models are related to each other.

We used the references data type to indicate that some of our models are related to others. But, we need to specify exactly how these relationships should be enforced. Open /app/models/issue.rb and you'll see this:

class Issue < ActiveRecord::Base
belongs_to :project
end

The belongs_to line indicates that each Issue record belongs to a Project record. But we still need to tell Rails that each Issue can have its own History records. Change your model declaration to look like this:

class Issue < ActiveRecord::Base
belongs_to :project
has_many :histories
end

Now Rails knows each Issue belongs to a Project, but can also have many History items.

Update the remaining models with their appropriate relationships:

/app/models/history.rb:

class History < ActiveRecord::Base
belongs_to :issue
end

/app/models/project.rb:

class Project < ActiveRecord::Base
has_many :issues
end

Connecting to the DoneDone API

Now that our models and relationships are ready, we can retrieve remote data from DoneDone and populate our local database. To get our data, we'll be using a Ruby gem called Faraday.

Open /Gemfile and add the following line to the bottom of the file:

gem 'faraday'

Save your Gemfile, then return to your console window and run this command:

bundle install

This will download the faraday gem and install it for use with your app. Faraday gives you an easy way to interact with remote APIs, just like DoneDone's REST API.

Next, we need to define a Ruby class that will retrieve data from the DoneDone API and store it in our local database. Create a new file named /app/services/done_done_api.rb. Place the following code in the new file:

class DoneDoneApi
 def self.conn
   connection = Faraday.new(:url => 'DONEDONE_API_BASE_URL')
   connection.basic_auth('DONEDONE_API_USERNAME', 'DONEDONE_API_TOKEN')
   connection
 end
 def self.sync_projects
   response = conn.get '/issuetracker/api/v2/projects.json'
   if response.body
     api_projects = JSON.parse(response.body)
     api_projects.each do |api_project|
       p = Project.find_or_initialize_by(project_id: api_project['id'])
       p.title = api_project['title']
       p.save
     end
   end
 end
 def self.sync_issues
   issues_response = conn.get '/issuetracker/api/v2/issues/all.json'
   if issues_response.body
     api_issues = JSON.parse(issues_response.body)['issues']
     api_issues.each do |api_issue|
       project_id = api_issue['project']['id']
       order_number = api_issue['order_number']
       created_on = convert_to_datetime(api_issue['created_on'])
       last_updated_on = convert_to_datetime(api_issue['last_updated_on'])
       last_updater = api_issue['last_updater']['name']
       p = Project.find_by(project_id: project_id)
       sync_issue_detail(p.id, project_id, order_number, created_on, last_updated_on, last_updater)
     end
   end
 end
 private
   def self.sync_issue_detail(local_project_id, project_id, order_number, created_on, last_updated_on, last_updater)
     issue_response = conn.get "/issuetracker/api/v2/projects/#{project_id}/issues/#{order_number}.json"
     if issue_response.body
       api_issue_detail = JSON.parse(issue_response.body)
       i = Issue.find_or_initialize_by(project_id: local_project_id, order_number: api_issue_detail['order_number'])
       i.title = api_issue_detail['title']
       i.description = api_issue_detail['description']
       i.status = api_issue_detail['status']['name']
       i.priority = api_issue_detail['priority']['name']
       i.fixer = api_issue_detail['fixer']['name']
       i.tester = api_issue_detail['tester']['name']
       i.creator = api_issue_detail['creator']['name']
       i.created_on = created_on
       i.last_updated_on = last_updated_on
       i.last_updater = last_updater
       if api_issue_detail['due_date']
         i.due_date = convert_to_date(api_issue_detail['due_date'])
       end
       i.save
       sync_issue_histories(i, api_issue_detail['histories'])
     end
   end
   def self.sync_issue_histories(issue, histories)
     histories.each do |api_history|
       h = History.find_or_initialize_by(issue: issue, history_id: api_history['id'])
       h.created_on = convert_to_datetime(api_history['created_on'])
       h.action = api_history['action']
       h.description = api_history['description']
       h.creator = api_history['creator']['name']
       h.save
     end
   end
   def self.convert_to_date(raw_val)
     num = raw_val.gsub(/[^0-9]/, '').to_f
     Time.at(num / 1000).to_date
   end
   def self.convert_to_datetime(raw_val)
     num = raw_val.gsub(/[^0-9]/, '').to_f
     Time.at(num / 1000).to_datetime
   end
end

This code may look a bit overwhelming, so let's step through what each method does:

The self.conn method creates a new Faraday connection, using the URL of your DoneDone account. It also uses your DoneDone username and API token to authenticate each request. Replace the placeholders here with your appropriate DoneDone values. Tip: Use ENV variables to store this data on a production environment.

The self.sync_projects and self.sync_issues methods simply use our connection function to request a URL, parse the data that's returned, and insert or update the data in our local database. self.sync_issues also uses a few private methods (defined at the bottom of the class) to loop through issues and their histories.

Since our service class lives in the /app folder, we can use it anywhere in our application's codebase. So, let's make it easy to execute our API functions directly from our model objects.

In /app/models/project.rb, add the following method just above the last end keyword:

def self.sync_from_api
DoneDoneApi.sync_projects
end

Now we can call Project.sync_from_api from anywhere in our application, and Rails will use our service method to connect to the API, download data, and populate our database. Easy! Let's do the same for Issues. In /app/models/issue.rb, add the following method just above the last end keyword:

def self.sync_from_api
DoneDoneApi.sync_issues
end

Now we can call Issue.sync_from_api to synchronize our issues.

Adding controllers and views

We've spent some time creating our back-end logic, but we still have no way to actually view any data in our app. Let's add some controllers and views so we can actually see our data.

Back in your console, run this command:

rails g controller issues index show

This creates a controller named issues_controller, which has two action methods: index and show. When a user visits our index URL, they'll see a list of all our issues. When a user clicks on a specific issue, they'll view its detail via the show URL.

Since the controller actions are responsible for preparing data for display to the user, let's update these methods in preparation for our views:

/controllers/issues_controller.rb:

class IssuesController < ApplicationController
 def index
   @issues = Issue.order(last_updated_on: :desc)
 end
 def show
   @issue = Issue.find(params[:id])
   @histories = @issue.histories.order(:created_on)
 end
end

The index method simply returns all Issues in our database, ordered by their last_updated_on date. The show method looks up a specific issue specified by the :id parameter (which will be part of the URL), and also retrieves its history data.

Each controller action also created its own view - this is where we'll write HTML to display our data. Let's update these:

/app/views/issues/index.html.erb:

<% if @issues.any? %>
<h2>Issues</h2>
<table class="table table-striped">
<thead>
<tr>
<th>Project</th>
<th>Priority</th>
<th>Title</th>
<th>Status</th>
<th>Fixer</th>
<th>Tester</th>
</tr>
</thead>
<tbody>
<% @issues.each do |issue| %>
<tr>
<td><%= issue.project.title %></td>
<td><span class="label label-<%= issue.priority.downcase %>"><%= issue.priority %></span></td>
<td><strong><%= link_to '#' + issue.order_number.to_s + ': ' + issue.title, issue %></strong></td>
<td><%= issue.status %></td>
<td><%= issue.tester %></td>
<td><%= issue.fixer %></td>
</tr>
<% end %>
</tbody>
</table>
<% end %>

/app/views/issues/show.html.erb:

<div class="row issue-detail">
<div class="col-md-8">
<ul class="list-inline">
<li>#<%= @issue.order_number %></li>
<li><span class="label label-<%= @issue.priority.downcase %>"><%= @issue.priority %></span></li>
<li><%= @issue.status %></li>
<% if @issue.due_date %>
<li>Due <%= @issue.due_date %></li>
<% end %>
</ul>
<h1><%= @issue.title %></h1>
<hr />
<p><%= @issue.description.html_safe %></p>
<hr />
<% @histories.each do |h| %>
<div class="panel panel-default">
<div class="panel-heading">
<h6><%= h.created_on %></h6>
<h4><%= h.action %></h4>
</div>
<div class="panel-body">
<%= h.description.html_safe %>
</div>
</div>
<% end %>
</div>
<div class="col-md-4">
<table class="table">
<tbody>
<tr>
<th>Project</th>
<td><%= @issue.project.title %></td>
</tr>
<tr>
<th>Fixer</th>
<td><%= @issue.fixer %></td>
</tr>
<tr>
<th>Tester</th>
<td><%= @issue.tester %></td>
</tr>
<tr>
<th>Created</th>
<td><%= @issue.created_on %><br />by <%= @issue.creator %></td>
</tr>
<tr>
<th>Last Updated</th>
<td><%= @issue.last_updated_on %><br />by <%= @issue.last_updater %></td>
</tr>
</tbody>
</table>
</div>
</div>

Finally, let's make our view look a bit nicer by including the Bootstrap CSS framework. Add the following to the bottom of /Gemfile:

gem 'twitter-bootstrap-rails'

Then run bundle install. Next, open /app/views/layouts/application.html.erb and replace the <%= yield %> line with the following:

<div class="container">
<div class="row">
<div class="col-md-12">
<%= yield %>
</div>
</div>
</div>

Next, add the following CSS content (you'll need to create the first file):

/app/assets/stylesheets/bootstrap_and_overrides.css:

/*
 =require twitter-bootstrap-static/bootstrap
 Use Font Awesome icons (default)
 To use Glyphicons sprites instead of Font Awesome, replace with "require twitter-bootstrap-static/sprites"
 =require twitter-bootstrap-static/fontawesome
*/

/app/assets/stylesheets/issues.scss:

.label-low {
 background: #6da79b;
}
.label-medium {
 background: #f39f09;
}
.label-high {
 background: #f26f21;
}
.label-critical {
 background: #952715;
}
.issue-detail {
 margin-top: 20px;
}
img {
 max-width: 100%;
}
.panel-heading h6 {
 margin-top: 0;
}
.panel-heading h4 {
 margin-bottom: 0;
}
.panel-body p:last-child {
 margin-bottom: 0;
}

This will give all our content a nice, simple layout.

Let's also generate a quick URL for manually synchronizing our issues. We'll then be able to visit this URL anytime we want to refresh data from the API:

rails g controller sync index

And in /app/controllers/sync_controller:

class SyncController < ApplicationController
 def index
   Project.sync_from_api
   Issue.sync_from_api
   redirect_to root_url
 end
end

This performs our sync functions, then redirects to the application root URL. Speaking of which, let's update our routes so that the root URL simply displays the list of issues. Open /config/routes.rb and update it to this:

Rails.application.routes.draw do
root 'issues#index'
resources :issues, only: [:show]
resources :sync, only: [:index]
end

This sets our Issue list view as the root URL. Finally, we're ready to go! In your console, run the following:

rails server

This will start a local web server at http://localhost:3000 - open that up in your web browser.

You'll see an empty page, but that's OK! We haven't synchronized our data from the API yet. Just go to http://localhost:3000/sync to populate your local data. Now you should see all your issues!

Note: In a production environment, we would typically create a recurring task to call the sync methods every few minutes, instead of triggering them manually.

Wrapping up

While this demo has covered quite a bit of ground, it's only meant to serve as a quick introduction to querying RESTful APIs with Rails. If you'd like to learn more, be sure to check out the following links:

• donedone-public-dashboard source code on GitHub
• DoneDone API
• Ruby on Rails
• The Ruby on Rails Tutorial
• Faraday gem