Stubbing RubyMotion HTTP

Coming from the Ruby and Rails world, I’m anchored in the paradigm that we should test our code. Our code should not only be tested, but it should be sure not to be hitting external services during those specs. I’ve had a hard time figuring out the best way to stub out HTTP calls effectively and easily in RubyMotion. I’m also not the best Objective-C programmer, so porting libraries already built was not on the cards in the timeframe I wanted.

However, I came across the WebStub library by Matt Green in my search for answers, and played around with it over the weekend. It’s fantastic. Coupled with Clay Allsop’s AFMotion library that creates a ruby bridge to the AFNetworking library.

Let’s run through how I went about testing a little API driven app I’m writing. I assume you have RubyMotion, and that you’re not new to using it, but you’re unfamiliar with how to test HTTP calls. It uses BubbleWraps HTTP calls, not AFMotion, however it’s the same testing method.

Create a RubyMotion Project

We’re going to clone the brilliant API Driven Example written by Clay Allsop. Did I mention he wrote the RubyMotion Book? It’s good. Buy it. The API Driven example on the RubyMotion Tutorial is ripe for a little sprinkle of test. So go ahead and run through that example if you want to play along with me adding specs to it.

Once you have that, we’ll start adding our specs. I’ve cloned the git repository that Clay has kindly placed this in, and taken the Colr application straight out of it.

In the Color class, we have 2 HTTP calls, one is a GET and the other is a POST. Let’s wrap a test around that GET request first. The find method looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def self.find(hex, &block)
  BW::HTTP.get("http://www.colr.org/json/color/#{hex}") do |response|
    result_data = BW::JSON.parse(response.body.to_str)
    color_data = result_data["colors"][0]

    # Colr will return a color with id == -1 if no color was found
    color = Color.new(color_data)
    if color.id.to_i == -1
      block.call(nil)
    else
      block.call(color)
    end
  end
end

Add WebStub to your Rakefile.

1
2
3
4
5
6
7
8
9
10
# -*- coding: utf-8 -*-
$:.unshift("/Library/RubyMotion/lib")
require 'motion/project'
require 'bubble-wrap'
require 'webstub'

Motion::Project::App.setup do |app|
  # Use `rake config' to see complete project settings.
  app.name = 'Colr'
end

Here’s the color_spec.rb spec I built to demonstrate testing that call.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
describe Color do
  extend WebStub::SpecHelpers

  before do
    disable_network_access!
  end

  describe "#find" do
    before do
      @successful_color_response = '{"colors": [{"timestamp": 1348628320, "hex": "ffba13", "id": 5490, "tags": [{"timestamp": 1129240205, "id": 14386, "name": "cheesy"}, {"timestamp": 1108442341, "id": 5076, "name": "fries"}, {"timestamp": 1108442340, "id": 5075, "name": "cheese"}, {"timestamp": 1108442341, "id": 5076, "name": "fries"}, {"timestamp": 1344743165, "id": 26414, "name": "yellowsnow"}, {"timestamp": 1109734601, "id": 6414, "name": "bee"}, {"timestamp": 1108110850, "id": 2542, "name": "yellow"}, {"timestamp": 1346850231, "id": 26442, "name": "kahit"}, {"timestamp": 1120839389, "id": 13877, "name": "ano"}, {"timestamp": 1348492735, "id": 26461, "name": "uniqum"}, {"timestamp": 1348627829, "id": 26463, "name": "somecolor"}, {"timestamp": 1348627829, "id": 26463, "name": "somecolor"}, {"timestamp": 1348627829, "id": 26463, "name": "somecolor"}, {"timestamp": 1348627829, "id": 26463, "name": "somecolor"}]}], "schemes": [], "schemes_history": {}, "success": true, "colors_history": {"ffba13": [{"d_count": 3, "id": "5075", "a_count": 5, "name": "cheese"}, {"d_count": 2, "id": "5076", "a_count": 5, "name": "fries"}, {"d_count": 1, "id": "5077", "a_count": 2, "name": "ham"}, {"d_count": 2, "id": "5078", "a_count": 2, "name": "spazz"}, {"d_count": 1, "id": "13840", "a_count": 1, "name": "01oct05"}, {"d_count": 1, "id": "14849", "a_count": 1, "name": "998"}, {"d_count": 1, "id": "3336", "a_count": 1, "name": "love"}, {"d_count": 0, "id": "14386", "a_count": 1, "name": "cheesy"}, {"d_count": 0, "id": "26414", "a_count": 1, "name": "yellowsnow"}, {"d_count": 0, "id": "6414", "a_count": 1, "name": "bee"}, {"d_count": 1, "id": "2542", "a_count": 2, "name": "yellow"}, {"d_count": 0, "id": "26442", "a_count": 1, "name": "kahit"}, {"d_count": 0, "id": "13877", "a_count": 1, "name": "ano"}, {"d_count": 0, "id": "26461", "a_count": 1, "name": "uniqum"}, {"d_count": 0, "id": "26463", "a_count": 4, "name": "somecolor"}]}, "messages": [], "new_color": "ffba13"}'
      @unsuccessful_color_response = '{"colors": [{"timestamp": 1356329465, "hex": "ffba1", "id": -1, "tags": []}], "schemes": [], "schemes_history": {}, "success": true, "colors_history": {}, "messages": [], "new_color": "ffba1"}'
    end

    it "converts a json color into a Color object" do
      stub_request(:get, "http://www.colr.org/json/color/ffba13").
        to_return(body: @successful_color_response, content_type: "application/json")

      @color = nil
      Color.find('ffba13') do |color|
        @color = color
        resume
      end

      wait_max 1.0 do
        @color.class.should == Color
      end
    end

    it "returns a nil when no color is found" do
      stub_request(:get, "http://www.colr.org/json/color/nothing").
        to_return(body: @unsuccessful_color_response, content_type: "application/json")

      @color = nil
      Color.find('nothing') do |color|
        @color = color
        resume
      end

      wait_max 1.0 do
        @color.class.should == NilClass
      end
    end

    it "tells the Colr api to add a tag to the color" do
      stub_request(:post, "http://www.colr.org/js/color/ffba13/addtag/").
        to_return(body: @successful_color_response, content_type: "application/json")

      @color = Color.new(hex: 'ffba13')
      @color.add_tag('mustard') do |color|
        @color = color
        resume
      end

      wait_max 1.0 do
        @color.class.should == NilClass
      end
    end
  end
end

Things to Note

First, we need to engage the WebStub library, and disable any network access. WebStub will let us know of any unauthorised networked access and block it, perfect for debugging url’s.

1
2
3
4
5
extend WebStub::SpecHelpers

before do
  disable_network_access!
end

Set up the response that WebStub should give you if you call the stubbed URL. This is better done from fixtures, but for verbosity I’m putting it here for now.

1
@successful_color_response = '{"colors": [{"timestamp": 1348628320...olor": "ffba13"}'

Now, set up WebStub to listen on the URL that we’re calling for this test.

1
2
stub_request(:get, "http://www.colr.org/json/color/ffba13").
  to_return(body: @successful_color_response, content_type: "application/json")

Here’s the fun part. HTTP calls are done in a separate thread from the main thread, so catching when they’re done requires a little ‘hooking’. We put the resume method call in the block, my understanding is that this is like calling thread.join in Ruby. We want to wait for it to return in our test thread, not go running off all by itself and returning results to the abyss.

1
2
3
4
5
6
7
8
9
10
11
12
13
# Local variables can go out of scope before a block finishes in RubyMotion, so we use an instance variable that won't be garbage collected.
@color = nil
# Call our Color#find method and set the instance variable to the response
Color.find('nothing') do |color|
  @color = color
  resume
end

# Now we need to wait for the thread to join (this is to go with the resume above)
wait_max 1.0 do
  # Normal test patterns resume here
  @color.class.should == NilClass
end

That’s all there is to it. If you’re using AFMotion like me, your specs will look just the same. The only difference being I usually abstract the service layer to it’s own module/class so I can switch it out later if I have issues.

I hope that helped.

Comments

My CodeClub Assembly

Today I gave a school assembly at Burley Oaks Primary School on the CodeClub class I’m a volunteer for. I was nervous! I was pitching my class to 106 Year 6 students, seeing who would be interested in learning to code using Scratch, making games and learning the building blocks of development. I got some great advice from the Head Teacher (Claire Lee).

I have to say it went better than I expected, as nearly everyone wanted to come to the class. A shame really because I can only really do 12 at a time, so we’re hoping to get more volunteers for the school.

If you just want the presentation files, skip to the Downloads at the bottom.

So here’s what I talked about, and some hints and tips for others who want a little headstart on their assembly.

Make it fun

My presentation had a few silly images, and a little humour. Honestly it was nothing different than I normally do for my talks with adults, I sprinkle in a little fun to break up the information.

Explain what coding is

Most children won’t understand what coding is, so I explained that it was about writing in computer language what you want the computer to do and then running it. I showed examples of different development to put it into context.

Relate it to gaming

I showed different games and explained that they were coded by either single people or teams, and that they did the same thing i was going to teach them, but with words instead of Scratch’s drag and drop interface.

Interact with the kids

Keep the kids engaged, there’s nothing worse than a presentation where the kids are made to listen without thinking. It turns them off, so ask them questions and get them to raise their hands or make noises. When I showed images of games, I asked the kids if they knew what games they were.

My Presentation

So, here’s how my presentation basically went, you could do something similar. I’ve put the rough wording I used with each slide before each:

Hi everyone, thanks for coming. There’s more of you than I thought, I hope you’re going to like what I talk about. Let’s get started. I’m here to talk a little about an after school club I am starting in a couple of weeks. It’s all about learning how to be a programmer. It’s called CodeClub.

Slide 1

Let’s start with who I am.

Slide 2

I’m Jamie van Dyke. I’m a very serious man. This is my serious face.

Slide 3

I run my own company, it’s called Fear of Fish. I’m not really scared of fish, but it’s a funny name isn’t it?

Slide 4

I do lots of different types of development. For example I do web development, which is making web sites.

Slide 5

I also do applications for iOS, which means the iPhone, iPad and iPod.

Slide 6

I make applications for Apple Mac’s.

Slide 7

I used to do Windows applications too, like the ones you use on your computers at school and maybe at home.

Slide 8

But we’re not here to talk about me, we’re here to talk about what we can do in our little club…and that’s make games. Do you recognise some of these games? Let’s see how many of you know what these fluffy creatures are for. Hands up if you know what game these are from? (get answers).

How about this one…and this one ( did the same for all 3 )

Slide 9

Do any of you know what this little clip is from? (get responses, my first answer was minesweeper which was a good guess) That’s right, it’s Tetris. This isn’t the original game, this is one that was made by someone else. How old do you think they were?

( take some guesses, have a giggle with them )

The person who wrote this version was just…wait for it.

Slide 10

9 Years Old! Yep, somebody your age wrote this. They wrote it in Scratch, which is what we’re going to learn. How amazing is that?

Slide 11

It’s crazy! Just like this littly kitty!

Slide 12

Let’s have a look at some other things that you might think would have been coded. Do you recognise these images? Each one of them had to be coded to tell the computer what to do. (Talk about each one and get the kids to guess what they are)

These are programs you could probably guess were programmed. How about some that maybe you wouldn’t.

Slide 13

These ones are little less obvious. We have some chocolates coming off a conveyor belt. How did the conveyor belt know when to stop and start, how did it know how much chocolate to put in each heart? It was programmed.

How about the watch? How does it know what time it is, and how does it know what to draw for this little chart here, or the timer here?

Finally, what about this plane? Yep, this is programmed too. A lot of planes nowadays can fly themselves, they can takeoff and land without a pilot! All of this had to be programmed into the plane.

Slide 14

How do we program these things? We use programming languages, computers know how to understand this code. It’s like learning English, except it looks a little scarier. It’s not scary though, once you learn it. This is code I use on a daily basis, it looks a little like English but the computer knows what I mean.

We’re not going to learn this though, this is what you could move into after you learn the basics of coding. We’re going to learn Scratch.

Slide 15

This is what Scratch looks like, it has very few words that you need to use. Instead we drag and drop these blocks around and each one tells Scratch what to do. If we want to move this carpet across the screen when we press the left arrow on our keyboard, we use a block that says “When I press my left arrow”, and we would link that to a block that says “Move the carpet to the left”.

it’s nice and simple. Let’s have a closer look.

Slide 16

Here we go. This one is a little more complicated, but you can see that it’s just little blocks that look like a puzzle when you put them together. By piecing these blocks together, we can make lots of different types of games.

We’re going to start off with a game which has a cat and a mouse, and you have to move the mouse around using your computer mouse to escape the cat. The longer you stay away from the cat the bigger your score gets. Who would like to make a game like that? (ask for a show of hands)

Slide 17

Okay, no you have the idea of what we can learn to do. Who wants to ask questions?

Slide 18

All Done!

This is basically how my talk went, there were lots of questions about what you could do with the app once you were done. Like can you share it, can you sell it (A young entrepeneur in the making!), can you do multiplayer. Make sure you understand Scratch before you do your presenation, they can ask some difficult questions!

If you want any other advice please get in touch I’d be happy to help. You can contact me through my email address or on irc #codeclubhq on Freenode, or twitter @fearoffish.

Good luck, fellow Code Club volunteers! Here are the links for the presentation download if you want to download the presentation:

Downloads

Keynote Presentation

Powerpoint Presentation

Comments

Where Are You, Jamie?

I’m still alive! I’ve been contracting for a Healthcare company in Nashville. I’ve been doing a multitude of jobs, the timeline went something a little like this:

  • A month of web development
  • Infrastructure development (cost reduction, increased redundancy, instance automation)
  • Infrastructure Manager
  • Captain of the Imaginarium (Research and Development)

The last one is clearly the strangest title I chose, but it reflected what I do. Here’s some examples of what I get tasked:

  • “Give us geographical redundancy at a low cost”
  • “Find a solution for Cynthia, see if you can automate that”
  • “We need to send and receive faxes, both from a web app and from the desktop”

I research each one, find possible solutions and then prototype them. The final choice is given to the Director.

What else have I been doing?

My main role has been as a DevOps, working with Amazon mostly. Their tools are fantastically designed to help reduce costs while increasing redundancy. I reduced the hosting costs by a third at PharmMD, increasing redundancy at the same time. I took an Advanced PostgreSQL course which helped understand how to architect the entire infrastructure.

Amazon Cloud Formation, coupled with all the AWS tools is great for architecting a redundant, scalable framework. I’ve also played with VMWare’s CloudFoundry to reduce the lock-in you get with a vendor like that.

I’m currently looking around to see who else might need my services, I’m available globally as I work from home. I’m happy flying out for on-location work too, and can be hugely beneficial for initial planning and working closely with others.

If you’re interested, get in touch through my contact details on the ‘about me’ page. I’m not immediately available, but I might be the one you need to plan the next stage of your infrastructure. I look forward to hearing from you.

Comments

Custom Triggers With the Bluepill Monitoring Gem

We use Chef for automating our server infrastructure, but I’ll show you the manual way of installing Bluepill (either locally or remotely) so you can play with it.

1
gem install bluepill

That wasn’t hard, was it? Dependencies will be resolved, and installed too. Now we’ve got it installed, let’s take a little look at the commands we can give it.

Check the status of all applications and processes

1
sudo bluepill status

Start all applications and stop all applications, or a single group.

1
2
3
sudo bluepill start
sudo bluepill stop
sudo bluepill start resque

Quit Bluepill.

1
$ sudo bluepill quit

You configure Bluepill with a configuration file. Here’s one we use to start multiple resque queues. It’s a work in progress but does what we want for now. I won’t explain the config file because there’s plenty of blogs (and of course the Bluepill README) that go into detail here.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
queues = {}
queues["comms"] = 1
queues["rules"] = 2
queues["patients"] = 4
queues["import"] = 1

@app_name   = "pharmmd"
@owner_name = "rails"
@rails_env  = "production"
@rake_command = "bundle exec rake "

Bluepill.application("#{@app_name}_resque", :log_file => "/var/log/bluepill.log") do |app|
  app.uid = app.gid = @owner_name

  queues.each do |queue, count|
    count.times.each do |idx|
      app.process("resque_worker_#{queue}_#{idx}") do |process|
        process.group = 'resque'
        process.pid_file = "/var/run/resque/#{@app_name}/worker_#{queue}_#{idx}.pid"
        process.working_dir = "/data/#{@app_name}/current"
        process.start_command = @rake_command +
          "RACK_ENV=#{@rails_env} RAILS_ENV=#{@rails_env} " +
          "QUEUE=#{queue} " +
          "resque:work"
        process.stop_command = <<-EOF
        kill -QUIT 
        sleep_count=0
        while [ -e /proc/ ]; do
          sleep 1
          let "sleep_count+=1"
          if [ $sleep_count -eq 60 ]; then
            kill -TERM 
          fi
          if [ $sleep_count -ge 70 ]; then
            kill -KILL 
          fi
        done
        EOF
        process.start_grace_time = 5.seconds
        process.stop_grace_time = 75.seconds
        process.restart_grace_time = 80.seconds
        process.daemonize = true

        process.checks :mem_usage, :below => 150.megabytes, :every => 1.minute, :times => 3
      end
    end
  end
end

What I wanted to add was a notification system that let me know when things had gone horribly wrong. Specifically I wanted Campfire notifications, as that’s where all the cool techies at PharmMD hang out.

For notifications, Bluepill allows us to write simple ‘Triggers’. These triggers can be added to a process in the config file, like zoh:

1
process.checks :some_trigger, :every => 1.minute

So let’s add a Campfire notifier. First things first, I’ll be using the tinder gem to talk to Campfire, have a look at their README for details on how it works (or the excellent tinder homepage). Install tinder.

1
gem install tinder

At the top of the Bluepill config file (anywhere above the application block) add the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
require 'rubygems'
require 'uri'
require 'tinder'

class Campfire < Bluepill::Trigger
  PARAMS = [:times, :within, :retry_in, :rails_env]

  attr_accessor *PARAMS

  def initialize(process, options = {})
    options.reverse_merge!(:times => 5, :within => 1, :retry_in => 5)

    options.each_pair do |name, val|
      instance_variable_set("@#{name}", val) if PARAMS.include?(name)
    end

    super
  end
  def notify(transition)
    @logger.info "I so just got triggered with a transition!"
    begin
      if transition.to == "down" && transition.from == "up"
        @logger.info "BluePill: #{@rails_env} worker(#{process.name}) went from #{transition.from_name} to #{transition.to_name}. Restarting them."
        campfire = Tinder::Campfire.new 'pharmmd', :token => 'TOKEN'
        room     = campfire.find_room_by_name('General')
        room.speak "BluePill: #{@rails_env} worker(#{process.name}) died. Restarting it."
      end
    rescue => x
      @logger.info "Totally failed with: #{x.message}"
    end
  end
end

In the initialize method I set up some parameters that I can pass in, as well as provide myself with some defaults to fall back on.

1
2
PARAMS = [:times, :within, :retry_in, :rails_env]
attr_accessor *PARAMS

I create instance variables for each option for access later on, and call super to allow the Bluepill::Trigger class to do its own initialisation.

1
2
3
options.each_pair do |name, val|
  instance_variable_set("@#{name}", val) if PARAMS.include?(name)
end

Next up I create one of the required methods that Bluepill will call periodically and sprinkle in a little logging (@logger is provided by Bluepill::Trigger).

1
@logger.info "I so just got triggered with a transition!"

Next up we wrap our Campfire notification in a rescue block, because we don’t want ay silly exceptions like a lack of net access from bombing Bluepill out. There are a few transitions we can use, the important one for me is when a process dies (even though Bluepill will restart it), I want to know when that happens so I can find out why and stop it.

1
if transition.to == "down" && transition.from == "up"

Tinder makes campfire access so simple. Connect to campfire, select a room, and then ‘speak’ into it.

1
2
3
campfire = Tinder::Campfire.new 'pharmmd', :token => 'TOKEN'
room     = campfire.find_room_by_name('General')
room.speak "BluePill: #{@rails_env} worker(#{process.name}) died. Restarting it."

That’s it for the trigger. Now all we need to do is tell the process that it should fire that trigger. Add the following to the process block just beneath where we checked for memory usage.

1
process.checks :campfire, :rails_env => @rails_env

That’s it. Now when you load the config file in, Campfire will get notifications if the process needs restarting.

Simple Campfire notifications using Tinder and Bluepill? Done.

Comments