The nest of despair

I've recently had the need to add a supplemental model to a user when they sign up, based on the signup info they provide.

For instance, if a user signs up the have to at least provide

email

password

password confirmation

However I also wanted to be able to register corporate users, so they can optionally supply

company name

and
company url
Enter accepts_pain, I mean accepts_nested_attributes_for.

This is in principle a very good idea and should make implementing this sooo easy.

So, here is my model before I add it in:

class User < ActiveRecord::Base

   has_one :company

   devise :registerable #keeping it simple here

   validates :email, :presence => true

   attr_accesible :email, :password, :password_confirmation

end

and here is the model after...

class User < ActiveRecord::Base

   has_one :company

   devise :registerable #keeping it simple here

   validates :email, :presence => true

   attr_accessible :email, :password, :password_confirmation, :company_attributes

   accepts_nested_attributes_for :company

end
At this point I should point out that you have to put accepts_asshattery after the association is defined. So in this case, after has_one :company

Ok now on to the views, and here is where the pain started.

A normal view from devise might look like this:

<%= form_for(resource, :as => resource_name, :url => registration_path(resource_name)) do |f| %>
  <%= devise_error_messages! %>

  <%= f.label :email %>

  <%= f.email_field :email %>

  <%= f.label :password %>
  <%= f.password_field :password %>

  <%= f.label :password_confirmation %>
  <%= f.password_field :password_confirmation %> 

  <%= f.submit "Sign up" %>

<% end %>

To get nested magic, you have to add the nested fields...

  ...

  <%= devise_error_messages! %>

  <%= f.fields_for :company do |builder| %>
    <%= builder.label :name %>
    <%= builder.text_field :name %>

    <%= builder.label :url %>
    <%= builder.url_field :url %>
  <% end %>
   
  <%= f.label :email %>
  <%= f.email_field :email %>

  ...

This however, had me running around in circles for a good long while, so I'll cut to the chase to you: The nested form requires you to specify "company_attributes" not just ":company"

The error is not, I dare say, very apparent.  In my case the User model was firing off several after_create handlers, and they were all fine. Just company never created. no failure, just not call at all.

The API docs were not much help, and pointed my in the intuitive but wrong direction:

This model can now be used with a nested fields_for, like so:

<%= form_for @person do |person_form| %>
  ...
  <%= person_form.fields_for :address do |address_fields| %>
    Street  : <%= address_fields.text_field :street %>
    Zip code: <%= address_fields.text_field :zip_code %>
  <% end %>
  ...
<% end %>

So there.

RSpec, Capybara and your mom

The Shirking

I've always found an excuse to do very minimal or no testing in the rails apps I write. Mostly (so I tell myself) it's because I write them all by my lonesome, so it's faster to just write it than to test it, and I trust my own code (who doesn't).

Recently though, the development team at Shuntyard has been growing and I find myself in need of some peace of mind when it comes to the code base.

The devs are good, or they would not be working with us in the first place. No, the tests are not so much for testing broken code as it is for testing broken functionality. You see, developers like to solve problems in different ways. Often simply on principle. So, when someone "cleverly" refactors a perfectly fine piece of code, we need to know if it breaks something down the line.

Enter RSpec.

I an not too concerned with unit tests or testing controllers. I trust these work well enough to get the job done, if the job gets done. So what I want to test is application behaviour. Can I log in ? Can I change my profile? Do I see a list of things when I click on the right links ? So, what I'm realy after is acceptance testing.

Enter Capybara.

There are quite a few tools out there to use for testing Rails apps. They al have merit (I presume) or they would not exist. For the testing of Wami though, I opted for something that cut close to the core, and did not abstract the testing too much.


I also make use of factory_girl_rails to construct models, instead of the default fixtures.

In a land before time...
So, on to the testing. I decided to start from scratch, so that I can see exactly how it all fit together .

First off, I added the gems for rspec, capybara to my Gemfile. I already had guard in there, having started this quest for fire on railscasts.com, guard seemed like a good tool. So here is the relevant part of my gemfile:

gem 'rspec-rails', :group => [:test, :development]

group :test do
  # Pretty printed test output
  gem 'turn', :require => false
  gem 'capybara'
  gem 'factory_girl_rails'
  gem 'guard-rspec'
end

The next step is to install rspec properly into the app:

$ rails g rspec:install
   identical  .rspec
       exist  spec
      create  spec/spec_helper.rb

Don't forget to wire capybara in, by adding this to your spec_helper:

require 'capybara/rails'

Now, to test that a user can sign up to my site. We use devise for authentication, but that is irrelevant to the test, as I'm testing behaviour only.

To do this, I need to create a spec for signing in.

$ rails g integration_test signup
      invoke  rspec
      create    spec/requests/signups_spec.rb

Automatic, sort of

Ok, here is where the headaches started. Guard, for all it's automated goodness, was not running my migrations when I added fields to models. The errors it gives as a result are also not too untuitive.

For instance, I added a cellnumber to my model, even before I first ran guard.

Failure/Error: user = Factory(:user)
ActiveRecord::UnknownAttributeError:
   unknown attribute: cellnumber
   # ./app/models/user.rb:31:in `new'
   # ./app/models/user.rb:31:in `create_primary_cell'
   # ./spec/requests/sign_ins_spec.rb:5:in `block (2 levels) in '
At some point I had apparently migrated my test database, but before I added this field with a migration.

I eventually figured that the migrations was the problem, when I manually ran a

$ rake spec:requests

and saw the migrations run explicitly there. Starting guard up after that got through most of the tests, at least past the stumbling block.

So, remember to migrate your test DB also!

The ugly...

... part about RSpec is the counter intuitive DSL. I suppose it's hard to get it right, and there is a good deal of helpers around, but I'm yet to see a concise list of valid rspec matchers...

I'll deal with them in separate posts as I discover them.

In the meantime, where are my two tests:

 describe "SignIns" do
  it "signs in a valid user" do
    user = Factory(:user)
    user.confirm!
    user.confirmed_at.should be_within(1.minute).of(Time.now)
  
    visit user_session_path
    fill_in "user_email" , :with => user.email
    fill_in "user_password" , :with => 'secret'
    click_button "Sign in"
    current_path.should eq(root_path)
    page.should have_content("Signed in as #{user.email}")
  end
end

 describe "PasswordResets" do
  it "emails users when requesting password reset" do
    user = Factory(:user)
  
    visit user_session_path
    click_link 'password'
    fill_in "Email" , :with => user.email
    click_button "reset password"
    current_path.should eq(user_session_path)
    page.should have_content("You will receive an email with instructions about how to reset your password in a few minutes")
    last_email.to.should include(user.email)
  end
end

RubySA: 3-2-offblast!

Good ideas are a good idea

It was pure happenstance that I met Angus Miller. I was not even going to check my email that evening, but the SABCs program lineup compelled me. Angus was looking for some help on a rails development project, and I had some free time on my hands. That was Wednesday 16th February.

By Saturday that week we had met in person. Angus' pet dog-bear nearly ate me, but it seemed more Yogi and less, well not-Yogi*.

Very soon after first starting work together, we realised that there was significant opportunity for collaboration. Saturday's meeting was all about that. The meeting went very well, and the rest they say is history.

Backtrack a bit

In 2009 I came up with an idea for a rails-specific market management application. It would allow allow people to meet, delegate and manage collaboration on rails projects. The project saw a prototype under the working name of "Bright Market", but nothing ever came of it. I got busy with paying projects, it gathered dust. But the seeds were planted.

Sidetrack

Angus has been sitting on an idea for a ruby/rails job portal. Inspired by the success of a sites like jobs.rubynow and workingwithrails, Angus felt there was a need for a local, homegrown ruby and rails jobs portal. Angus even had the foresight to register a domain, in the hopes of one day getting around to creating this app.

Back to the future

We meet, we talk, RubySA is born! The production version will be switched on on Monday, March 7th.

Ruby and Rails in SA

I think that Ruby and Rails are fantastic tools. In the words of Albert Schweitzer: I fancy it.
And I'm not alone. The ruby and (by reasonable extension) rails communities are growing at a steady pace. This is not a fad, it's not a flavour of the month. It's a good tool, and people are realising it and using it.

With RubySA we hope to give the the local Ruby crows in South Africa, a place to call home. For now it's a job portal, cause ruby developers have got to eat, what it will be tomorrow, is up to the crowd.

* pedobear? Nothing happened swear! It was just a lick.

Recursion

Recursion Extends Code Usefulness Rather Seriously In Our Neuron
Smoke that!

After running quite a few iterations for the net, it became apparent that very little useful linking went on. From time to time a long chain would emerge only to be dismantled a few hundred iterations later. For the most part, neurons simply were never active, never linked to the source or the drain.

The problem seemed to be that random linking neurons is simply not effective enough. Even with a high number of neurons, once the critical path between source and drain is broken, it's rare to see it re-established.

Moreover, because of the pathway-rewards and active decay, once the source-drain path is broken, no pathways are rewarded, because none provide output.

So it seems a modification is in order: Neurons must be added to an existing pathway

This means that a basic net consists of at least one input, directly connected to at least one output (neurons with callbacks registered). A net can then have any number of free, unlinked, neurons at startup. These will randomly link with the stipulation that they must link in such a way that they compose a path between the source-drain pair.

So, a neuron may intercede between two connected neurons, effectively extending the path. Or it may bridge entire sections of an existing path, forming a fork. In more complicated instances such a bridged connection may join two existing paths.

Consider:

src->a->b->c->drain

Connect x and get

src->a->x->b->c->drain
or
src->a->b->c->drain and src->a->x->drain

where x now bridges the b->c section of the original path

All that is good and well, but we will need some help in linking neurons up in a sensible way. To do this, we add two helpers to the Neuron class: indexToDrain and pathToDrain, detailed below.

The basic gist is that a neuron needs to know which of it's neigbours ultimately leads to a drain. In these functions, a further stipulation is that it must be the shortest route to a drain. This should keep the net's as tight as they can be. Long neuron chains/paths are still possible, with interlinking.

We use recursion to resolve this problem, so that each neuron just needs to know it it is a drain, and if not, if it's direct neighbours are.

There is also an added guard against doubling back on the path, so a neuron will not consider it's caller (by definition a neighbour) when considering routes to a drain.

neuron.h:

...
public
int indexToDrain();
int distanceToDrain(Neuron* from);
...

neuron.cpp:

int Neuron::indexToDrain() {
  int index = ERR_NOT_FOUND;
  if (callback != 0) {
    return NEURON_RANK;
  }  else {
    int distance = 0;
    for (int i=0; i< NEURON_RANK; i++) {
      if (_links[i] != 0) {
        int link_distance = 1 + _links[i]->distanceToDrain(this);
        if ((link_distance > 0) &&((link_distance < distance) || (distance == 0))) {
          //if distance is still 0 here, we are not a drain and we have not found a drain either(yet)
          distance = link_distance;
          index = i;
        }
      }
    }
    if (distance == 0) {
      return ERR_NOT_FOUND;
    }
  }
  return index;
}


int Neuron::distanceToDrain(Neuron* from) {
  int distance = 0;
  if (callback != 0) {
    return THIS_NODE;
  }  else {
    for (int i=0; i< NEURON_RANK; i++) {
      if (_links[i] != 0 && _links[i] != from) {
        int link_distance = 1 + _links[i]->distanceToDrain(this);
        if (link_distance < distance || distance == 0) {
          //if distance is still 0 here, we are not a drain and we have not found a drain either(yet)
          distance = link_distance;
        }
      }
    }
  }
  if (distance == 0) {// we never found a drain
    return ERR_NOT_FOUND;
  }
  return (distance);  
}

With these in hand, and tested of course, I can now make the net to do proper linking to maintain src->drn pathways...

#include "code"

Last post I covered the project overview, the basic Makefile and also introduced some classes which seemed to fit the bill for what I want: bR41nzz...

In this post I will delve a little deeper into the class implementations themselves. I'm not going to explain every line of code, I presume you can read enough of it to follow.

So, lets start with the smallest building block, and digress from there...

Note: I'll be describing the system in headers files first, building them up as I go along. Then, when it's time to test the concept, we will write the test and code to satisfy that test.

Another note: I'll be writing the most basic c++ I can, always going for the simplest route rather than the more correct one. For instance, rather than using a private member with a getter and setter, I'll use a public member, until such time as I find I need to implement accessors for some reason.

Neuron
./source/neuron.h

1. #ifndef __NEURON_H__


2. #define __NEURON_H__


3. class Neuron {


4. //...


5. };


6. #endif /* __NEURON_H__ */

First we will need a header guard, it's ALWAYS a good idea to put them in, they prevent the header file from being included more than once in any compile.

Next, I'll add some obvious parts to the class:

• constructor/destructor
• store a value for the neuron (an int for now, as I mentioned before)
• some way to act/fire the neuron
• a way to pass input to and receive output from the neuron
• a way to link to other neurons
  

./source/neuron.h

1. class Neuron {


2. public:


3. Neuron();


4. ~Neuron();




6. int connect(Neuron *dest); //connect to another neuron
   


7. Neuron* operator[](const int index) const; //get a connection by index


8. bool input(const int value);


9. TFunctor* callback;


10. private:


11. int (*_gate)(int a, int b); //the mathematical function to use when this neuron fires


12. int _value;


13. Neuron* _links[NEURON_RANK]; //all the neurons this one connects to
    




15. };

So, now I can create a neuron, and destroy it. I can connect it to another neuron and access linked neurons by index. I have a gate/action , and I have a value that the neuron stores. I also have a callback to get output from the neuron.

The callback is a functor, I'll write up more on that in the next post

_gate is a function pointer. I've opted to implement them separately as non-class in-line functions, so that they are easily interchangeable (to my thinking, a neuron should know it's value, not what to do with it.).

The gate functions are implemented like this:

./source/gate.h

1. #ifndef __GATE_H__


2. #define __GATE_H__


3. inline int add(int a, int b){ return(a+b);}


4. inline int subtract(int a, int b){ return(a-b);}


5. inline int multiply(int a, int b){ return(a*b);}


6. inline int divide(int a, int b){ if (b>0) {return(a/b);} else { return 1;}}


7. inline int pass_a(int a, int b){ return a;}


8. inline int pass_b(int a, int b){ return b;}


9. #endif /* __GATE_H__ */

Where pass_a and pass_b are special gates which just propagate one of their inputs.
(I think some neurons don't have to do anything, they just relay info. Some will relay what is told to them, others will relay their own values regardless of what is told to them)

Before I delve into how neurons work together, I think it's time to test a single neuron on it's own.
Test it like a bi-curious nun
./test/test_neuron.cpp

1. //... implementation omitted for brevity


2. int main() {


3. printf("test_constructor: %d\n", test_constructor());


4. printf("test_connect: %d\n", test_connect());


5. printf("test_operator[]: %d\n", test_operator());


6. printf("test_input: %d\n", test_input());


7. printf("test_callback: %d\n", test_callback());


8. printf("end of tests\n");


9. }

The bird men are coming!

...also known as: Some c++ for the hell of it.

From time to time I fire up a c++ compiler. I started my career out as a c++ developer and have written some fairly cool stuff over the years, I like to think.

Every few years I return to one of two problems, haunting me since my varsity days. The first: I've always wanted to write my own text MUD, from scratch. The second: Neural networks and code learning.

In the next few entries in this blog, I'll take on problem two, again*.

* I looked at mudconnect today to see whats out there.. man, muds are sooo retro. It's like playing a modfile and pressing the turbo button on your PC

Brainz...
Bear in mind I have no extensive formal education in neural networking or what passes under the deceptively cool sounding banner of "AI" in computer science.

To my mind, most of what they teach in AI, even at university level, is just rubbish. There is a lot of formulaic neural nets and that weird-as-shit algebra they apply to it. But in the end, all they are building is a system of weights and counter-weights which calculate a known result. Whats the use of that ?

I must give some credit to efforts along the lines of swarming and hive intelligence. There is definitely something to what these guys are up to. I realise some of them build neural nets into the swarming parts, but in the end thats just to get an "expert system", the AI itself never learns. It just gets better at doing a known thing.

Now, enough of my opinionated rant (hey, this is a blog) and on to some code...

The wind up
I work on linux, so all the examples and references here are in that context. If you are using anything else, I'm sorry. If you are using Visual c++, just fuck off, and take your crap compiler with you.

First, there is the makefile. I won't go through the details of a makefile here, there are many many many pages on the topic already, go read them. I opted for a simple makefile, foregoing my old nemesis autotools. I love autotools, it's a fantastic toolset, but I just don't have the time or patience to read up on it now (it's been years, I used to know the auto-book backwards.), and this is not supposed to be such a big project, so I will manage the makefile manually.

The project folders layout will look like this:

./
./bin
./source
./test
./objects

If you build it they will come
Makefile:

PROGRAM = zombie

INCLUDES = \
-I./source

OBJ_DIR = ./objects
BIN_DIR = ./bin
SRC_DIR = ./source
TST_DIR = ./test

CXX_SOURCES = main.cpp
CXX_OBJECTS = $(CXX_SOURCES:%.cpp=$(OBJ_DIR)/%.o)

TEST_FOO_SOURCES = foo.cpp test_foo.cpp

CXX_FLAGS = -c
CXX = g++

all: $(PROGRAM) tests

tests: test_foo

$(PROGRAM): $(CXX_OBJECTS)
$(CXX) $(CXX_OBJECTS) -o $(BIN_DIR)/$@

$(OBJ_DIR)/%.o: $(SRC_DIR)/%.cpp
$(CXX) $(INCLUDES) $(CXX_FLAGS) $< -o $@

clean:
$(RM) $(OBJ_DIR)/* $(BIN_DIR)/*

$(OBJ_DIR)/%.o: $(TST_DIR)/%.cpp
$(CXX) $(INCLUDES) $(CXX_FLAGS) $< -o $@

test_foo: $(TEST_FOO_SOURCES:%.cpp=$(OBJ_DIR)/%.o)
$(CXX) $(TEST_FOO_SOURCES:%.cpp=$(OBJ_DIR)/%.o) -o $(BIN_DIR)/$@

We will swap out 'foo' with something a bit more useful later on.

Building blocks
I'll try and run through this in the same way that I approached the programming.

First off, we need to sense of what we want to build. The idea is to build a learning brain. Thats a very ambitious statement, but not if you brack it down into it's component parts.... Lets start with building a brain. The idea is that if we build a half decent brain, the learning will take care of itself.

So, building blocks: Brains have neurons, our brain will need them too. Moreover, brains have functional groups of neurons, networks of them that all work together to achieve the same goal. So we will need 'nets'. That is 1 brain, many nets per brain, many neurons per net.

To keep things simple on the code front, I've decided to only deal with integers (int) for now. You will later see that swapping this out with any other arbitrarily complex type is trivial. I just did not want to get bogged down with anything but the core for now.

So, some classes we will likely need:

neuron
net
brain

I will also need some tests to verify progress and test parts of the classes. I will be writing these myself, that are fairly trivial. I will do a unit test program per class:

test_neuron
test_net
test_brain

Something I should mention here is that every neuron needs to do something. Neurons store information, and act on that information. You can imagine it with logic gates, or in this case mathematical functions. I've decided to use 4 basic integer functions:

add
subtract
multiply
devide

Next episode... the code

Busy Button Benevolence

Rails as a lot of different ways of doing almost anything. Some of those are frowned on, others hailed as the 'right way'. Of course, the 'right way' changes depending on who you are speaking to, and to what the latest craze is in the rails community.

Buttons, specifically form submit buttons are no exception. My mission today was to prevent 'double clicks', which are a problem when the submit action takes long to resolve: users grow bored and stress test the button.

Ordinarily, one could simply rely on the rails goodness of :disable_with => 'some value'. However, this happens to break ajax forms. Soo...

After some searching on google and a couple of tries, I tossed all the crap out and went back to basics [people suggested using observers and the like to detect the click. Mission!].

Surely html tags have basic events, like onclick, onmouseover and the like? A quick test showed me that <input..../> indeed responds to onclick. So, all I had to do was wire the onclick to some js to turnoff the button.

Here is when I came up with:

<% form_remote_for @foo, :url => foo_path(@foo)  do |f| %>
<%= f.error_messages %>
<table cellspacing="0" cellpadding="0">
  <caption>Update a foo <caption>
  <%= render :partial => "form", :object => f %>
    <tr>
        <td colspan="2">
    <%= submit_tag "Update", :onclick => "this.disabled=true;" %>
    </td>
    </tr>
</table>
<% end %>

Yup, that simple. It works partly because of how my page reloads/ajax is designed. The app uses a "content" div to flip ajax content. So, when a user hits 'Update' in this case, the div is replaced with a new 'show' partial or a new(same) 'edit' partial. Point is the button is rendered and you don't need to know to re-enable it on error.

Do do this, I use these little treasures of RJS goodness I concocted myself:

_create.rjs

if object.errors.empty?
    #the two lines below allow us to call this magic for models associated with other controllers
    path = "/#{object.class.name.underscore.pluralize}/"
    class_name_underscore = object.class.name.underscore
    
    #create a pass-through variable if none exists
    eval "@#{class_name_underscore} = #{object.class}.find(#{object.id})" unless (eval "@#{class_name_underscore}")
    
    page.insert_html :bottom, "list-body", :partial => "#{path}#{class_name_underscore}", :collection => [object]
    page.replace_html "detail", :partial => "#{path}show"
    page.visual_effect :highlight, "#{class_name_underscore}-#{object.id}", :endcolor=>"#c0c0c0", :restorecolor=>"#c0c0c0", :duration=>3
else
    page.replace_html "detail", :partial =>  "#{path}new"
end
page.replace_html "flasher", :partial => "/flasher"

_update.rjs

if object.errors.empty?
    #the two lines below alow us to call this magic for models associated with other controllers
    path = "/#{object.class.name.underscore.pluralize}/"
    class_name_underscore = object.class.name.underscore
    
    #create a pass-through variable if none exists
    eval "@#{class_name_underscore} = #{object.class}.find(#{object.id})" unless (eval "@#{class_name_underscore}")
    
    page.replace "#{class_name_underscore}-#{object.id}", :partial => "#{path}#{class_name_underscore}", :collection => [object]
      page.replace_html "detail", :partial => "#{path}show"
      page.visual_effect :highlight, "#{class_name_underscore}-#{object.id}",  :endcolor => "#c0c0c0", :restorecolor => "#c0c0c0", :duration=>3
else
    page.replace_html "detail", :partial => "#{path}edit"
end
page.replace_html "flasher", :partial => "/flasher"
flash.discard

You keep these in app/views/ and forget about them. They do all the heavy lifting ajax in my projects, now that's DRY.