Tuesday, November 16, 2010


Recursion Extends Code Usefulness Rather Seriously In Our Neuron
Smoke that!

After running quite a few iterations for the net, it became apparent that very little useful linking went on. From time to time a long chain would emerge only to be dismantled a few hundred iterations later. For the most part, neurons simply were never active, never linked to the source or the drain.

The problem seemed to be that random linking neurons is simply not effective enough. Even with a high number of neurons, once the critical path between source and drain is broken, it's rare to see it re-established.

Moreover, because of the pathway-rewards and active decay, once the source-drain path is broken, no pathways are rewarded, because none provide output.

So it seems a modification is in order: Neurons must be added to an existing pathway

This means that a basic net consists of at least one input, directly connected to at least one output (neurons with callbacks registered). A net can then have any number of free, unlinked, neurons at startup. These will randomly link with the stipulation that they must link in such a way that they compose a path between the source-drain pair.

So, a neuron may intercede between two connected neurons, effectively extending the path. Or it may bridge entire sections of an existing path, forming a fork. In more complicated instances such a bridged connection may join two existing paths.



Connect x and get

src->a->b->c->drain and src->a->x->drain

where x now bridges the b->c section of the original path

All that is good and well, but we will need some help in linking neurons up in a sensible way. To do this, we add two helpers to the Neuron class: indexToDrain and pathToDrain, detailed below.

The basic gist is that a neuron needs to know which of it's neigbours ultimately leads to a drain. In these functions, a further stipulation is that it must be the shortest route to a drain. This should keep the net's as tight as they can be. Long neuron chains/paths are still possible, with interlinking.

We use recursion to resolve this problem, so that each neuron just needs to know it it is a drain, and if not, if it's direct neighbours are.

There is also an added guard against doubling back on the path, so a neuron will not consider it's caller (by definition a neighbour) when considering routes to a drain.


int indexToDrain();
int distanceToDrain(Neuron* from);


int Neuron::indexToDrain() {
int index = ERR_NOT_FOUND;
if (callback != 0) {
} else {
int distance = 0;
for (int i=0; i< NEURON_RANK; i++) {
if (_links[i] != 0) {
int link_distance = 1 + _links[i]->distanceToDrain(this);
if ((link_distance > 0) &&((link_distance < distance) || (distance == 0))) {
//if distance is still 0 here, we are not a drain and we have not found a drain either(yet)
distance = link_distance;
index = i;
if (distance == 0) {
return index;

int Neuron::distanceToDrain(Neuron* from) {
int distance = 0;
if (callback != 0) {
return THIS_NODE;
} else {
for (int i=0; i< NEURON_RANK; i++) {
if (_links[i] != 0 && _links[i] != from) {
int link_distance = 1 + _links[i]->distanceToDrain(this);
if (link_distance < distance || distance == 0) {
//if distance is still 0 here, we are not a drain and we have not found a drain either(yet)
distance = link_distance;
if (distance == 0) {// we never found a drain
return (distance);

With these in hand, and tested of course, I can now make the net to do proper linking to maintain src->drn pathways...

Thursday, November 4, 2010

#include "code"

Last post I covered the project overview, the basic Makefile and also introduced some classes which seemed to fit the bill for what I want: bR41nzz...

In this post I will delve a little deeper into the class implementations themselves. I'm not going to explain every line of code, I presume you can read enough of it to follow.

So, lets start with the smallest building block, and digress from there...

Note: I'll be describing the system in headers files first, building them up as I go along. Then, when it's time to test the concept, we will write the test and code to satisfy that test.

Another note: I'll be writing the most basic c++ I can, always going for the simplest route rather than the more correct one. For instance, rather than using a private member with a getter and setter, I'll use a public member, until such time as I find I need to implement accessors for some reason.


  1. #ifndef __NEURON_H__

  2. #define __NEURON_H__

  3. class Neuron {

  4. //...

  5. };

  6. #endif /* __NEURON_H__ */

First we will need a header guard, it's ALWAYS a good idea to put them in, they prevent the header file from being included more than once in any compile.

Next, I'll add some obvious parts to the class:
  • constructor/destructor
  • store a value for the neuron (an int for now, as I mentioned before)
  • some way to act/fire the neuron
  • a way to pass input to and receive output from the neuron
  • a way to link to other neurons

  1. class Neuron {

  2. public:

  3. Neuron();

  4. ~Neuron();

  5. int connect(Neuron *dest); //connect to another neuron

  6. Neuron* operator[](const int index) const; //get a connection by index

  7. bool input(const int value);

  8. TFunctor* callback;

  9. private:

  10. int (*_gate)(int a, int b); //the mathematical function to use when this neuron fires

  11. int _value;

  12. Neuron* _links[NEURON_RANK]; //all the neurons this one connects to

  13. };

So, now I can create a neuron, and destroy it. I can connect it to another neuron and access linked neurons by index. I have a gate/action , and I have a value that the neuron stores. I also have a callback to get output from the neuron.

The callback is a functor, I'll write up more on that in the next post

_gate is a function pointer. I've opted to implement them separately as non-class in-line functions, so that they are easily interchangeable (to my thinking, a neuron should know it's value, not what to do with it.).

The gate functions are implemented like this:


  1. #ifndef __GATE_H__

  2. #define __GATE_H__

  3. inline int add(int a, int b){ return(a+b);}

  4. inline int subtract(int a, int b){ return(a-b);}

  5. inline int multiply(int a, int b){ return(a*b);}

  6. inline int divide(int a, int b){ if (b>0) {return(a/b);} else { return 1;}}

  7. inline int pass_a(int a, int b){ return a;}

  8. inline int pass_b(int a, int b){ return b;}

  9. #endif /* __GATE_H__ */

Where pass_a and pass_b are special gates which just propagate one of their inputs.
(I think some neurons don't have to do anything, they just relay info. Some will relay what is told to them, others will relay their own values regardless of what is told to them)

Before I delve into how neurons work together, I think it's time to test a single neuron on it's own.
Test it like a bi-curious nun

  1. //... implementation omitted for brevity

  2. int main() {

  3. printf("test_constructor: %d\n", test_constructor());

  4. printf("test_connect: %d\n", test_connect());

  5. printf("test_operator[]: %d\n", test_operator());

  6. printf("test_input: %d\n", test_input());

  7. printf("test_callback: %d\n", test_callback());

  8. printf("end of tests\n");

  9. }

Wednesday, November 3, 2010

The bird men are coming!

...also known as: Some c++ for the hell of it.

From time to time I fire up a c++ compiler. I started my career out as a c++ developer and have written some fairly cool stuff over the years, I like to think.

Every few years I return to one of two problems, haunting me since my varsity days. The first: I've always wanted to write my own text MUD, from scratch. The second: Neural networks and code learning.

In the next few entries in this blog, I'll take on problem two, again*.

* I looked at mudconnect today to see whats out there.. man, muds are sooo retro. It's like playing a modfile and pressing the turbo button on your PC

Bear in mind I have no extensive formal education in neural networking or what passes under the deceptively cool sounding banner of "AI" in computer science.

To my mind, most of what they teach in AI, even at university level, is just rubbish. There is a lot of formulaic neural nets and that weird-as-shit algebra they apply to it. But in the end, all they are building is a system of weights and counter-weights which calculate a known result. Whats the use of that ?

I must give some credit to efforts along the lines of swarming and hive intelligence. There is definitely something to what these guys are up to. I realise some of them build neural nets into the swarming parts, but in the end thats just to get an "expert system", the AI itself never learns. It just gets better at doing a known thing.

Now, enough of my opinionated rant (hey, this is a blog) and on to some code...

The wind up
I work on linux, so all the examples and references here are in that context. If you are using anything else, I'm sorry. If you are using Visual c++, just fuck off, and take your crap compiler with you.

First, there is the makefile. I won't go through the details of a makefile here, there are many many many pages on the topic already, go read them. I opted for a simple makefile, foregoing my old nemesis autotools. I love autotools, it's a fantastic toolset, but I just don't have the time or patience to read up on it now (it's been years, I used to know the auto-book backwards.), and this is not supposed to be such a big project, so I will manage the makefile manually.

The project folders layout will look like this:

If you build it they will come

PROGRAM = zombie


OBJ_DIR = ./objects
BIN_DIR = ./bin
SRC_DIR = ./source
TST_DIR = ./test

CXX_SOURCES = main.cpp

TEST_FOO_SOURCES = foo.cpp test_foo.cpp

CXX = g++

all: $(PROGRAM) tests

tests: test_foo

$(CXX) $(CXX_OBJECTS) -o $(BIN_DIR)/$@

$(OBJ_DIR)/%.o: $(SRC_DIR)/%.cpp
$(CXX) $(INCLUDES) $(CXX_FLAGS) $< -o $@

$(RM) $(OBJ_DIR)/* $(BIN_DIR)/*

$(OBJ_DIR)/%.o: $(TST_DIR)/%.cpp
$(CXX) $(INCLUDES) $(CXX_FLAGS) $< -o $@

test_foo: $(TEST_FOO_SOURCES:%.cpp=$(OBJ_DIR)/%.o)
$(CXX) $(TEST_FOO_SOURCES:%.cpp=$(OBJ_DIR)/%.o) -o $(BIN_DIR)/$@
We will swap out 'foo' with something a bit more useful later on.

Building blocks
I'll try and run through this in the same way that I approached the programming.

First off, we need to sense of what we want to build. The idea is to build a learning brain. Thats a very ambitious statement, but not if you brack it down into it's component parts.... Lets start with building a brain. The idea is that if we build a half decent brain, the learning will take care of itself.

So, building blocks: Brains have neurons, our brain will need them too. Moreover, brains have functional groups of neurons, networks of them that all work together to achieve the same goal. So we will need 'nets'. That is 1 brain, many nets per brain, many neurons per net.

To keep things simple on the code front, I've decided to only deal with integers (int) for now. You will later see that swapping this out with any other arbitrarily complex type is trivial. I just did not want to get bogged down with anything but the core for now.

So, some classes we will likely need:

I will also need some tests to verify progress and test parts of the classes. I will be writing these myself, that are fairly trivial. I will do a unit test program per class:
Something I should mention here is that every neuron needs to do something. Neurons store information, and act on that information. You can imagine it with logic gates, or in this case mathematical functions. I've decided to use 4 basic integer functions:


Next episode... the code

Wednesday, June 16, 2010

Busy Button Benevolence

Rails as a lot of different ways of doing almost anything. Some of those are frowned on, others hailed as the 'right way'. Of course, the 'right way' changes depending on who you are speaking to, and to what the latest craze is in the rails community.

Buttons, specifically form submit buttons are no exception. My mission today was to prevent 'double clicks', which are a problem when the submit action takes long to resolve: users grow bored and stress test the button.

Ordinarily, one could simply rely on the rails goodness of :disable_with => 'some value'. However, this happens to break ajax forms. Soo...

After some searching on google and a couple of tries, I tossed all the crap out and went back to basics [people suggested using observers and the like to detect the click. Mission!].

Surely html tags have basic events, like onclick, onmouseover and the like? A quick test showed me that <input..../> indeed responds to onclick. So, all I had to do was wire the onclick to some js to turnoff the button.

Here is when I came up with:
<% form_remote_for @foo, :url => foo_path(@foo)  do |f| %>
<%= f.error_messages %>
<table cellspacing="0" cellpadding="0"
<caption>Update a foo <caption>
<%= render :partial => "form", :object => f %>
<td colspan="2">
<%= submit_tag "Update", :onclick => "this.disabled=true;" %>

<% end %>

Yup, that simple. It works partly because of how my page reloads/ajax is designed. The app uses a "content" div to flip ajax content. So, when a user hits 'Update' in this case, the div is replaced with a new 'show' partial or a new(same) 'edit' partial. Point is the button is rendered and you don't need to know to re-enable it on error.

Do do this, I use these little treasures of RJS goodness I concocted myself:


if object.errors.empty?
#the two lines below allow us to call this magic for models associated with other controllers
path = "/#{object.class.name.underscore.pluralize}/"
class_name_underscore = object.class.name.underscore

#create a pass-through variable if none exists
eval "@#{class_name_underscore} = #{object.class}.find(#{object.id})" unless (eval "@#{class_name_underscore}")

page.insert_html :bottom, "list-body", :partial => "#{path}#{class_name_underscore}", :collection => [object]
page.replace_html "detail", :partial => "#{path}show"
page.visual_effect :highlight, "#{class_name_underscore}-#{object.id}", :endcolor=>"#c0c0c0", :restorecolor=>"#c0c0c0", :duration=>3
page.replace_html "detail", :partial => "#{path}new"
page.replace_html "flasher", :partial => "/flasher"


if object.errors.empty?
#the two lines below alow us to call this magic for models associated with other controllers
path = "/#{object.class.name.underscore.pluralize}/"
class_name_underscore = object.class.name.underscore

#create a pass-through variable if none exists
eval "@#{class_name_underscore} = #{object.class}.find(#{object.id})" unless (eval "@#{class_name_underscore}")

page.replace "#{class_name_underscore}-#{object.id}", :partial => "#{path}#{class_name_underscore}", :collection => [object]
page.replace_html "detail", :partial => "#{path}show"
page.visual_effect :highlight, "#{class_name_underscore}-#{object.id}", :endcolor => "#c0c0c0", :restorecolor => "#c0c0c0", :duration=>3
page.replace_html "detail", :partial => "#{path}edit"
page.replace_html "flasher", :partial => "/flasher"

You keep these in app/views/ and forget about them. They do all the heavy lifting ajax in my projects, now that's DRY.

Wednesday, June 2, 2010

Getting bigger

Well, the time is nigh for ShuntYard to start employing drones. This is a good thing, but also rather overwhelming.

In my quest to build ShuntYard into the most kick-ass development house I've even seen, I'm faced with a bit of an existential crisis right out of the gate: should I employ people dumber than myself?

Lets reason this through: fuck no.

If I'm smarted than the people who work for me, it stands to reason I can do their jobs better*. So, if I can do it better, then I can do it faster also. sooo, why am I hiring them?

The short answer is I'm not. But, as with all things, there is a but.

The first mitigating factor here is company culture. I believe that building a successful business is all about the culture. Hire a bunch of depro losers and you will end up with a depro loser company. Hire rockstars and you will probably be in the news for an office shooting, but damn, fame is fame.

At this early stage of ShuntYard's existense, when it's mostly just me, I need to pick people almost as much for their attitude as for their abilities. So, I'm looking for smart, self-motivated, young people, who can grow into what I want/need them to be.

If I go out and hire stagnated experts with their own jaded view of the world, I'll only end up with the kind of company they want, not what I want.

So, even though I'm looking for smart people, I'm looking for people with potential, not necessarily knowledge under the belt.

So my final verdict on this is: Hire people, not skills. Skills can be learned (though dear god, dont hire complete novices, you need to get work done, not play pin-the-algorithm-on-the-noob**)... where was I going with this... oh yeah, duchebags are duchebags for ever, and ShuntYard is only needs the one.

* With in a narrow field of expertise, I don't presume to be a sales drone, for instance. So, for the sake of argument, presume we are talking developers here, people who are supposed to directly take a load off my shoulders.


Friday, May 7, 2010

Yonder employer

Working remotely is definitely an enormous adjustment to the regular office grind. I'm not talking about just being in a remote office, I mean going solo and working by yourself, with only electronic communication at your disposal.

Working solo deprives one (to a certain extent) of the ability to bounce ideas off other people. There is undeniable benefit in getting other opinions on a matter, even if they are dead wrong*.

When you are physically separated from your colleagues (in my case a 2hr flight), you tend to depend a lot more on other resources and your own judgment.

Conveying a technical idea verbally takes a few minutes. Doing it over a tool like gchat is arduous and often leads to frustration and misunderstandings. Nowadays, rather than quickly running an idea past a co-worker, I would now Google it, see if anyone else has had a similar thought. Or I would simply just dive in and do it, see how it turns out and then either keep it or chuck it.

I think there are two key ingredients to making a success of telecommuting: discipline and communication.

You have to constantly check yourself. Have I wasted time investigating this idea? Is this taking longer than it should ? Am I understanding this correctly? There is the ever present danger of just buggering off on a tangent, never to be heard from again.

Having people on the far side who are available and reachable and (more importantly approachable) is critical. I can't imagine my work day without at least a few quick checks like "hey, is this what you had in mind?" or "what the hell are you talking about in that bug report?". You need people on the other end of the line you can converse with over an instant messenger. Email also works, but the hassle of writing and email to just fire of a short question is not justified.

The last thing you want is your communication to always be rigid and formal. You need an easy way to interact with people immediately, the same as shouting across the room or darkening their door for a quick chat. Remeber, you are in the office, your just not in the office.

Also, keep people in the loop. I fire regular short progress update emails during the day. It gives people a handle on what happening, helps them manage their own expectations.

Tools of the trade
My instant messaging is overwhelmingly done in gchat, with a tiny portion on MSN. Most of my clients hosts their email with Google (something I heartily recommend, I do it for ShuntYard also), so everyone in the office is mostly available through gchat.

My email is exclusively done through GMail (private and work), so I have a central place to keep tabs on everything.

*I guess this relates to always running your plans for world domination past a 5 year old.

Thursday, May 6, 2010

Small change is a big change

From time to time I scream myself to sleep.

People around me are presented with two questions.
The first: How can I just sleep at my desk like that.
The second(and more relevant): What makes me scream so ?

The screaming today was triggered by a very simple concept I try and live by (as a developer), that is release early, release often*, or rather the breach thereof.

Following this simple edict lets you stay the course in long projects, it lets you develop the right product, it lets you identify practical problems early, it even makes you appear too fast for love** when it comes to client delivery expectations.

Todays mess and the source of my aggravation was me breaking my own rule. I had waited too long to deploy to production. Sure, I have been deploying to my test and staging environments regularly, but production was about 5 versions behind staging...

The result was that the database on production had missed a couple of upgrades along the way and the data no longer loaded into the more recent schema from staging.

It took a mysqldump of the schema on either side, a diff and some manual vim'ing to get a schema migration script going. Not pleasant. I vastly prefer to do nothing, over doing things as a stupidity tax.

So, from now on, small steps, deploy early deploy often, deploy to production also.

*double entendre alert
**Basil Koufus - every time he gets a chance

Wednesday, May 5, 2010

Testing the waters

From time to time I apply for jobs I have no interest in taking.

I do this to keep my interviewing skills from atrophying, and to meet interesting people.

My latest foray into the world of the job-seeker was with a Stellenbosch tech incubator.

I went through the initial email stage, on to a phone interview where the do the obligatory questions about AND and XOR and all sorts of rubbish no-one cares about, not even them (I asked).

From there it was the face-to-face interview. They booked a 5 hour chunk of my time which I found quite amusing to start with. At least the saw to lunch.

I postponed the meeting by a week, to see how interested they were. They accommodated me and it was full steam ahead.

I was interviewed by 4 different people, one after the other. They could have interviewed me together and got it done in 1 hour.

Their offices were nice, very "we got VC and spent it on beanbags". The people were nice but all drones.

In the end I did not get a job offer. They were very kind to send me a rejection email with a few reasons: apparently I respond to strongly when challenged...

Hmm.. do I have a bad attitude? Surely not! I think rejection was in the mail when I told the development manager that I have no respect for authority (by that time he had started to bore me with silly questions and was contradicting himself).

Why is it that managers think they need to be respected? Surely respect is something you earn, and when it comes to technical ability, even more so.

I find I cant be a drone anymore... been there and done that. I need choice/control over who I work with, I've grown to jaded to work with people who "just dont get it".

Grails, GORM database migration

Yesterday I ran into an interesting problem on a Grails application deploy.

I am writing a financial system in grails. It's a work in progress and so far it's been ticking along ok. Early on, I defined my amounts as 'float' in the domains.

A few days ago we started testing huge amounts for transactions, and realised that they get rounded past 6 digits. So, 1234567.89 would turn out as 1234570.00

The problem: floats in java are not very large, and doubles would have been a better choice. Fair enough, I'm not a java person, so I dont even feel bad about this. underpinning all this is that Grails then magically creates a database table using the mysql float type to match. It has the same limitation.

So, change all the domains where this is of importance and redeploy right ? Wrong. Grails is to addled to tell that I have changed my type from float to double, and does nothing to automagically update my table definition.

So, I ran a schema-only dump in my fresh (correct) dev database, and a data-only dump in my production environment. I then recreated the database with the new schema and overlaid the old data onto it. Dumb luck is that the change was just from float to double and the data could insert fine.

This all got me looking into decent db migration tools for grails. Of which there are none. Liquibase and associated Autobase for grails are just silly with a cludgy DSL. Grails simply has nothing that compares to the ease and elegance of Rails migrations.

When I started this project, the hair on the backk of my neck went up because DB control was left with the framework... seems me foreboding was not misplaced.