support: can't get tokenizer to work

[jsilvestre]

Hi,

I was trying to get the lib to work, but couldn't manage to success, so I hoped you could help me. I have tried the various examples and they all work, but can't get my head around my use case:

Tokenizer = require('tokenizer');
t = new Tokenizer();
t.addRule(/^#[\w\d-_]+$/, 'tag');
t.addRule(/^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})([\/\w \.-]*)*\/?$/, 'url');
t.addRule(/^[\s]+$/, 'space');
t.addRule(/^[\w]+$/, 'symbol');
t.on('token', function(token, type) {
    console.log(token, type);
});
t.write("Test with #tag and a http://google.fr/ #url");

Then I get

I think there might be something wrong with my url regex, but I didn't made it up. Is there any chance that I miss something in the way tokenizer work?
Thank you very much, and merry christmas!