NeuralDrift

NeuralDrift

*** Moved from my previous website ***

In September, Montreal hosted its first Wearables Hackathon called : WearHacks. In collaboration with Hacking Health, La Commune and others, the team behind WearHacks organized a 48 hours hackathon.

NeuralDriftHave you seen the movie Pacific Rim ?
Yes : This is what we created in 48h. Monsters aside.
No : Go watch the movie and continue reading after.

During the 48h hackathon, we created a 2 player neurogame where both players have to synchronize their brain activity in order to control a robot. Using the Muse Headband from InteraXon, each player controls one side or the robot. The robot is a little smaller than in the movie because we used a Lego Mindstorm EV3 instead of a giant Jagger. Behind the vehicle we put an Android tablet to give some feedback to the users.

NeuralDrift WWTO

With this idea and its execution, we won the creativity prize at the end of the Hackathon. Even more, we got invited at ‘We Are Wearables in Toronto (WWTO)’, to demo it ! Seeing the success of the project and the potential of it, we will continue to work on it and try to hack and exploit (pun intended) its medical purpose : neurofeedback.

Friday
I arrived at the venue, took my name tag and went upstairs to the soon to start pitch clinic. There was a little bit of confusion around that ‘Pitch Clinic’ and a lot of shy people as well. I didn’t pitch, not because I’m shy, but because I wanted to discuss with my team first. Since we were not looking for more teammates, I didn’t know if I were to pitch or not ! Less than an hour later we all go in the main room, where the real pitches will occur. Some more confusion here and there, but we go through all the pitches. About 40 of them ? Give or take 10.

The idea was to do a 60 seconds pitch about your idea that you’d previously registered on the WearHacks’ website. Few of them was good, few of them was not so good, few of the pitchers were naturals, few of them not so natural ;). I guess I was somewhere in between. I pitched the idea of Pacific Rim, and NeuralDrift was born. Our project was to create a 2 player neurogame, where both players have to synchronize their brain activity together in order to control a robot. Very much like Pacific Rim, monsters left aside. Ambitious to create such a game over a weekend ? Indeed ! We knew we could do it though, because after a brainstorm session 2 weeks prior to the event we had some other good ideas, but we picked this one because it was fun, simple, doable and people would relate to this idea and those who’ve seen the movie will simply love it.

Saturday
Saturday was all about hacking. Arriving early and getting started as soon as possible ! We were the first ones to arrive on the third floor so we could pick our spot, we later realized that instead of the best spot, it was the worse one because that roof window does not only bring light, but heat. In a room crowded with hackers and computers do don’t need that extra direct sun heat. Anyway, we start with a general description of the project and task distribution. We would then work on our side for 2 hours and then talk again to see how it’s going. Hours fly by while me and Hubert, members responsible of finding the mental state we will use to control the robot. In the literature there is a lot of good example for paralyzed people controlling wheelchairs and computers with the brain activity, we knew that. We both study in the field, we knew we could work something out. The problem ? We’d never tried the Muse headset and didn’t know what kind of signal we would get out of it. After spending several hours, literally doing research, we decided to stop because we only had 1 day to make this work. We then decided to go with the easiest thing to use in this field of Brain-Computer Interfaces. Alpha band ! Simply put, your brain activity is an electrical signal changing in time that can be converted in frequency (Fourier Transform, for my engineer friends). The Alpha band relates to the frequency between 8 and 12 (approx.) When you relax (or close your eyes to make the process faster), your brain activity will slow down and your alpha band will dominate. That level of relaxation vs no relaxation can be measured and map as a speed for the robot to move.

NeuralDrift - Team Working

The design, the Bluetooth communication, the paradigm was now done ! We had a kick ass team, very focused, very effective (when we stopped doing research ^^). We now need to implement our machine learning algorithm to be able to classify if you are relaxing and so is your partner ! It’s a 2 player game, remember ? Now it’s late at night and 1 member of the team go home to sleep, as did 3/4 of the participants. The rest of our team stayed all night to code and test the game. Reading the brain activity of 2 people, sending that information via Bluetooth to a laptop that will process it and use a machine learning algorithm to decide what to send to the robot and then send some information to the Android tablet as well for some feedback. That’s a lot of communication between devices. At 2h am I draw the communication architecture on a small white board we had, put my headphones on, drank a redbull and code until 6h when I was done with the 7 client/server pairs.

NeuralDrift Hacking

Sunday
Sunday, we were tired. We didn’t implement much, we tested it and fine tune the game mechanic to make it fun. We start working on our pitch for the afternoon. 3 judges came for a technical evaluation of our project. I tried to quickly explain and overview the amazing software design we did in only hours !

NeuralDrift - Technical Judges

A very long day of presentation followed. It was hot in the room of 200 hackers and exhausting listening to all the presentations.
Our plan was simple, we do a demo and I talk my way through the demo. Simple, fun and efficient. But we knew we had to twist the pitch with a business angle. We dislike the idea of talking about the market and all, we wanted to explain how awesome the project was, not its market. We had 5 minutes. We wanted the world (ok, the room) to see how 2 people were able to control a robot with their mind ! We did all that in 36 hours ! The pitches start and we see that nobody is really demoing actual stuff. They have pictures and screenshot, but they talk about the project from a startup point of view. We decided to play along and focus on the pitch itself rather than the demo.

NeuralDrift“, your turn.

I go on stage, start presenting the team and then I see the robot moving in front of me, coming towards my feet. Got distracted and lost control of ‘my show’. I didn’t want to talk about that business stuff, but rather the demo, but now I didn’t even know what they were doing with the robot. So here is a good example of ‘how to mess up a pitch’ :

I learned that with no sleep, 36 hours on redbull hackin’, my ‘improvisation’ skills are not as acute as they would normally be.

The Team
You can see the team here : http://neuraldrift.net/?page_id=12

The Technology
You can see a full description of the technology here : http://neuraldrift.net/?page_id=10

The Experience
The experience was so nice, despite the confusion, that I decided to host my own Hackathon on Brain-Computer Interfaces in 2015 ! BCIMontreal is NPO that I recently co-founded and we organize events, workshop, conference and we just add a Hackathon on our list !

BCIMontreal Meetup : http://www.meetup.com/bci-Meetup/
BCIHack : http://bcihack.com/

Stay tuned !

WWTO
After winning the creativity prize, we got invited by Tom Emrich at WWTO of October to demo NeuralDrift. We accepted the invitation and we had a blast in Toronto demoing our project to 400 people. Lots of people tried it, but since the idea is to relax to control the robot, having a crowded open space where everyone wants to try it was not a good environment. Few people really controlled the robot properly with their copilot. Most of them like the idea and would have liked it to work, but understood that the environment was not really favorable to have fun playing that game by relaxing.