Friday night was the 2nd Kiandra hackathon and attended by most our software team including BAs, PMs and QAs.
Our hackathons are fun competitive events (with prizes!) and also give everyone a chance to work on a challenging problem that’s probably different to the type of work they would do normally, and with people they might not usually work with.
Prior to each hackathon we agree upon the theme to give direction and get people working in different areas. Once we have a theme we create a user voice site where people put up ideas for projects which are then voted on. People then choose the teams they want to work on with the ideas that interest them.
At the start of each hackathon participants get a hackathon pack containing snacks, caffeine and a few silly items that help create a fun atmosphere.
The first hackathon’s theme was touch and gesture. My team worked with the Kinect APIs to produce an interactive story wall, kind of like Minority report – well that was the idea anyway…
Last night’s theme was big data/mashups and we had a number of different projects:
- “What is the most manly movie?” (some random title no one had ever heard of apparently!)
- A movie recommendation engine “Netflox” (sorry Cal but the film “Free Willy” should never be recommended to anyone)
- An analysis of traffic data
- A dashboard showing company information
- A look at the stock market and correlation between stocks rising and falling
These projects had varying degrees of success!
I worked on the traffic data team. Although some of our team members had set up Hadoop prior to the hackathon we decided that as no one on the team had used it before and we only had 4-5 hours our time was better invested concentrating on the actual problem rather than wrestling with Hadoop. We thus decided to import the data into something we were familiar with – SQL server.
In hindsight we should have given this more thought as the dataset was very large (hundreds of gigs) and it took forever to import. We should have setup indexes prior to import as they took forever to create afterwards and without them querying was far too slow.
Luckily some of the other members of our team were more successful in their tasks and managed to divide up the data into grid squares and show visualization based on fake data.
One of the things the hackathon really hammered home to me was all the cool stuff that can and will be done with large amounts of data and parallel/grid computation. A science fiction book I read recently discussed how a city’s traffic system was kept flowing by a super computer monitoring traffic flow – it doesn’t seem so farfetched and we are starting to see some of this type of thing already e.g. see Audi’s traffic light detection system.
I look forward to Kiandra hackathon 3!