Kue: Under the hood. Using the Job Que Externally.

Kue: Under the hood. Using the Job Que Externally.

Kue is a priority job queue backed by Redis, built for Node.js. http://automattic.github.io/kue I started using it in my projects where I was supposed to convert the results of computer vision algorithms into GeoJSON. These algorithms generate millions of segments and are run on very large images. Hence the conversion process takes a lot of time! My goal was to provide REST endpoints to users to upload their results and view them in GeoJSON. Kue fits perfectly in this setting since my application server was in Node.js. The users submitted their results which triggered the conversion job that kue tracked and

Continue Reading

Using DC.js with crossfilter on the server.

A while back I wrote Big data visualization with dc.js and crossfilter which proved to be pretty useful for a lot of people. In the post I sketched just an outline of the solution. For someone not familiar with the workings of dc.js and crossfilter it could be pretty vague. So I've decided to write a full working example of using crossfilter on a Node.js server to render dc.js charts. I've pushed the code on Github and over the next few days I hope to refine it. The README doc would contain details of things happening under the hood.

Continue Reading

Minimal React build environment

I've wasted a lot of time with Javascript build environments. I spend a lot of time trying to optimize my development environment in general. A lot of my projects use React for the fronted. This post is mainly for my reference to kick start a react project. Things I'm looking from my tool/toolchain: Transpiling ES6 and React to Javascript. Bundle all the component codes into a single bundle.js Have a "watch" mode to watch for edits on files SPEED! There's nothing worse than painfully slow builds! There is a whole array of posts and boilerplates available on Github. But most

Continue Reading

Javascript workflow/toolchain

I spend a lot of time writing javascript. Over the past few days I've tried to figure out the ideal workflow that works for me. I'm posting it here so that others struggling with similar issues can draw some inspiration and even help me improve my workflow. I use Node.js to do most of my javascript development, both frontend and backend. Lets break down the workflow into components: Terminal I recently switched to OSX and have started using iTerm which seems to be suiting all my needs. For Ubuntu I was using Terminator. I like being able to split my window

Continue Reading

The wonderful and terrifying implications of computers that can learn

Continue Reading

Starting a data science blog: Dataexperiments.net

Dataexperiments.net has been up for a couple of weeks now. Its a data science blog that I have started for journaling my data science related experiments and other interesting stuff pertaining to data science, machine learning, data visualization, big data etc. The post titled "The Goal of this Blog" clearly describes the ambitions and aims of the blog.  I'll continue writing about other cool computer science related stuff here, but would have lesser data science related posts as dataexperiments would be their rightful place.

Continue Reading