I recently have to crawl the web to get some data from a website, even if I’m more a Java addict and convinced that JavaScript is the 21 century basic, this language seemed to be the more efficient way to do this task.
I discovered for this purpose the use if crawler : a javascript library for node.js
This post details this experimentation.