Tuesday, January 16, 2007

Creating a social network from crawling the web

Social networks are everywhere these days. Lots of people are creating crawlers that crawl the web site and generate a social network graph. But lots of these crawlers are very primitive and most you have to download and run locally, not online. I just came across this crawler called IssueCrawler (thanks to one of the contributors on the SOCNET mailing list for this!), which seems really neat. You input a list of URLs and then submit a job to this site. Then they e-mail you back that your job has completed, then you can view the network that you've created and even save it in various different formats. I created a job inputting some URLs from my blog and other sites I visit to see the results. Here is the network graph shown below as created by IssueCrawler attached below.

No comments: