Extract links from comments from 4 blogging platforms

| | I need to be able to load a .txt file with a list of urls and then have it extract all the links in the comments section only. The platforms I need to extract links from are as follows: Wordpress BlogEngine Moveable Type B2eveolution I only want to extract links from the comments section, no where else on the page. Ideally this would be a file that I can run from my windows desktop or server. I have both windows 7 and windows server 2003. 64bit and 32 bit. It doesn't have to be in C/C++ it just has to run from my pc. If it can't be done reasonably then I can find a web server to run it from, but I would prefer not to. I need it to output this list in a .txt file as a simple list of urls. Ideally it would remove the duplicate urls as well. (or have the option to). I would like the source code. Also I would prefer someone I can work with over time, as I will likely have revisions to the program I would want at some point (I realize this would cost more when that happens) and likely more work to follow. | ## Deliverables I would like this to be multi threaded, 50+ would be nice. I would like it to be able to add a delay, such as a 2 second delay, when accessing pages all from the same domain. I may use a file that has 900 pages from the same domain and I don't want that server to see my IP as hammering. Also the option to put in a proxy or proxies would be ideal, or allowing me to change the proxy in the Internet Explorer settings and utilize that. (HTTP proxy). (this is not a deal breaker if it can't be done so long as a delay can be added as mentioned above, and if it is going to cost a lot extra I may want to opt out of this feature). I need to be able to work with large files. Like import 20,000 urls and extract 700,000 urls etc...