Crawler to read TITLES and DESCRIPTIONS from URLs

Crawler should work the way similar to <[login to view URL]>. Well I don't want it to display URL after URL manually. I want it to read the URLs from the file, crawl the site and write discovered data into the file. There will be 2-3 parameters (tags) that need to be handled. Data is currently in CSV format and need to be stored into the CSV, but this issue is negotiable. Preferance - tested script. Let me know your if you have one. Please let me know: A. Platform. B. Speed & Condition of script. It's very important. I need to process large amount of urls. So please make sure computer or server will not be crashe with the large data. C. Possibility of customization. I don't think that it will be needed, because it's basically one time activity. So I don't need it to be too fancy ## Deliverables 1) Complete and fully-functional bug-free working program(s) in executable form as well as complete source code of all work done. 2) Deliverables must be in ready-to-run condition and be tested on performance issue with the large amount of data. ## Platform Linux / PHP / MySQL