I need a scraper or script that will scrape major Shopping websites or feed for it's product inventory keyword list. The script should be able to scrape the product list and then export the list of keywords to a text or csv file.
It would be ideal if the script could group the keywords thematically into verticals, such that electrical products are grouped in an electronic group separate to sports related products for example.
The script should be able to handle millions of keywords and should be able to differentiate between the product inventory and general keywords which aren't required. It should be able to provide the keywords in a neat, manageable format, with the expectation that it will be scraping millions of keywords, and thus be able to cope with this.