Build 2 web scraping tools and populate MySQL database
$750-1500 AUD
Płatność przy odbiorze
Stagecoach Software Pty Ltd requires a freelancer to create two automated web scraping tools and a MySQL database suitable to serve as a back end to a forthcoming website to be built on Ruby on Rails.
The first tool needs to run on an automated schedule, initially at a set time every week. The tool is required to scrape a single large web site listing thousands of unique products and then populate a MySQL database with details of the products and prices sought. The details to be scraped appear in a consistent format on the site. The first scrape is required to obtain details for all products whilst subsequent scrapes are required to track new products listed or changes of price on existing products. The freelancer must populate the DB using the scraper as part of the project.
The second tool is required to automatically download and scrape 2 to 4 pdf consistently formatted text files published every week on a different website that list the realized prices for a significant proportion of the products listed in the first website. The second tool is required to populate a further table in the MySQL database with these products and price results.
The MySQL database must then automatically combine the data from the two scraping tools and calculate the difference between the realized price and the initial listed price(s). The DB must be loaded to a host site nominated by me. The scraper tools must be able to be easily activated from the host site and full operating instructions must be provided.
Full details of the sites to be scraped and the required DB columns will be supplied to the successful bidder.
This project will be followed with the creation of a RoR website that will present graphs showing aggregate and individual data obtained from the database and allow user searches. Preference will be given to bids that make that better facilitate this second stage.
Numer ID Projektu: #4975478
O projekcie
13 freelancerów złożyło ofertę za $1129 w tym projekcie
I'm interested and would like to know details and see site to scrape and sample PDF document as well.
We are a company expert in WEB SCRAPING with hundreds of clients. This is a sponsorized bid (we had to pay for that) because we are the only company that can do this job. Reply in private to see our demo.
Hi, We are a team of experience web application developers. We have past experience with scraping projects. We use html parsing framework works to ensure data accuracy and compliance with web standards. Please get i Więcej
Hello, We have a team of designers and developers focus on developing and implementing enterprise products and solutions. We have over five years of experience, a passion for providing great service. Since we had si Więcej
