Create a cloud web scrapper for me

Hi,yavaar19! I saw your requirements and I'm ready to recreate the code you have in Python using best practices of multithreading and multiprocessing (so data could be scraped in parallel and not url-by-url) Please contact me to discuss project details, Best regards, Vlad

Hi there,I am Miljan,Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I can start immediately and finish it within the agreed deadline. Check out my profile, portfolio, and former clients feedback - that'll let you know everything about me. Please feel free to contact me so that we can discuss further details. Thank you for taking the time to read my proposal.I am looking forward to hearing from you. Kind regards

Hi, I have developed similar scraper/crawler and data/web automation projects. Please let me know if you are interested and I am available to start right away.

Here is a senior C# scraper, langlang from China. I have read your job description with exciting mood :) In one word, I can say <I can do your project> proudly !!!! I have really GOOD exp in C# Programming, PHP, Python, Software Architecture, Web Scraping, any other many things ... Please check my reviews; www.freelancer.com/u/langlangfan I am interested in your projects and I believe I can deliver rich features of the application to beat your imagination. Let's work together to make it happen. Let me know if you are interested and we can always start with something small.

Hi. I did lots of crawlers in python and I think this is the right path for it. Yes we can add concurrency to the crawler. Do you need as well a web interface? Let me know details and we can discuss. Thx.

Greetings of the day! I am the best fit to your requested requirement. I can help you in Create a cloud web scrapper My Expertise & Experience: I am skilled in C# Programming, PHP, Python, Software Architecture, Web Scraping I would love to work with you on this project. Carrying total experience of 6.5 years. I can assure you, you won't disappointed with class of the work I produce. I can start the work on the go as per your need! Communication: I will be available on the freelancer.com chat 18 hours in the day. (FLEXIBLE WITH TIMINGS IN CASE OF THE NEED) I am just a message away in case of any queries.

Dear Sir, I'm very much delighted to let you know that i did data scraping with PHP-cURL, PhantomJS, Node.js, Selenium from many sites. I just scraped the data from web site and then wrote the data in mysql database or excel or csv or xml file. I worked on many similar projects, I have big experience in data mining projects. I have written hundreds of web scrapers which scrape millions of pages each day. I'm ready to fulfill your requirement. I can finish this task in short time, with the best quality. I can assure 100% accuracy. Please give me the opportunity to do the work. With Kind Regards, Debdulal Roy Proshanta

Hello yavaar19, We have gone through the details you have provided for "Create a cloud web scrapper for me" and would be pleased to work on this with you to deliver the results that you have expected Also I do have similar experience doing something similar to yours therefore I am looking forward to discuss and move ahead. Relevant Skills and Experience I have 10+ years of experience in C# Programming, PHP, Python, Software Architecture, Web Scraping I can do this within your budget too $250 - $750 USD Let's have chat so i can explain you in deep. Thank you, Regards, Mmadi, Flowgica Technologies.

Hi there I can help your URL checks to run in parallel over proxies. For my bid I will create a service that performs the necessary work, which you can deploy in the cloud. I would recommend something like Azure Functions - it is very cheap to run and you only pay per time that you call your function. Then I can update your client app to call those functions in parallel instead of one by one. You will get the same result but at massive scale. You will need to register for an Azure account to run the app, I can help you set that up - it is super easy to do. Hope to hear from you soon! Regards, Joon

Hi there. I can create you a scraper to do this task. I have created such scrapers before in C# and can create you exactly one you are looking for. Please share the website as i can't see it clearly on the picture you have attached. looking forward to hearing back from you. Have a good day. Thanks

how are you,sir? I am a professional developer who has rich experience in this field. If you contact me, you and i will all be happy. Thank you for your reply in advance. Create a cloud web scrapper for me C# Programming, PHP, Python, Software Architecture, Web Scraping,

Hi I have more than 5 years of hand on experience in web development using PHP and MySql I have expertise in following technologies and tools: HTML CSS Bootstrap Anguarjs Jquery PHP My Sql Wordpress codeigniter I have good eye on design and details and can convert a psd to html if required. I have good analytical ability to breakdown the complex problems in to small units. I have ability to write object oriented and maintainable code. I am hard working and honest person and looking for along term relationship. Quality code in timely manner is my highest priority. I can communicate in English and can we available on skype. I am in IST Time zone but ready to work in any time zone. I can give you 40 hrs/week. Looking for a positive response. Thanks

Ready to start the work to Create a cloud web scrapper, We can discuss more over chat, Thanka Arjun S.

Your project comes under our area of interest, together we can finish your project with our technical expertise and dedicated team.

Interesting project you've got. My experience with building web scrapers ranges from static web pages for simple periodical report creation of product inventories and as far as dynamically loaded pages via JavaScript with bot prevention systems provided by Cloudflare and Distil Networks protecting: ticketing solutions websites, stock market, and Forex analysis. Depending on the level of sophistication of the websites you need to scrape we can evaluate the proper amount of complexity in response. We can use tools such as browser kit spoofing, responsible scraping methods, human like actions, and of course the use of rotating proxies; In the scope of reducing the visibility and increasing the longevity of the scraper. For this kind of job, I personally feel more comfortable using a JavaScript framework such as PhantomJS or SlimerJS inside a Node.js application. I'm not sure what production platform on the cloud you have been looking in but Node.js should fit well and we can also look into some alternatives depending on your needs. Please do get in touch with me, I’d be curious to hear more about your project and I’ll be glad to answer any questions you may have or provide advice if you need. All the best, Andy

Hi, In the end it depends on how well protected the site is. If there are limitations, you can't do too many things... you have to have some delays. But you are correct - running in a cloud environment might help. For example if you run the scraping process on 5 servers that means 2000 records/server. If we calculate an average of 3 seconds/page = 6000 seconds = 100 minutes to scrape all the records. BUT as i have said - that depends on the limitations of the server... maybe you need 5 seconds for a page (including sleep time). The scenario would be: store all liks/urls in either a mysql or redis database (redis has some advantages). Using DigitalOcean as a cloud provider you can create 5 vms that are using the same backend database of urls to run the scraping scripts. If you need more info feel free to contact me via chat.

Hey There !! We have seen the job post of yours and very interested to start work with you and we completely understand the subject of yours and what is required to tailored out. I am having major 13+ years experience in Asp.net,C# mssql,html,html5, .net, MVC, MVC4,MVC5,php, MySQL,css,css3,laravel, yii, wordpress, codeigniter and also with its related frame work. I have developed various application in ajax,.net-framework,C#,css ,html5, Javascript, jquery, MVC4,MVC5,MsSql using Graphicdesign, php, laravel,yii,MySQL and deployed over the internet,If you want then i will show you the robust applications which i have developed so far. Some of the software projects and websites development that i have undertaken and successfully delivered include:( Customized ERP system for Schools and Universities, Management System for Tour and Travels industry,Real states,ticket booking software,complete ecommerce solution, Various MLM portals,bitcoin webiste Web Application for Media Company,Booking Software,Various types of CRM/ERP). Please feel free to contact us for any further discussion. Hoping for a fruitful business and chance to serve you. Regards Dunitech

I did scarping with Amazon Seller Center using C#. For this kind of scraping I need to know the site details (from where you want to get data). So let's discuses.

i have read your requirements. i have done already this job . you can get better service here. i am expert in web scrapping

My specialty is web scraping and multi threaded applications. Testing today on a dual xeon 32gb server, i was able to simultaneously run 400 threads. With that being said, it really depends on your server. I can write a thread engine around your customized needs. My specialty is in the ticket industry, which is web scraping and purchasing of tickets multi-threaded, proxies, etc. Quite complicated. Thanks, Kevin