Find Jobs
Hire Freelancers

API Expert to grab Millions of Items from external JSON API and save in my SQL or NoSQL DB

€30-250 EUR

Zamknięte
Opublikowano ponad 7 lat temu

€30-250 EUR

Płatne przy odbiorze
API Expert to grab Millions of Items from external JSON API and save in my SQL or NoSQL DB. I have different external resource which provides me daily updated json file with 100k-1M items each. This items need to be imported in my own DB and needs to be checked for daily updates like price changes or deleted and added products. I need an API which connects my App with this recourses in a fast and efficient way. If you apply to this job, please describe in short what technology or framework you will use and how you will achieve the efficient way for updating the items frequently. I will have more recent work like this in future.
Identyfikator projektu: 11687401

Informację o projekcie

34 ofert
Zdalny projekt
Aktywny 7 lat temu

Szukasz sposobu na zarobienie pieniędzy?

Korzyści ze składania ofert na Freelancer.com

Ustal budżet i ramy czasowe
Otrzymuj wynagrodzenie za swoją pracę
Przedstaw swoją propozycję
Rejestracja i składanie ofert jest bezpłatne
34 freelancerzy składają oferty o średniej wysokości €216 EUR dla tej pracy
Awatar Użytkownika
Hello Good Day I know how to process large JSON files and storing in database. I also have a great track record here. Looking forward to work with you. Thanks Rinsad
€210 EUR w 5 dni
4,9 (1121 opinii)
9,1
9,1
Awatar Użytkownika
I have read your project details, and I am interested in offering my services. I provide 100% satisfaction guaranteed on all my work. Milestone is required to begin; to be released when the job has been satisfactorily completed. I estimate a 4-5 day turnaround (including weekends) on this project. ftp details to your server will be required. Contact me at your soonest convenience so we may get your solution delivered quickly. Thanks, coderXO
€220 EUR w 4 dni
4,9 (759 opinii)
8,3
8,3
Awatar Użytkownika
Hello sir , For the things you mentioned , to achieve this I can make efficient php script that will cinnect through curl to the api and will fetch the response from it and will make some udf fubctions that will check whether the items returned from api are already db if yes then we will update otherwise we will do insert. Waiting for your positive response so we can start the work soon. Thanks
€177 EUR w 1 dzień
5,0 (551 opinii)
7,8
7,8
Awatar Użytkownika
Hi, I have 5 years and 3 months experience in HTML, CSS, Javascript, JQuery, PHP, MYSQL, AJAX, Wordpress, Drupal, ZenCart, OSCommerce, OpenCart, Magento, Joomla, Prestashop and Smarty I am expert in this kind of work and can complete all your requirements. I will use CorePHP, jquery and javascript methods do this. I assure you that i will provide you best quality work and you will definitely hire me again for future work. I am mostly available online if you want to discuss anything or have any questions and you will get reply quickly. Get quality work from me within your time. So please award me this project and give me chance to work with you. Looking forward for reply. Thanks Kishan
€555 EUR w 10 dni
4,9 (119 opinii)
6,3
6,3
Awatar Użytkownika
Hi Sir, I can write an API in golang so parse your json and do the bulk insert to mysql db. golang run in parallel very fast so it can handle big of records to mysql. Thanks Thanh
€200 EUR w 3 dni
4,8 (66 opinii)
6,4
6,4
Awatar Użytkownika
Hi, I can help you to create a PHP script (no framework at all) to do that and save the data into MySQL DB. But firstly, I need to know the returned data structure of JSON and those external API URL/service(s) so I can review it better and adjust my bid & completion time precisely. For its procedure, briefly, each received items (from external json) will be checked in the database first, if not found then it will simply inserted to the DB, but if found, then it will update the item data. After all items from that external json has been processed then the update date (or time if necessary) from all items in the database will be checked and any not-yet-processed item will be marked as deleted (since it is not provided by that json). Regards, Ibnu
€200 EUR w 5 dni
5,0 (43 opinii)
5,9
5,9
Awatar Użytkownika
Hi there! I've been working with the PHP stack for over a decade, and that is the stack I'd be using for this project. I'd probably just use one script to read the api output, then fork the php process to run the mysql statements to test the data and update/insert. Would love a few minutes to chat about what columns you'd be wanting to test incoming data against. Regards, Joseph Cardwell
€271 EUR w 1 dzień
4,8 (37 opinii)
5,8
5,8
Awatar Użytkownika
Hi, This looks an easy task. I'll be using PHP code to read the json data which can be initiated with click of a button OR we can setup cron job to run the code at a fixed time each day. We can then store the data in MySQL. kindly ping to discuss further on your project.
€77 EUR w 15 dni
5,0 (18 opinii)
5,2
5,2
Awatar Użytkownika
Hello, As you mentioned, need to fetch data using JSON and item count to be 100K-1M, using framework would be an overhead for the system. I would prefer: - Core PHP + JSON - MySQLi - Using cron job About Myself: I have been in the industry for a long time now and worked on an e-commerce solution that used to get feed from the vendor on as xml. So more or less, the job is similar and I know the issues that can crop up in such assignments. I am working alone under 'TrickMyIdea' (all-in-one) and would provide the best solution. Thanks & Regards Hardeep Singh TrickMyIdea
€250 EUR w 4 dni
5,0 (28 opinii)
4,7
4,7
Awatar Użytkownika
Hello, I have worked on many API integrations and consumed the API calls in a right manner. I will use .NET(C#) technologies to do this job. Im over 8 years experience in .NET. If you are interested, Please let me know. Thanks, Sheik
€111 EUR w 5 dni
4,9 (18 opinii)
4,4
4,4
Awatar Użytkownika
Hi There, I can do this job for you. Thank You
€155 EUR w 2 dni
4,6 (17 opinii)
4,0
4,0
Awatar Użytkownika
i have done a similar project and can help in implementing the same.
€166 EUR w 5 dni
5,0 (2 opinii)
3,6
3,6
Awatar Użytkownika
The tech i will use is. 1. use python to fetch data from the web(we should see how they send data, I hope they send data in bulk because requsting for each record is a pain, if they are providing us a file of 10M records in json data, thats OK) 2. python or php to upload data to mysql db 3. Setup cron tab to run every day 15 days is what you need to build test and make sure it is working fine. you will definitely need support till that time. Thanks, Raj
€166 EUR w 15 dni
5,0 (4 opinii)
3,4
3,4
Awatar Użytkownika
I am interested in this project as it is exactly within my scope of expertise. I am an 10-year experienced web/ app design and engineering consultant. I have very good experience in development of applications, websites and web portals. Worked on integration of various API, Payment Gateways into websites and implemented open source products. I am very anxious to develop a relationship with you and I would be grateful for any opportunity to enable me to earn your confidence in our services. I can personally assure you that your business is very meaningful to me and will receive the highest degree of attention and that your confidence in me will not be misplaced.
€150 EUR w 5 dni
4,7 (6 opinii)
3,6
3,6
Awatar Użytkownika
hello sir i have 4 years of experience in web development specially in large scale application which process lots of data at a time . as per your requirement I will do this project in hadoop, big data and Java. because that much of data can not be processed using Php and nosql . big data and hadoop can process this data in less time. drop me a message for more discussion. looking forward to hear from you. thanks for reading. have a good day ahead !!
€222 EUR w 10 dni
5,0 (4 opinii)
3,5
3,5
Awatar Użytkownika
Hi, this is Vikas . I have been into Web Development and Design since past 7 years. I have been creating awesome websites for myself and various clients around the world. I focus on great user experience and efficient code in order to optimize the sites both for speed and performance. I also have hands on open source frameworks like Wordpress, Drupal, Joomla, Laravel, Zend, Symphony, Django, Express NodeJS, MVC, ASP.Net etc. Here is a list of my skills- 1. Custom Website building from scratch 2. Website Performance improvment for faster page load 3. PSD to HTML and HTML to Dynamic Website 4. Building Scalable E-commerce Marketplaces 5. Building Great UI and User Experience 6. API Based Integration into websites. I am good at - PHP, HTML, CSS, Javascript, Jquery, Angular JS, Node JS. I have read your project description and I think I would be the right person to do your project. I will ensure great communication throughout the project with timely updates about the progress of the project. I would like to have a chat with you about the project and discuss more about how I will be approaching this task. Ping me and lets get started. Thanks :)
€111 EUR w 3 dni
5,0 (3 opinii)
1,2
1,2
Awatar Użytkownika
Hi! I'm full stack web developer. My skils: - use OOP style coding in all lang. -php (cli, Yii, Yii2, ZendFM, silex) - python (for some reason) pyramid, flask, Django (!!!amasing i'm use Django =) ) - Mysql, MongoDB, Redis, Memcached - Js, Jquery, BackBone, HandleBars or mustach. - gulp, scss - BootstrapTwitter UI - RabbitMQ ( php+python+supervisor) - Build REST API (as back-end) - Build web-apps (admin-front, user-front) as client for REST API (main appication) - Android SDK (native android apps) - git - install and configure server on any vds/vps (Linux only) Nginx + php-fpm + Data_BAse - composer - scrum (Jira, OrangeScrum) What i can't: - Make design, - Make HTML teamplate from PSD I think that using in projects this stack of technologies and lang are beautiful) Each of them are good if work in right place); Fore example We can use flask (python) to get and sort big data in bite-array (very speed and productive).
€222 EUR w 20 dni
5,0 (2 opinii)
0,8
0,8
Awatar Użytkownika
Hi, As per the requirement, we can use Laravel 5.3 framework which has awesome queue worker which will handle job scheduling. By using job, we can call that JSON API and grap the item and then insert ino mysql database. We can schedule this job for running in specified interval . I recently completed an app in laravel which will grap info from PODIO api and then generate pdf . I can do this job with best quality.
€277 EUR w 15 dni
0,0 (0 opinii)
0,0
0,0
Awatar Użytkownika
Trabajo con QCodo, puedo hacer perfectamente un proceso vía CronTab que se ejecute cada cierto tiempo y mantenga actualizada la base de datos luego de recibir el JSON.
€200 EUR w 3 dni
0,0 (0 opinii)
0,0
0,0
Awatar Użytkownika
Hi, I would be using Java with Spring Framework (if it is not too memory intensive for your operation). Initially 2 threads will be started - Thread 1 will begin reading the existing data from the DB into a map wherein ID against the respective product object would be cached Thread 2 will start loading objects from the JSON file into a map using a key. Once both these threads have finished we would compare both these objects maps for changes in attrbutes, addition or deletion of products. Based on the result we would maintain the operations to be carried out as results and perform the same Reading from JSON file will be done serially. However writing to database can be performed in parallel using multiple threads - configurable Since multithreading comes into picture for writing to the DB we would be using connection pooling thus making writing to DB operations faster
€222 EUR w 3 dni
0,0 (0 opinii)
0,0
0,0

O kliencie

Flaga GERMANY
Hude, Germany
4,6
3
Zweryfikowana metoda płatności
Członek od kwi 8, 2014

Weryfikacja Klienta

Dziękujemy! Przesłaliśmy Ci e-mailem link do odebrania darmowego bonusu.
Coś poszło nie tak podczas wysyłania wiadomości e-mail. Proszę spróbować ponownie.
Zarejestrowani Użytkownicy Całkowita Liczba Opublikowanych Projektów
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Wczytywanie podglądu
Udzielono pozwolenia na Geolokalizację.
Twoja sesja logowania wygasła i zostałeś wylogowany. Proszę, zalogować się ponownie.