data extraction and parameter mapping code project

Anulowany Opublikowano Oct 3, 2011 Płatność przy odbiorze
Anulowany Płatność przy odbiorze

The project entails creating an administrator setup page and a user page with associated backend coding to facilitate the extraction of data from the target websites coding.

* * *

SavedURI :[Show URL][1]

SavedURI :[Hide URL][1]

<[url removed, login to view]>

## Deliverables

The concept of this project is to build the capability to allow for the extraction of parameter names (Fields) and parameter values from the FORMS of a website. So for example, I may use an automotive site in which I have entered values into a form for my make, model and year of car. The code would then have the ability to log into my account and pull both the Fields names for the three variables representative of the make, model and year of the car as well as the actual values of those parameters. (e.g. CarName, CarYear, CarModel as well as 1999, Ford, Thunderbird).

The specific approach and interface does not necessarily need to be followed, but is provided as a possible guideline to which to reference. If the coder wishes to change the approach to an alternative, the project adminstrator is open to that with discussion. Note that the approach defined should work for html-coded target websites. A similar approach should be defined to allow for use with websites that use javascript and other coding approaches. The project includes both an adminstrator setup page, an example mock up is provided to demonstrate the requirement, but it is incomplete. The system should also use a dashboard page. An example mock up is not provided. Admin Page 1) the user is able to enter a value representing the category of website under consideration. While this mockup shows only a single category parameter, it would be more desirable to have 6 levels of categorical values. The category values would be dynamically pulled from the database as existing possibilities to select within a dro down list. if the specific value is not represented within the existing data, the ability to add the value is made available. Similarly, if the existing data is in correct, the ability to edit and even delete the categorical value should be provided. Note that when the admin asks to delete a categorical value which has underlying data, the system should prompt the admin that underlying data exists and prevent deletion of the category until it empty, Once a categorical value is selected, the user is then moves on to the next section. 2) In this section, the user enters a label representing the specific website of interest. The data should be validated against the value preexsting in the list. The list should be dynamically pulled from the database for the given categorical selection from part 1). If the data already exists within the table, the system should prevent addition of the repeat value and warn the admin that the value already exists. If the value does not exist within the drop down list, the user may add a new value. Similarly, the user may edit or delete an existing value by highlighting that value within the drop down and having the ability to edit the label or delete the value. In this case, it does not matter if there is existing parameters below the company name. The system may still allow the admin to delete the value and all associated parameters below it. Once the company/website name is established from part 2) and associated with the categorical hierachy from part 1), the admin may proceed to the next section. 3) In this section, the admin will enter a series of URLs within the website. Additionally, the admin with define the primary purpose of the page type and the desired order of the page type. One method to do this is to highlight the page name, page type and order and then click save to save the association within the database. The ability to add possible values for the page, and page type should be made available. Similarly, the ability to edit and delete such values should be made available. The page assignments to the order should be unique, but the page assignments to the page type do not need to be unique. Once the pages are defined and assoicated to the page type and order, the user then executes to get the associated page forms. the code with them visit each of these URLS and extract the page forms and the associated fields for each form, which will be saved to the database and show up within the drop down menues for sections 4 and 5. The user then proceeds to section 4. 4) The admin will highlight a given web page from section 3 in order to operate on the data in sections 4 and 5. In section 4, the specific forms associated with the given URl will show up in the drop down menu. The user then highlights a given form and defined a form type from the drop down for type. The ability to add, edit and delete form type should be made available. Once the form to type associate is made, the user saves that association to the database. Once all forms have been assoicated and saved, the admin then proceeds to section 5. 5) In section 5, the the admin again highlights any specific field and defines the assoication to the field type. While not shown in this picture, the save entry box for field type should be made with the ability to add, edit, delete a field type. Again, the list of field types is pulled dynamically from the database, but for only that particuarly category. the ability to display the data or more preferably export the data to CSV or even more preferably export it to excel should be provided directly from this page so that the admin is able to sanity check the overall data set. Once the associations are done for website/company, the admin should then be able to enter the log-in credentials for that site (userID, password, pin, email etc) and save them to the database. The system should then be able to go into the web pages of the website in the order indicated from the setup and pull the values of the Fields. The data presentation and export should then include the Field value in its data presentation. Dashboard Page: 1) on the dashboard page, the user is able to search or browse for a given company. If browsing, the user simply uses the categorical hierarchy to identify the company. If the user is searching, they enter the company name and the system searches for the value from the list of companies in the database. The system should show the categorical hierarchy for any chosen company. Once the company is selected, the ability for the user to enter their log in credentials should be made. the system saves the log in credentials then extracts the data that was configured by the administrator. The user account data is saves to the users account database.

JavaScript PHP

Numer ID Projektu: #3611325

O projekcie

Zdalny projekt Aktywny Oct 4, 2011