Web scraping movie data

1) Go to [login to view URL]

2) Click on ‘yearly’ along the ‘box office’ section on the left panel

3) You will need to extract box office data on wide releases from all ‘wide release’ movies from 2009-2015. You must now click on a particular year to get the movies for that year. For example, click on 2015 for 2015 movies.

4) You will now see a list of all the movies released for that year. To get a list of the ‘wide releases’ click on the ‘wide releases’ link at the top of the page

5) For the list of movies displayed, you must extract ‘Movie Title’, ‘Studio’, ‘Total Gross’, ‘Total Gross Theaters’, ‘Opening’ dollars, ‘Opening Theaters’, ‘Wide’, and ‘Close’ for each movie from the table (note for ‘Wide’, it might just display the month and day e.g. 1/16, if it does this add the year that you are currently working on. So if you clicked on 2015, make ‘Wide’ = 1/16/2015)

6) Now you must extract data for each movie in the list for a year. Click on the movie title link and you will be taken to the webpage for that movie.

7) From the summary box at the top, extract the following information: ‘’Release Date’, ‘Genre’, ‘MPAA Rating’, ‘Runtime’, ‘Budget’

8) Scroll down to the ‘Domestic Summary’ section of the page. If the section states Limited Opening Weekend and Wide Opening Weekend like:

Limited Opening Weekend:

$633,456

(#22 rank, 4 theaters, $158,364 average)

Wide Opening Weekend:

$89,269,066

(#1 rank, 3,555 theaters, $25,111 average)

Extract the $ values and number of theaters for the limited opening weekend ($633,456 and 4 theaters) and for the wide opening weekend ($89,269,066 and 3,555 theaters)

Instead of this, the domestic summary might just display Opening Weekend

Opening Weekend:

$67,877,361

(#1 rank, 3,845 theaters, $17,653 average)

In this case, just extract the opening weekend $ and the number of theaters ($67,877,361 and 3,845 theaters)

9) Now you will collect daily box office $ for the movie. Click on the ‘Daily’ tab next to ‘Summary’. Then, on the next page, click the ‘Chart View’ link

10) Extract all data in the table for every day the movie was shown

11) You must repeat the process for every year from 2009-2015 and for all ‘wide release’ movies in the year

12) You deliverable will be 2 tab delimited files: (a) movie information for all the movies (separate line for each movie) and (b) daily box office information for all the movies (each line will be box office information for a particular day for a movie)

Pozyskiwanie danych z Internetu

Numer ID Projektu: #7337727

O projekcie