Export my netflix ratings?

sfoster

sfoster

Staff Member
Moderator
I would love to export my netflix ratings and the associated data with those films into a csv or xml file or something.

Years back I wrote some fantastic data visualization software. It allows you to categorize and visually drill down into data graphs. It would be really fun if I could apply that to the movies I've seen but it's daunting to think about the data entry it would require to accomplish it manually.
 
Sort by date Sort by votes
Based on a quick look at their documentation it appears you can request a list of a specific user's ratings via their javascript API:

http://developer.netflix.com/docs/read/REST_API_Conventions

The specific endpoint would be http://api-public.netflix.com/users/user_id/ratings

Better do it soon though, as they're shutting down access to the API in November.

UPDATE: nevermind, while the API is still live they're no longer accepting new applications. Your only option would be to find someone who has an existing application and see if you could use their API keys.
 
Last edited:
Upvote 0 Downvote
ItDonnedOnMe said:
Based on a quick look at their documentation it appears you can request a list of a specific user's ratings via their javascript API:

http://developer.netflix.com/docs/read/REST_API_Conventions

The specific endpoint would be http://api-public.netflix.com/users/user_id/ratings

Better do it soon though, as they're shutting down access to the API in November.
Click to expand...

It seems like it's a basic enough request and some people had code to do it in 2010 but the links are no longer active.

I was really hoping I wouldn't have to code this myself. :no:
Great to know that there is still time
 
Upvote 0 Downvote
ItDonnedOnMe said:
UPDATE: nevermind, while the API is still live they're no longer accepting new applications. Your only option would be to find someone who has an existing application and see if you could use their API keys.
Click to expand...

As luck would have it I requested an API key back in 2012.
I think I tried to do this for like 10 minutes and then gave up. :lol:
 
Upvote 0 Downvote
It'd be pretty easy to screen scrape it.

https://www.netflix.com/MoviesYouveSeen:
Code: 
<li data-series="" data-movieid="[COLOR="DarkRed"]611444[/COLOR]">
  <div class="col date nowrap">[COLOR="DarkRed"]7/11/07[/COLOR]</div>
  <div class="col title"><a href="/WiMovie/611444?trkid=200250784">[COLOR="DarkRed"]The Hunt for Red October[/COLOR]</a></div>  
  <div class="col rating nowrap">
  <div class="starbar yellow oneRow" data-predicted-rating="" data-average-rating="" [COLOR="DarkRed"]data-your-rating="4"[/COLOR] data-titleid="611444">
...snip...
</li>

Relevant data highlighted red.
In order: netflix movie id (not really necessary), date rated, movie title, your rating (out of 5)

The tricky bit is it's an ajax loaded page, so it loads more as you scroll to the bottom of the list, that might be tough to manage with code.. but you could just open the page in the browser, scroll down until it stops loading, grab page source, and run it through a script.
 
Upvote 0 Downvote
Will Vincent said:
It'd be pretty easy to screen scrape it.

https://www.netflix.com/MoviesYouveSeen:
Code: 
<li data-series="" data-movieid="[COLOR="DarkRed"]611444[/COLOR]">
  <div class="col date nowrap">[COLOR="DarkRed"]7/11/07[/COLOR]</div>
  <div class="col title"><a href="/WiMovie/611444?trkid=200250784">[COLOR="DarkRed"]The Hunt for Red October[/COLOR]</a></div>  
  <div class="col rating nowrap">
  <div class="starbar yellow oneRow" data-predicted-rating="" data-average-rating="" [COLOR="DarkRed"]data-your-rating="4"[/COLOR] data-titleid="611444">
...snip...
</li>

Relevant data highlighted red.
In order: netflix movie id (not really necessary), date rated, movie title, your rating (out of 5)

The tricky bit is it's an ajax loaded page, so it loads more as you scroll to the bottom of the list, that might be tough to manage with code.. but you could just open the page in the browser, scroll down until it stops loading, grab page source, and run it through a script.
Click to expand...

I went to https://www.netflix.com/MoviesYouveSeen
It only shows a small selection of movies and I have to hit a next button

Are you sure that link is correct? I can screen shot.

For ItDonnedOnMe - I sent an email regarding my 'inactive' key, it hasn't been used for two years. Hopefully they will reactivate it even though they are retiring the api
 
Upvote 0 Downvote
maybe it's your browser.. works for me in chrome.

--removed non-functional link--

EDIT: viewing page source only seems to grab the first screenful.. need to grab from within developer tools after the full thing has loaded. :(

Might be better suited as a command line task.. grab the code and have a play yourself.
http://willvincent.com/netflix-parser.zip

EDIT 2: Need to up the max file size constant in the simple_html_dom.php include file.. tack a couple zeros on the end :)
 
Last edited:
Upvote 0 Downvote
Update: I installed firebug with firefox and now the page loads as I scroll down. I'm trying to inspect the element with firebug but IDK if I know what I'm doing.
 
Upvote 0 Downvote
In chrome, scroll down until it stops loading.. rightclick somewhere and 'inspect element' in the developer tools window that comes up if you right click on the opening html tag you can 'copy as html' that'll include all the rows added by ajax. view source does NOT include the ajax loaded stuff.

Also note the web tool I posted will fail on anything of any substantial size.. download the zip and do it locally with the cli tool (after updating the max file size value in the include file)
 
Upvote 0 Downvote
Actually.. duh.. dunno why I didn't think of it sooner. Instead of dealing with the hassle of copying and pasting several MB of markup, just scroll until it stops loading and then SAVE the page out of the browser (works in chrome anyway).

move that file into a directory with the php files from the archive linked above. Update the max file size setting in the simple_html_dom.php file so that it's at least as large as your filesize. Update the cli-tool.php file to reference the appropriate filename.. the fire it up on the command line with php
 
Upvote 0 Downvote
Will Vincent said:
In chrome, scroll down until it stops loading.. rightclick somewhere and 'inspect element' in the developer tools window that comes up if you right click on the opening html tag you can 'copy as html' that'll include all the rows added by ajax. view source does NOT include the ajax loaded stuff.

Also note the web tool I posted will fail on anything of any substantial size.. download the zip and do it locally with the cli tool (after updating the max file size value in the include file)
Click to expand...

Well I am getting closer! I finally found it in the 'inner html' attribute in the DOM but your way is much easier and doesn't require poking around. :lol:

Okay I am getting close. I guess I need to install TomCat or some type of local server for the amount of data that I have? What's the easiest way to run php files locally on my mac
 
Upvote 0 Downvote
OSX should already have php installed natively.

on mac, unzip the archive, make requisite changes mentioned above, drop your exported html into the same directory and then on the command line:
php cli-tool.php > netflix_ratings.csv

(from within that directory, of course)
 
Upvote 0 Downvote
Will Vincent said:
Actually.. duh.. dunno why I didn't think of it sooner. Instead of dealing with the hassle of copying and pasting several MB of markup, just scroll until it stops loading and then SAVE the page out of the browser (works in chrome anyway).

move that file into a directory with the php files from the archive linked above. Update the max file size setting in the simple_html_dom.php file so that it's at least as large as your filesize. Update the cli-tool.php file to reference the appropriate filename.. the fire it up on the command line with php
Click to expand...

Doh.. so close
File 2.5 MB

define('MAX_FILE_SIZE', 48000000);


Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 72 bytes) in /Users/sfoster/Desktop/netflix-parser/simple_html_dom.php on line 1495

hold on a second .. yeah idk
 
Last edited:
Upvote 0 Downvote
Will Vincent said:
drop two of those zeros and you'll be all set
Click to expand...

Hah I didn't mean to post that many zeros, I kept added them while debugging.

With less I get this error

Fatal error: Call to a member function find() on a non-object in /Users/sfoster/Desktop/netflix-parser/cli-tool.php on line 5

Perhaps I have the file name syntax wrong in cli-tool

<?php
require 'simple_html_dom.php';
$html = file_get_html("Netflix.html");
$reviews = array();
foreach ($html->find('#ratingHistorySection ul li') as $item) {
$reviews[] = array( .. etc

it is in the same directory as the php files where i am executing from the terminal


Netflix.html begins:
<!DOCTYPE html>
<html><head><title>Netflix</title><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta name="viewport" content="width=device-width,initial-scale=1.0,minimum-scale=1.0"><script type="text/javascript">if (! window.netflix) { window.netflix = {}; }netflix.presTracking = {};netflix.presTracking.uiTrackingConstants = {REQUEST_ID:
..etc
 
Last edited:
Upvote 0 Downvote
You must log in or register to reply here.
Back
Top