Netflix Party: synchronize Netflix video playback
December 28, 2015
Over the weekend I made Netflix Party, a Chrome extension to synchronize Netflix video playback on multiple computers. You know, for Netflix nights with that long-distance special someone. You can get it here from the Chrome Web Store.

Figure 1: watching The Best Offer with Netflix Party.
The server
The basic architecture of Netflix Party is a client-server model. The clients connect to the server to synchronize their playback state. For the server, I wanted something low-latency that many clients could connect to at a time. Node.js fit the bill nicely.
The server turned out to be the easiest part of the project. It uses a simple in-memory store (a JavaScript object) to hold the sessions, which means that the sessions are lost if the server is ever restarted. For an app like Netflix Party, that isn’t a big deal, but maybe later I’ll switch to something like Memcached or Redis. Setting up the session store is just one line of code:
// in-memory store of all the sessions // the keys are the session IDs (strings) // the values have the form: { // id: '84dba68dcea2952c', // 8 random octets // lastActivity: new Date(), // used to find old sessions to vacuum // lastKnownTime: 123, // milliseconds from the start of the video // lastKnownTimeUpdatedAt: new Date(), // when we last received a time update // state: 'playing' | 'paused', // whether the video is playing or paused // videoId: 123 // Netflix id the video // } var sessions = {};
Sessions that have been idle for more than an hour are vacuumed.
The server also renders a simple homepage at netflixparty.com, using my minimalistic templating engine for Node.js.
The client
The browser extension turned out to be obnoxiously nontrivial to build, since Netflix doesn’t expose any JavaScript functions for manipulating video playback (or at least I couldn’t find them). That means, unfortunately, we have to resort to simulating user actions like clicking on buttons.
Playing and pausing
It didn’t take too long to figure out how to play and pause the video. We simply simulate a mouse click on the play/pause button:
var pause = function() { uiEventsHappening += 1; $('.player-play-pause.pause').click(); return delay(1)().then(hideControls).then(function() { uiEventsHappening -= 1; }); }; var play = function() { uiEventsHappening += 1; $('.player-play-pause.play').click(); return delay(1)().then(hideControls).then(function() { uiEventsHappening -= 1; }); };
The
uiEventsHappening
variable keeps track of how many simulated UI actions are in-flight. We don’t respond to user input unless uiEventsHappening === 0
, to distinguish between simulated user actions and real user actions.Hiding the playback controls
The Netflix player automatically shows the playback controls when the cursor is moved or any other user action occurs, and the controls hide automatically after a couple seconds. Unfortunately, naively firing a click event on the play/pause buttons causes the controls to stay visible forever. The fix is to simulate a subsequent click somewhere on the video:
var hideControls = function() { uiEventsHappening += 1; var player = $('#netflix-player'); var mouseX = 100; // relative to the document var mouseY = 100; // relative to the document var eventOptions = { 'bubbles': true, 'button': 0, 'screenX': mouseX - $(window).scrollLeft(), 'screenY': mouseY - $(window).scrollTop(), 'clientX': mouseX - $(window).scrollLeft(), 'clientY': mouseY - $(window).scrollTop(), 'offsetX': mouseX - player.offset().left, 'offsetY': mouseY - player.offset().top, 'pageX': mouseX, 'pageY': mouseY, 'currentTarget': player[0] }; player[0].dispatchEvent(new MouseEvent('mousemove', eventOptions)); return delay(1)().then(function() { uiEventsHappening -= 1; }); };
That magical spell was obtained by lots of trial and error. But that’s not even the hackiest part.
Seeking in the video
Controlling the playback position—the main point of Netflix Party—was by far the most difficult part. It took me two days to come up with this wretched incantation:
var showControls = function() { uiEventsHappening += 1; var scrubber = $('#scrubber-component'); var eventOptions = { 'bubbles': true, 'button': 0, 'currentTarget': scrubber[0] }; scrubber[0].dispatchEvent(new MouseEvent('mousemove', eventOptions)); return delay(10)().then(function() { uiEventsHappening -= 1; }); }; var seek = function(milliseconds) { uiEventsHappening += 1; var eventOptions, scrubber; return showControls().then(function() { // compute the parameters for the mouse events scrubber = $('#scrubber-component'); var factor = milliseconds / getDuration(); var mouseX = scrubber.offset().left + Math.round(scrubber.width() * factor); // relative to the document var mouseY = scrubber.offset().top + scrubber.height() / 2; // relative to the document eventOptions = { 'bubbles': true, 'button': 0, 'screenX': mouseX - $(window).scrollLeft(), 'screenY': mouseY - $(window).scrollTop(), 'clientX': mouseX - $(window).scrollLeft(), 'clientY': mouseY - $(window).scrollTop(), 'offsetX': mouseX - scrubber.offset().left, 'offsetY': mouseY - scrubber.offset().top, 'pageX': mouseX, 'pageY': mouseY, 'currentTarget': scrubber[0] }; // make the "trickplay preview" show up scrubber[0].dispatchEvent(new MouseEvent('mouseover', eventOptions)); }).then(delay(10)).then(function() { // simulate a click on the scrubber scrubber[0].dispatchEvent(new MouseEvent('mousedown', eventOptions)); scrubber[0].dispatchEvent(new MouseEvent('mouseup', eventOptions)); scrubber[0].dispatchEvent(new MouseEvent('mouseout', eventOptions)); }).then(delay(1)).then(hideControls).then(function() { uiEventsHappening -= 1; }); };
Here are the steps it takes:
- Jiggle the mouse so the playback controls appear. Wait 10ms for the UI to respond. 1ms is apparently not enough.
- Simulate the cursor hovering over the playback slider so the thumbnail preview appears, because apparently the playback slider doesn’t respond to input unless the preview is visible. Wait 10ms for the UI to respond.
- Click on the playback slider (called a “scrubber control” in the minified Netflix code) at the appropriate position to seek the video.
- Invoke the
hideControls()
magic from above.
The above snippet is almost certainly the worst code I have ever written.
Synchronization
First, we have this helper function that all Ajax requests go through:
var roundTripTimeRecent = []; var roundTripTimeMedian = 0; var localTimeMinusServerTimeRecent = []; var localTimeMinusServerTimeMedian = 0; var ajax = function(relativeUrl, method, data) { return new Promise(function(resolve, reject) { var startTime = (new Date()).getTime(); $.ajax({ url: 'https://www.netflixparty.com' + relativeUrl, method: method, data: method === 'POST' ? JSON.stringify(data) : data }).always(function(data, textStatus, jqXHR) { // calculate round trip time roundTripTimeRecent.push((new Date()).getTime() - startTime); if (roundTripTimeRecent.length > 10) { roundTripTimeRecent.splice(0, roundTripTimeRecent.length - 10); } var sorted = roundTripTimeRecent.concat().sort(); roundTripTimeMedian = sorted[Math.floor(sorted.length / 2)]; if (data.lastActivity !== undefined) { // calculate client-server time offset localTimeMinusServerTimeRecent.push((new Date()).getTime() - (new Date(data.lastActivity)).getTime()); if (localTimeMinusServerTimeRecent.length > 10) { localTimeMinusServerTimeRecent.splice(0, localTimeMinusServerTimeRecent.length - 10); } sorted = localTimeMinusServerTimeRecent.concat().sort(); localTimeMinusServerTimeMedian = sorted[Math.floor(sorted.length / 2)] - Math.round(roundTripTimeMedian / 2); } }).done(resolve).fail(reject); }); };
Note the instrumentation. The
roundTripTimeMedian
variable stores an estimate for the round-trip delay time to the server. It is the median of the measurements from the last 10 Ajax requests. I chose a simple median filter because it is robust to outliers. The round-trip time is used to compensate for network latency (we simply divide it by 2 and assume the one-way latencies are symmetrical).The
localTimeMinusServerTimeMedian
variable estimates how the local time is different from the server time. Since it is measured, we don’t need to rely on accurate time zone information from the browser. Again, we take the median of the last 10 values to get a stable measurement. In the future, I might try a fancier clock synchronization protocol like NTP. But for now, the approximation is good enough.Whenever a user plays, pauses, or seeks the video, the event is sent to the server with a timestamp (
lastKnownTimeUpdatedAt
) in server time. The server only knows about its own clock. Each client is responsible for converting its own times into the reference frame of the server.Now, here’s the tricky part. Suppose the client receives some nominal playback position from the server. Seeking to that position, in general, takes nonzero time. Buffering can take several seconds. By the time you’ve jumped to that position, buffered, and started playing the video, the other clients are several seconds ahead!
My solution is to jump a little bit ahead of the nominal time (2 seconds in this case), and pause the video there while buffering in the background. A timer is set to wait there until the other clients have caught up, and then resume playback—and hopefully buffering is complete by then.
This has the added benefit of increasing the available temporal resolution. By just clicking on the scrubber, you get about 3-5 seconds of precision (a pixel covers about that much time depending on the window size and video length). But by jumping a little bit ahead and pausing for a moment to compensate, the resolution increases to a couple dozen milliseconds.
Conclusion
This was a fun (and, at times, stressful) weekend project, but now I can organize remote Netflix parties. Happy New Year!