The end of last Summer and much of Autumn was spent working on a project which came to be known as Palestine Remix (palestineremix.com).

An ambitious project conceived by Al Jazeera — its remit to take 17 documentaries on the subject of Palestine in four languages, augment them with written narratives and make the material both accessible and remixable.

PalestineRemix.com was to exist outside the usual confines of a news interactive. The idea was that it would act — in its own way — as an encyclopedic perspective on events and the history of Palestine.

Enter Hyperaudio

In order to make it remixable we decided to use a modified version of hyperaudio.js — a library we developed to create the hyperaud.io application. The idea was to make interactive transcripts and take advantage of the copy, paste, drag and drop functionality of the Hyperaudio Pad. This would allow people to express perspectives by (1) remixing the documentaries from the actual transcripts and/or (2) using social media to share selected parts of the transcripts and associated video.

Quite differently to how hyperaud.io works, PalestineRemix had to function entirely in the browser — that is to say independently from any server logic. This meant we couldn't store data in a database. Instead we stored the state of a remix in the browser's address bar.

Remixing.

We also made changes to the drag-and-drop remixing interface, so that by default paragraphs were selected, for fine-tuning these could be split once dropped.

Mobile

One of the requirements was to produce an interface that would well enough on mobile devices and tablets. This turned out to be quite a challenge and fixed deadlines dictated we make some tough calls. One decision was to remove the ability to remix content on mobile, although we do allow viewing and sharing.

Two screens from the mobile version.

There were two issues here — the creation of an alternative user experience that would make editing as intuitive on mobile as it is on desktop — and performance.

Performance was the predominant issue and so we started looking at and writing a lighter version of hyperaudio.js but this was considered beyond scope of this version of the project. We are talking about using something like that at some point.

Tools and Data

By far our biggest challenge was creating word-aligned transcripts in English, Arabic, Bosnian and Turkish. While services exist to word-align English (hyperaud.io currently also allows French, Spanish and German) we knew of no service that could deal with our other target languages.

Word alignment can be an innaccurate process and so results would have to be manually checked and corrected where needed.

So we decided to rely on subtitles and estimate word-level timings from these. Unfortunately some of the original subtitles were only available 'burned on' to the video, in this case Optimal Character Recognition (OCR) based software was created especially to extract those.

Where subtitles were available in a propriety format we transformed these to a subtitle (SRT) format — a common format that we could then run through the Hyperaudio Converter to get our word timings.

The idea was that this process should be as automatic as possible and could be used for other projects, but there were many many challenges working with, verifying and editing this amount of word data in four languages some of which flowed right to left!

With large quantities of data, search is all important and sp we created a tool that indexes all paragraphs in all documentaries using lunr.js — again all happening in the browser.

The Team

The team working on PalestineRemix include people who'd worked on Hyperaud.io and Al Jazeera staff and contractors. It was a fairly large and coordinated effort worked on by a crazy mix of film-makers, journalists, designers and developers. Much coffee was consumed.

Palestine Remix Team Caricature courtesy of Al Jazeera Network.

Give it a Whirl

It's worth taking a look at PalestineRemix.com just for the breadth and depth of the documentaries and narratives. If you use the remix part, please feel free to let us know what does and (more importantly) what doesn't work for you.

If you're interested in using hyperaudio.js in your own projects, don't hesitate to get in touch.

The Making of PALESTINE REMIX (30s Video)

Palestine Remix - Piotr Fedorczyk (Blog Post)