Annotations

Movie Annotations

Here you will find word and face annotations for all the movies screened for the NNDb.

Contents

1 12 Years a Slave
- 1.1 Word Annotations
- 1.2 Face Annotations
2 500 Days of Summer
- 2.1 Word Annotations
- 2.2 Face Annotations
3 Back to the Future
- 3.1 Word Annotations
- 3.2 Face Annotations
4 Citizenfour
- 4.1 Word Annotations
- 4.2 Face Annotations
5 Little Miss Sunshine
- 5.1 Word Annotations
- 5.2 Face Annotations
6 Pulp Fiction
- 6.1 Word Annotations
- 6.2 Face Annotations
7 Split
- 7.1 Word Annotations
- 7.2 Face Annotations
8 The Prestige
- 8.1 Word Annotations
- 8.2 Face Annotations
9 The Shawshank Redemption
- 9.1 Word Annotations
- 9.2 Face Annotations
10 The Usual Suspects
- 10.1 Word Annotations
- 10.2 Face Annotations

Words and Faces were annotated in the movies using fully automated approaches. For Words, the audio track and the subtitle track were extracted from each movie. The audio file was input into ‘Amazon Transcribe’ from Amazon Web Services. To estimate the on and offset times of the words not transcribed, a script was written that first uses dynamic time warping (DTW) to align word onsets from the speech-to-text transcript to corresponding subtitle words in each individual subtitle page. Subtitle words that matched or that were similar to the transcriptions during the DTW procedure inherited the timing of the transcriptions. Remaining subtitle words not temporally labeled were then estimated, with different degrees of accuracy. For Faces we used the AWS ‘Amazon Rekognition’ API to obtain ML-based faces annotations, with no script modifications.

Please note that although accuracy is very high, there is still room for improvement. For example, Words have four accuracy levels, in order:

- "Matched" means that the word is 100% accurate

- "Continuous" means the word is in between two perfect matches, so still highly accurate

- "Partial" means that more than one word was found in between two perfect matches, so on/offset is somewhat accurate

- "Full" means that the word timing was completely estimated, therefore low accuracy If you are interested in analysing stimuli with ML, please do get in touch!