Exploratory data analysis and film form

Following on from my earlier posts on the editing structure of slasher films, this week I have a draft of a paper that combines my early observations (much re-written) along with an analysis of the relationship between editing and the narrative structure of Friday the Thirteenth (1980)

Exploratory data analysis and film form: The editing structure of slasher films

We analyse the dynamic editing structure of four slasher films released between 1978 and 1983 with simple ordinal time series methods. We show the order structure matrix is a useful exploratory data analytical method for revealing the editing structure of motion pictures without requiring a priori assumptions about the objectives of a film. Comparing the order structure matrices of the four films, we find slasher films share a common editing pattern closely comprising multiple editing regimes with change points between editing patterns occur with large changes in mood and localised clusters of shorter and longer takes are associated with specific narrative events. The multiple editing regimes create different types of frightening experiences for the viewer with slower edited passages creating a pervading sense of foreboding and rapid editing linked to the frenzied violence of body horror, while the interaction of these two modes of expression intensifies the emotional experience of watching a slasher film.

The paper can be accessed here: Nick Redfern – The Editing Structure of Slasher Films.

The shot length data for all four films can be accessed as a single Excel file: Nick Redfern – Slasher Films.

Analysing the editing structure of these slasher films is only part of this paper. Another goal was to outline exploratory data analysis as a data-driven approach to understanding film style that avoids a specific problem of existing ways of thinking about film style.

Existing methods of analysing film style make a priori assumptions about the functions of style and then provide examples to support this assertion. This runs the risk of begging the question and circulus in probando, in which the researcher’s original assumption is used as a basis for selecting the pertinent relations of film style which are then used to justify the basis for making assumptions about the functions of film style. We would like to avoid such logically flawed reasoning whilst also minimising the risk that we will miss pertinent relations that did not initially occur to us. By adopting a data-driven approach we can derive the functions of film style by studying the elements themselves without the need to make any such a priori assumptions. Exploratory data analysis (EDA) allows us to do this by forcing us to attend to the data on its own terms.

Although this is a method developed within statistics, EDA can be applied not just to numerical data but to any situation where we need to understand the phenomenon before us. For example, I had not noticed that the number of scenes between hallucinations in Videodrome reduces by constant factor until I sat down and wrote out the narrative structure of the film (see here).

Two very useful references are:

Behrens JT 1997 Principles and practices of exploratory data analysis, Psychological Methods 2 (2): 131-160.

Ellison AM 1993 Exploratory data analysis and graphic display, in SM Scheiner and J Gurevitch (eds.) Design and Analysis of Ecological Experiments. New York: Chapman & Hall: 14-45.

In this paper I discuss some relations between editing and the emotional experience of watching slasher films, and below are listed some interesting references that follow on from last week’s collection of paper on neuroscience and the cinema:

Bradley MM, Codispoti M, Cuthbert BN, and Lang PJ 2001 Emotion and motivation I: defensive and appetitive reactions in picture processing, Emotion 1 (3): 276-298.

Bradley MM, Lang PJ, and Cuthbert BN 1993 Emotion, novelty, and the startle reflex: habituation in humans, Behavioural Neuroscience 107 (6): 970-980.

Lang PJ, Bradley MM, and Cuthbert BN 1998 Emotion, motivation, and anxiety: brain mechanisms and psychophysiology, Biological Psychiatry 44 (12): 1248-1263.

Lang PJ, Davis M, and Öhman A 2000 Fear and anxiety: animal models and human cognitive psychophysiology, Journal of Affective Disorders 61 (3): 137-159.

Willems RM, Clevis K, and Hagoort P 2011 Add a picture for suspense: neural correlates of the interaction between language and visual information in the perception of fear, Social Cognition and Affective Neuroscience 6 (4): 404-416.

Neuroscience and the cinema

This week some papers on the neuroscience of viewing and remembering films.

Carvalho S, Leite J, Galdo-Álvarez S, and Gonçalves OF 2011 Psychophysiological correlates of sexually and non-sexually motivated attention to film clips in a workload task, PLoS One 6 (12): e29530.

Some authors have speculated that the cognitive component (P3) of the Event-Related Potential (ERP) can function as a psychophysiological measure of sexual interest. The aim of this study was to determine if the P3 ERP component in a workload task can be used as a specific and objective measure of sexual motivation by comparing the neurophysiologic response to stimuli of motivational relevance with different levels of valence and arousal. A total of 30 healthy volunteers watched different films clips with erotic, horror, social-positive and social-negative content, while answering an auditory oddball paradigm. Erotic film clips resulted in larger interference when compared to both the social-positive and auditory alone conditions. Horror film clips resulted in the highest levels of interference with smaller P3 amplitudes than erotic and also than social-positive, social-negative and auditory alone condition. No gender differences were found. Both horror and erotic film clips significantly decreased heart rate (HR) when compared to both social-positive and social-negative films. The erotic film clips significantly increased the skin conductance level (SCL) compared to the social-negative films. The horror film clips significantly increased the SCL compared to both social-positive and social-negative films. Both the highly arousing erotic and non-erotic (horror) movies produced the largest decrease in the P3 amplitude, a decrease in the HR and an increase in the SCL. These data support the notion that this workload task is very sensitive to the attentional resources allocated to the film clip, although they do not act as a specific index of sexual interest. Therefore, the use of this methodology seems to be of questionable utility as a specific measure of sexual interest or as an objective measure of the severity of Hypoactive Sexual Desire Disorder.

Frings L, Mader I, and Hüll M 2010 Watching TV news as a memory task – brain activation and age effects, BMC Neuroscience 11: 106.

Background

Neuroimaging studies which investigate brain activity underlying declarative memory processes typically use artificial, unimodal laboratory stimuli. In contrast, we developed a paradigm which much more closely approximates real-life situations of information encoding.

Methods

In this study, we tested whether ecologically valid stimuli – clips of a TV news show – are apt to assess memory-related fMRI activation in healthy participants across a wide age range (22-70 years). We contrasted brain responses during natural stimulation (TV news video clips) with a control condition (scrambled versions of the same clips with reversed audio tracks). After scanning, free recall performance was assessed.

Results

The memory task evoked robust activation of a left-lateralized network, including primarily lateral temporal cortex, frontal cortex, as well as the left hippocampus. Further analyses revealed that – when controlling for performance effects – older age was associated with greater activation of left temporal and right frontal cortex.

Conclusion

We demonstrate the feasibility of assessing brain activity underlying declarative memory using a natural stimulation paradigm with high ecological validity. The preliminary result of greater brain activation with increasing age might reflect an attempt to compensate for decreasing episodic memory capacity associated with aging.

Furman O, Dorfman N, Hasson U, Davachi L, and Dudai Y 2007 They saw a movie: long-term memory for an extended audiovisual narrative, Learning and Memory 14 (6): 457-467.

We measured long-term memory for a narrative film. During the study session, participants watched a 27-min movie episode, without instructions to remember it. During the test session, administered at a delay ranging from 3 h to 9 mo after the study session, long-term memory for the movie was probed using a computerized questionnaire that assessed cued recall, recognition, and metamemory of movie events sampled ∼20 sec apart. The performance of each group of participants was measured at a single time point only. The participants remembered many events in the movie even months after watching it. Analysis of performance, using multiple measures, indicates differences between recent (weeks) and remote (months) memory. While high-confidence recognition performance was a reliable index of memory throughout the measured time span, cued recall accuracy was higher for relatively recent information. Analysis of different content elements in the movie revealed differential memory performance profiles according to time since encoding. We also used the data to propose lower limits on the capacity of long-term memory. This experimental paradigm is useful not only for the analysis of behavioral performance that results from encoding episodes in a continuous real-life-like situation, but is also suitable for studying brain substrates and processes of real-life memory using functional brain imaging.

Hasson U, Furman O, Clark D, Dudai Y, and Davachi L 2008 Enhanced intersubject correlations during movie viewing correlate with successful episodic encoding, Neuron 57 (3): 452-462.

While much has been learned regarding the neural substrates supporting episodic encoding using highly controlled experimental protocols, relatively little is known regarding the neural bases of episodic encoding of real-world events. In an effort to examine this issue, we measured fMRI activity while observers viewed a novel TV sitcom. Three weeks later, subsequent memory (SM) for the narrative content of movie events was assessed. We analyzed the encoding data for intersubject correlations (ISC) based on subjects’ subsequent memory (ISC-SM) performance to identify brain regions whose BOLD response is significantly more correlated across subjects during portions of the movie that are successfully as compared to unsuccessfully encoded. These regions include the parahippocampal gyrus, superior temporal gyrus, anterior temporal poles, and the temporal-parietal junction. Further analyses reveal (1) that these correlated regions can display distinct activation profiles and (2) that the results seen with the ISC-SM analysis are complementary to more traditional linear models and allow analysis of complex time course data. Thus, the ISC-SM analysis extends traditional subsequent memory findings to a rich, dynamic and more ecologically valid situation.

Hasson U, Malach R, and Heeger DJ 2010 Reliability of cortical activity during natural stimulation, Trends in Cognitive Sciences 14 (1): 40-48.

Response reliability is complementary to more conventional measurements of response amplitudes, and can reveal phenomena that response amplitudes do not. Here we review studies that measured reliability of cortical activity within or between human subjects in response to naturalistic stimulation (e.g., free viewing of movies). Despite the seemingly uncontrolled nature of the task, some of these complex stimuli evoke highly reliable, selective, and time-locked activity in many brain areas, including some brain regions that often do not show much response modulation with conventional experimental protocols. This activity provides an opportunity to address novel questions concerning natural vision, temporal scale of processing, memory, and the neural basis of inter-group differences.

Jääskeläinen IP, Koskentalo K, Balk MH, Autti T, Kauramäki J, Pomren C, and Sams M 2008 Inter-Subject Synchronization of Prefrontal Cortex Hemodynamic Activity During Natural Viewing, The Open Neuroimaging Journal 2: 14-19.

Hemodynamic activity in occipital, temporal, and parietal cortical areas were recently shown to correlate across subjects during viewing of a 30-minute movie clip. However, most of the frontal cortex lacked between-subject correlations. Here we presented 12 healthy naïve volunteers with the first 72 minutes of a movie (“Crash”, 2005, Lions Gate Films) outside of the fMRI scanner to involve the subjects in the plot of the movie, followed by presentation of the last 36 minutes during fMRI scanning. We observed significant between-subjects correlation of fMRI activity in especially right hemisphere frontal cortical areas, in addition to the correlation of activity in temporal, occipital, and parietal areas. It is possible that this resulted from the subjects following the plot of the movie and being emotionally engaged in the movie during fMRI scanning. We further show that probabilistic independent component analysis (ICA) reveals meaningful activations in individual subjects during natural viewing.

And following on from this research:

Kauppi J-P, Jääskeläinen IP, Sams M, and Tohka J 2010 Inter-subject correlation of brain hemodynamic responses during watching a movie: localization in space and frequency, Frontiers in Neuroinformatics 4: 5.

Cinema is a promising naturalistic stimulus that enables, for instance, elicitation of robust emotions during functional magnetic resonance imaging (fMRI). Inter-subject correlation (ISC) has been used as a model-free analysis method to map the highly complex hemodynamic responses that are evoked during watching a movie. Here, we extended the ISC analysis to frequency domain using wavelet analysis combined with non-parametric permutation methods for making voxel-wise statistical inferences about frequency-band specific ISC. We applied these novel analysis methods to a dataset collected in our previous study where 12 subjects watched an emotionally engaging movie “Crash” during fMRI scanning. Our results suggest that several regions within the frontal and temporal lobes show ISC predominantly at low frequency bands, whereas visual cortical areas exhibit ISC also at higher frequencies. It is possible that these findings relate to recent observations of a cortical hierarchy of temporal receptive windows, or that the types of events processed in temporal and prefrontal cortical areas (e.g., social interactions) occur over longer time periods than the stimulus features processed in the visual areas. Software tools to perform frequency-specific ISC analysis, together with a visualization application, are available as open source Matlab code.

Wang HX, Freeman J, Merriam EP, Hasson U, and Heeger DJ 2012 Temporal eye movement strategies during naturalistic viewing, Journal of Vision 12 (1): 16.

The deployment of eye movements to complex spatiotemporal stimuli likely involves a variety of cognitive factors. However, eye movements to movies are surprisingly reliable both within and across observers. We exploited and manipulated that reliability to characterize observers’ temporal viewing strategies while they viewed naturalistic movies. Introducing cuts and scrambling the temporal order of the resulting clips systematically changed eye movement reliability. We developed a computational model that exhibited this behavior and provided an excellent fit to the measured eye movement reliability. The model assumed that observers searched for, found, and tracked a point of interest and that this process reset when there was a cut. The model did not require that eye movements depend on temporal context in any other way, and it managed to describe eye movements consistently across different observers and two movie sequences. Thus, we found no evidence for the integration of information over long time scales (greater than a second). The results are consistent with the idea that observers employ a simple tracking strategy even while viewing complex, engaging naturalistic stimuli.