Neuroscience and the cinema
This week some papers on the neuroscience of viewing and remembering films.
Carvalho S, Leite J, Galdo-Álvarez S, and Gonçalves OF 2011 Psychophysiological correlates of sexually and non-sexually motivated attention to film clips in a workload task, PLoS One 6 (12): e29530.
Some authors have speculated that the cognitive component (P3) of the Event-Related Potential (ERP) can function as a psychophysiological measure of sexual interest. The aim of this study was to determine if the P3 ERP component in a workload task can be used as a specific and objective measure of sexual motivation by comparing the neurophysiologic response to stimuli of motivational relevance with different levels of valence and arousal. A total of 30 healthy volunteers watched different films clips with erotic, horror, social-positive and social-negative content, while answering an auditory oddball paradigm. Erotic film clips resulted in larger interference when compared to both the social-positive and auditory alone conditions. Horror film clips resulted in the highest levels of interference with smaller P3 amplitudes than erotic and also than social-positive, social-negative and auditory alone condition. No gender differences were found. Both horror and erotic film clips significantly decreased heart rate (HR) when compared to both social-positive and social-negative films. The erotic film clips significantly increased the skin conductance level (SCL) compared to the social-negative films. The horror film clips significantly increased the SCL compared to both social-positive and social-negative films. Both the highly arousing erotic and non-erotic (horror) movies produced the largest decrease in the P3 amplitude, a decrease in the HR and an increase in the SCL. These data support the notion that this workload task is very sensitive to the attentional resources allocated to the film clip, although they do not act as a specific index of sexual interest. Therefore, the use of this methodology seems to be of questionable utility as a specific measure of sexual interest or as an objective measure of the severity of Hypoactive Sexual Desire Disorder.
Frings L, Mader I, and Hüll M 2010 Watching TV news as a memory task – brain activation and age effects, BMC Neuroscience 11: 106.
Neuroimaging studies which investigate brain activity underlying declarative memory processes typically use artificial, unimodal laboratory stimuli. In contrast, we developed a paradigm which much more closely approximates real-life situations of information encoding.
In this study, we tested whether ecologically valid stimuli – clips of a TV news show – are apt to assess memory-related fMRI activation in healthy participants across a wide age range (22-70 years). We contrasted brain responses during natural stimulation (TV news video clips) with a control condition (scrambled versions of the same clips with reversed audio tracks). After scanning, free recall performance was assessed.
The memory task evoked robust activation of a left-lateralized network, including primarily lateral temporal cortex, frontal cortex, as well as the left hippocampus. Further analyses revealed that – when controlling for performance effects – older age was associated with greater activation of left temporal and right frontal cortex.
We demonstrate the feasibility of assessing brain activity underlying declarative memory using a natural stimulation paradigm with high ecological validity. The preliminary result of greater brain activation with increasing age might reflect an attempt to compensate for decreasing episodic memory capacity associated with aging.
Furman O, Dorfman N, Hasson U, Davachi L, and Dudai Y 2007 They saw a movie: long-term memory for an extended audiovisual narrative, Learning and Memory 14 (6): 457-467.
We measured long-term memory for a narrative film. During the study session, participants watched a 27-min movie episode, without instructions to remember it. During the test session, administered at a delay ranging from 3 h to 9 mo after the study session, long-term memory for the movie was probed using a computerized questionnaire that assessed cued recall, recognition, and metamemory of movie events sampled ∼20 sec apart. The performance of each group of participants was measured at a single time point only. The participants remembered many events in the movie even months after watching it. Analysis of performance, using multiple measures, indicates differences between recent (weeks) and remote (months) memory. While high-confidence recognition performance was a reliable index of memory throughout the measured time span, cued recall accuracy was higher for relatively recent information. Analysis of different content elements in the movie revealed differential memory performance profiles according to time since encoding. We also used the data to propose lower limits on the capacity of long-term memory. This experimental paradigm is useful not only for the analysis of behavioral performance that results from encoding episodes in a continuous real-life-like situation, but is also suitable for studying brain substrates and processes of real-life memory using functional brain imaging.
Hasson U, Furman O, Clark D, Dudai Y, and Davachi L 2008 Enhanced intersubject correlations during movie viewing correlate with successful episodic encoding, Neuron 57 (3): 452-462.
While much has been learned regarding the neural substrates supporting episodic encoding using highly controlled experimental protocols, relatively little is known regarding the neural bases of episodic encoding of real-world events. In an effort to examine this issue, we measured fMRI activity while observers viewed a novel TV sitcom. Three weeks later, subsequent memory (SM) for the narrative content of movie events was assessed. We analyzed the encoding data for intersubject correlations (ISC) based on subjects’ subsequent memory (ISC-SM) performance to identify brain regions whose BOLD response is significantly more correlated across subjects during portions of the movie that are successfully as compared to unsuccessfully encoded. These regions include the parahippocampal gyrus, superior temporal gyrus, anterior temporal poles, and the temporal-parietal junction. Further analyses reveal (1) that these correlated regions can display distinct activation profiles and (2) that the results seen with the ISC-SM analysis are complementary to more traditional linear models and allow analysis of complex time course data. Thus, the ISC-SM analysis extends traditional subsequent memory findings to a rich, dynamic and more ecologically valid situation.
Hasson U, Malach R, and Heeger DJ 2010 Reliability of cortical activity during natural stimulation, Trends in Cognitive Sciences 14 (1): 40-48.
Response reliability is complementary to more conventional measurements of response amplitudes, and can reveal phenomena that response amplitudes do not. Here we review studies that measured reliability of cortical activity within or between human subjects in response to naturalistic stimulation (e.g., free viewing of movies). Despite the seemingly uncontrolled nature of the task, some of these complex stimuli evoke highly reliable, selective, and time-locked activity in many brain areas, including some brain regions that often do not show much response modulation with conventional experimental protocols. This activity provides an opportunity to address novel questions concerning natural vision, temporal scale of processing, memory, and the neural basis of inter-group differences.
Jääskeläinen IP, Koskentalo K, Balk MH, Autti T, Kauramäki J, Pomren C, and Sams M 2008 Inter-Subject Synchronization of Prefrontal Cortex Hemodynamic Activity During Natural Viewing, The Open Neuroimaging Journal 2: 14-19.
Hemodynamic activity in occipital, temporal, and parietal cortical areas were recently shown to correlate across subjects during viewing of a 30-minute movie clip. However, most of the frontal cortex lacked between-subject correlations. Here we presented 12 healthy naïve volunteers with the first 72 minutes of a movie (“Crash”, 2005, Lions Gate Films) outside of the fMRI scanner to involve the subjects in the plot of the movie, followed by presentation of the last 36 minutes during fMRI scanning. We observed significant between-subjects correlation of fMRI activity in especially right hemisphere frontal cortical areas, in addition to the correlation of activity in temporal, occipital, and parietal areas. It is possible that this resulted from the subjects following the plot of the movie and being emotionally engaged in the movie during fMRI scanning. We further show that probabilistic independent component analysis (ICA) reveals meaningful activations in individual subjects during natural viewing.
And following on from this research:
Kauppi J-P, Jääskeläinen IP, Sams M, and Tohka J 2010 Inter-subject correlation of brain hemodynamic responses during watching a movie: localization in space and frequency, Frontiers in Neuroinformatics 4: 5.
Cinema is a promising naturalistic stimulus that enables, for instance, elicitation of robust emotions during functional magnetic resonance imaging (fMRI). Inter-subject correlation (ISC) has been used as a model-free analysis method to map the highly complex hemodynamic responses that are evoked during watching a movie. Here, we extended the ISC analysis to frequency domain using wavelet analysis combined with non-parametric permutation methods for making voxel-wise statistical inferences about frequency-band specific ISC. We applied these novel analysis methods to a dataset collected in our previous study where 12 subjects watched an emotionally engaging movie “Crash” during fMRI scanning. Our results suggest that several regions within the frontal and temporal lobes show ISC predominantly at low frequency bands, whereas visual cortical areas exhibit ISC also at higher frequencies. It is possible that these findings relate to recent observations of a cortical hierarchy of temporal receptive windows, or that the types of events processed in temporal and prefrontal cortical areas (e.g., social interactions) occur over longer time periods than the stimulus features processed in the visual areas. Software tools to perform frequency-specific ISC analysis, together with a visualization application, are available as open source Matlab code.
Wang HX, Freeman J, Merriam EP, Hasson U, and Heeger DJ 2012 Temporal eye movement strategies during naturalistic viewing, Journal of Vision 12 (1): 16.
The deployment of eye movements to complex spatiotemporal stimuli likely involves a variety of cognitive factors. However, eye movements to movies are surprisingly reliable both within and across observers. We exploited and manipulated that reliability to characterize observers’ temporal viewing strategies while they viewed naturalistic movies. Introducing cuts and scrambling the temporal order of the resulting clips systematically changed eye movement reliability. We developed a computational model that exhibited this behavior and provided an excellent fit to the measured eye movement reliability. The model assumed that observers searched for, found, and tracked a point of interest and that this process reset when there was a cut. The model did not require that eye movements depend on temporal context in any other way, and it managed to describe eye movements consistently across different observers and two movie sequences. Thus, we found no evidence for the integration of information over long time scales (greater than a second). The results are consistent with the idea that observers employ a simple tracking strategy even while viewing complex, engaging naturalistic stimuli.
Posted on April 26, 2012, in Cognitive Film Theory, Film Studies, Film Theory, Neuroscience and tagged Cognitive Film Theory, Film Studies, Film Theory, Neuroscience. Bookmark the permalink. Leave a comment.