Other posts on this blog have listed links to papers addressing the indexing of video by analysis of the stylistic components of the video itself (shot lengths, colour, sound energy, etc). An alternative approach is not to look at the video but to look at the viewer watching the video or to look at the viewer’s brain whilst they watch. The papers presented adopt a range of approaches to understanding films by understanding viewers and to understanding viewers by understanding films. They are an example of the very interesting empirical research taking place across diverse subject areas that has yet to make any impact on film studies.
The linked-to version may not be the final published version.
Calcanis C, Callaghan V, Gardner M, and Walker M 2008 Towards end-user physiological profiling for video recommendation engines, 4th International Conference on Intelligent Environments, 21-22 July 2008, Seattle, USA.
This paper describes research aimed at creating intelligent video recommendation engines for broadband media services in digital homes. The aim of our research is to harness physiological signals to characterise people’s video selection preferences which we plan to integrate into new generations of video recommendation engines. We describe an initial experiment aimed at determining whether videos produce useable variations in physiology and linking these with emotional changes elicited by video material. We discuss our results and consider the possibility of utilising physiological sensing methods to build profiles that can be treated as signatures. Finally, we conclude by describing the future directions of our work.
Canini L, Gilroy S, Cavazza M, Leonardi R, and Benini S 2010 Users’ response to affective film content: a narrative perspective, 8th International Workshop on Content-based Multimedia Indexing, 23-25 June, 2010, Grenoble, France.
In this paper, we take a human-centred view to the definition of the affective content of films. We investigate the relationship between users physiological response and multimedia features extracted from the movies, from the perspective of narrative evolution rather than by measuring average values. We found a certain dynamic correlation between arousal, derived from measures of Galvanic Skin Resistance during film viewing, and specific multimedia features in both sound and video domains. Dynamic physiological measurements were also consistent with post-experiment self-assessment by the subjects. These findings suggest that narrative aspects (including staging) are central to the understanding of video affective content, and that direct mapping of video features to emotional models taken from psychology may not capture these phenomena in a straightforward manner.
Cooray SH, Hyowon L, and O’Connor NE 2010 A user-centric system for home movie summarisation, 17th International Conference on Multimedia Modeling, 5-7 January 2011, Taipei, Taiwan.
In this paper we present a user-centric summarisation system that combines automatic visual-content analysis with user-interface design features as a practical method for home movie summarisation. The proposed summarisation system is designed in such a manner that the video segmentation results generated by the automatic content analysis tools are further subject to refinement through the use of an intuitive user-interface so that the automatically created summaries can be effectively tailored to each individual’s personal need. To this end, we study a number of content analysis techniques to facilitate the efficient computation of video summaries, and more specifically emphasise the need for employing an efficient and robust optical flow field computation method for sub-shot segmentation in home movies. Due to the subjectivity of video summarisation and the inherent challenges associated with automatic content analysis, we propose novel user-interface design features as a means to enable the creation of meaningful home movie summaries in a simple manner. The main features of the proposed summarisation system include the ability to automatically create summaries of different visual comprehension, interactively defining the target length of the desired summary, easy and interactive viewing of the content in terms of a storyboard, and manual refinement of the boundaries of the automatically selected video segments in the summary.
Joho H, Jose JM, Valenti R, and Sebe N 2009 Exploiting facial expressions for affective video summarisation, ACM International Conference on Image and Video Retrieval, 8-10 July, 2009, Santorini, Greece.
This paper presents an approach to affective video summarisation based on the facial expressions (FX) of viewers. A facial expression recognition system was deployed to capture a viewer’s face and his/her expressions. The user’s facial expressions were analysed to infer personalised affective scenes from videos. We proposed two models, pronounced level and expression’s change rate, to generate affective summaries using the FX data. Our result suggested that FX can be a promising source to exploit for affective video summaries that can be tailored to individual preferences.
Joho H, Staiano J, Sebe N, and Jose JM 2011 Looking at the viewer: analysing facial activities to detect personal highlights of multimedia contents, Multimedia Tools and Applications 51 (2): 505-523.
This paper presents an approach to detect personal highlights in videos based on the analysis of facial activities of the viewer. Our facial activity analysis was based on the motion vectors tracked on twelve key points in the human face. In our approach, the magnitude of the motion vectors represented a degree of a viewer’s affective reaction to video contents. We examined 80 facial activity videos recorded for ten participants, each watching eight video clips in various genres. The experimental results suggest that useful motion vectors to detect personal highlights varied significantly across viewers. However, it was suggested that the activity in the upper part of face tended to be more indicative of personal highlights than the activity in the lower part.
Peng W-T, Huang W-J, Chu W-T, Chou C-N, Chang W-Y, and Chang C-H, and Hung T-P 2009 A user experience model for home video summarization, 15th International Multimedia Modeling Conference on Advances in Multimedia Modelin, 7-9 January 2009, Sophia-Antipolis, France.
n this paper, we propose a novel system for automatically summarizing home videos based on a user experience model. The user experience model takes account of user’s spontaneous behaviors when viewing videos. Based on users’ reaction when viewing videos, we can construct a systematic framework to automate video summarization. In this work, we analyze the variations of viewer’s eye movement and facial expression when he or she watching the raw home video. We transform these behaviors into the clues of determining the important part of each video shot. With the aids of music analysis, the developed system automatically generates a music video (MV) style summarized home videos. Experiments show that this new type of editing mechanism can effectively generate home video summaries and can largely reduce the efforts of manual summarization.
Wang S and Hu Y 2010 Affective video analysis by using users’ EEG and subjective evaluation, International Conference on Kansei Engineering and Emotion Research, 2-4 March 2010, Paris.
This paper describes a research project conducted to study the relationship between videos and users’ induced physiological and psychological responses. Firstly, a set of 43 film clips are carefully chosen, and 20 subjects are invited to participate in our experiment. They watch several of chosen clips while their EEG signals are recorded synchronously. After each clip, the subject is required to report his real induced emotion using emotional valence, arousal, basic emotion category and intensity. Secondly, several classical movie features and EEG features are extracted, and feature selections are conducted by computing the correlation between each feature and the arousal or valence. Thirdly, selected movie features and EEG features are used to simulate the arousal and valence respectively by employing the linear relevance vector machine. Fourthly, selected movie features are used to simulate the EEG feature values, and vice verse. The results show that arousal/valence can be well estimated by either video features or EEG features. Apart from that, they also indicate that there exist certain relationship between the videos and induced EEG signals, and some relation models are acquired. Finally, clustering is conducted to map the emotion dimensions to emotion categories. Thus, the gap between videos and emotion categories, as well as the gap between the EEG and emotion categories, has been bridged to some extent. This result could provide a reference to applications in brain-computer interaction field.
Back in January, when I posted an bibliographical update of works published on cognitive film theory, I was asked if I would do an update of the larger bibliography of research in this area that appeared the year before. Six months later, I have finally done this, and the link below will take you to the new version as a pdf file:
This bibliography now contains 425 different items, which means that 70 new references (highlighted in red) have been added since the old version was put on-line. By ‘new’ I mean only that items did not appear in the earlier version, and though many of these are from as recent as 2010 some are much older. Again, this bibliography makes no claim to being exhaustive, but what is there should be accurate. (I’ve also tidied up a few things since the last version but these were very minor).
I haven’t included any references from 2011 becasue the year isn’t over yet, and you’ll just have to wait until next January for that update.
One area that I have also left to one side, but which could make a very large contribution to the bibliography is research on cognition that uses film clips to explore cognition but which is not related to film studies (i.e. the research doesn’t necessarily use commerical films). One area of considerable research using film in this way are studies of ‘affective style,’ and a handful of references with links to papers by Richard J Davidson are listed below as examples. I may decide at a later date to add such research to the bibliography, but for the time being it remains in limbo.
Davidson RJ 1994 Asymmetric brain function, affective style, and psychopathology: the role of early experience and plasticity, Development and Psychology 6: 741-758.
Davidson RJ 1998 Anterior electrophysiological asymmetries, emotion, and depression: conceptual and methodological conundrums, Psychophyisiology 35: 607-614.
Davidson RJ 2003 Darwin and the neural bases of emotion and affective style, Annals of the New York Academy of Sciences 1000: 316-336.
Davidson RJ2004 Well-being and affective style: neural substrates and biobehavioural correlates, Philosophical Transactions of the Royal Society of London B: Biological Sciences 359 (1449): 1395-1411.
This weeks post is a bibliography of materials on the subject of cognitive film theory I have amassed on and off over the past few years. Although it contains some 355 items it is neither exhaustive nor up to date, although it should be accurate (barring any changes in the URLs for web-based resources). I’m sure most of what is there is well-known to those interested in this area, but there is almost certainly something you will not have come across before.
The file can be downloaded here as a pdf: Nick Redfern – CognitiveFilmTheoryBibliography1-19.
Finally, to bring to your attention an interesting article I came across recently on the subject of Luchino Visconti and Federico Fellini in Frontiers of Neurology and Neuroscience and their series on the impact of neurological disorders on famous artists, which looks at the impact of strokes on the creativity of two of Italy’s greatest filmmakers.
Dieguez, S., Assal, G., Bogousslavsky, J. (2007) Visconti and Fellini: from left social neorealism to right-hemisphere stroke, Frontiers of Neurology and Neuroscience 22: 44-74.
The acclaimed Italian directors Luchino Visconti and Federico Fellini had very different life trajectories that led them to become major figures in the history of cinema. Similarities, however, can be found in their debuts with the neorealist genre, their personalities, creative styles and politicocultural involvement, and ultimately in the neurological disease that struck them at the end of their careers. Both suffered a right-hemispheric stroke that left them hemiplegic on the left side. We review their life and career to put that event into perspective, and then discuss its aftermath for both artists in the light of our current knowledge of right-hemispheric functions. Visconti showed a tremendous resilience following the accident and managed to direct several films and plays as an infirm, whereas Fellini had to put an end to his career but still was able to display his talents to the neuropsychologists that treated him. A speculative account is given of the links between right-hemispheric symptomatology and the premorbid personality of these highly prolific patients.
Two genres that have been significant in post-war Hollywood cinema are the conspiracy movie and the disaster film. With the end of World War II in the destruction of Hiroshima and Nagasaki and the nuclear arms race that followed, and the paranoia of the Cold War, it is conspiracy and disaster movies that have voiced America’s deepest fears about its relationship to the rest of the world on the one hand, and the struggle to define what is American and what is un-American at home.
From the point of view of someone interested in how cinema deals with these types of questions these genres are interesting because of the way they occur together and interact. Both genres deal with fundamentally the same problem: that our deepest fears may be realised – that the world is coming to an end and that the person we share our life is not who we think they are. Both these genres deal with anxiety, the prolonged, persistent, irrational belief that something (although we may not know what) is going to happen.
By conspiracy movie I mean a film in which there is some paranoid element that leads us to conclude that the world as we experience it is not the world as it is – films such as Invasion of the Body Snatchers (1954), Seven Days in May (1964), The Parallax View (1974), or Shadow Conspiracy (1997) are good examples of how people are not what they seem, that the US government is ruled by the military, or that through mind control can be used to turn individuals into assassins. Timothy Melley refers to this pervasive strain in American popular culture ‘agency panic:’
An intense anxiety about an apparent loss of autonomy or self-control – the conviction that one’s actions are being controlled by someone else, that one has been ‘constructed’ by powerful agents (2001: 62).
Agency panic provides a model of conspiracy in American culture based around a notion of diminished human agency in which the individual is subject to a broad array of social controls (the conspiracy). Conspiracy theory, in Melley’s view is a defence of the integrity of the self in the face of anxiety about the nature of individual action. As I have discussed elsewhere, the conspiracy movie is characterised by the emotional response of anxiety. Agency panic from The Hidden Persuaders, to the Unabomber’s manifesto, and to Invasion of the Body Snatchers is such an emotional response.
Disaster movies are harder to define as the cause of the disaster may vary considerably, but are characteristic by a certain scale of their events – disasters should be big, especially in Hollywood – or by their magnified impact on a small but largely self-contained social group (i.e. Jurassic Park (1993), Airport ’77 (1977)). Global Pandemics (Panic in the Streets (1950), Outbreak (1997)), geological disaster (Earthquake (1974), Volcano (1997)), or the complete and utter destruction of the Earth (Armageddon (1997)) are all recurrent topics. There is also a sub-genre of films in which mass transit systems are out to get Americans – having watched films such as Airport (1970), Speed (1994), or even Titanic (1997) is it any wonder that investment in US public transport is lacking and that that car is supreme?
Like conspiracy films, disaster movies put the viewer in the position of being unable to control a situation: earthquakes, swarms of killer bees, meteors cannot be reasoned with. Something terrible will happen and we will not be able to control it. The potentiality of the disaster is the terrible thing – it is this that produces in the viewer a sense of anxiety.
Disaster movies are an essentially earthbound form: they operate, almost by definition, within the realm of the possible. People must believe ‘it’ could – indeed, very well might – happen to them (Roddick 1980: 246).
There is an initial loss of agency in the disaster movie leading to panic – but, and this is where the genre diverges from the conspiracy film, that loss of agency can ultimately be recovered. The world may never be the same again but human beings survive. We will be able to land the plane safely, the meteor will be destroyed (at the cost of Bruce Willis), the aliens will be defeated by a computer virus (which in no way plagiarises The War of the Worlds) – there will be a plan and that plan will lead to the continuation of the human race.
There are also some films that involve both conspiracies and disasters: Deep Impact (1997), for example, starts off with a journalist trying to uncover what she thinks is a conspiracy but in fact uncovers a disaster (a meteor heading for earth); while in The China Syndrome (1979) California is a risk because of cover-up at a nuclear power plant.
To chart the changing impact of the conspiracy film and the disaster movie I have searched books, databases and the internet to find Hollywood’s output since the end of World War II and have come up with two samples on which I am going to base my analysis of anxiety in Hollywood cinema. I have identified some 93 conspiracy films and 102 disaster movies produced in Hollywood from 1947 to 2006 inclusive (not including TV movies, straight-to-video), and plotting the number of these types of films released by 5 year periods we can see some clear trends (Figure 1). Obviously, cycles of films do not fit neatly into five year periods, and this data set will continue to grow as I carry on the research but it is a useful guide.
FIGURE 1 Hollywood conspiracy and disaster films released from 1947 to 2006
Figure 1 shows that:
- There have been three major cycles of conspiracy movies: the ‘red scare movies’ of the early Cold War (1947-1959, 30 films); the New Hollywood films, in which the individual is threatened by state institutions (1965-1979, 25 films); and from 1990 to 2006, which includes the nostalgia/history films of the 1990s (e.g. JFK (1991)), bog-standard conspiracy genre-fare (e.g. Shadow Conspiracy), and new millennium films that deal primarily with the problematic nature of memory (Paycheck (2003)), identity and agency (The Bourne Identity (2002)), and reality (The Matrix trilogy (1999-2003).
- Although the conspiracy film never disappears, it does drop off markedly in the early 1960s (8 films) and the 1980s (7 films).
- The disaster movie is more of a constant feature of post-war Hollywood cinema, and does not have such large swings in popularity as the conspiracy film. Nonetheless, there are clusters of disaster movies – in the 1950s and 1960s there are 15 and 14 films, respectively; in the 1970s this increases to 28 films; there are 13 films in the 1980s (almost all of which are released in the first half of the decade); before another increase in the 1990s to 24 films (of which 18 come in the second half of the decade); and 8 in the 2000s, with 7 released from 2000-2004.
- The peak years for disaster movies are 1979 (7 films) and 1997 (8 films).
- Of 102 disaster movies, the disasters are: alien invasion (5 films), disease (10 films), man-made disasters (i.e. fire) (5 films), natural disasters (35 films), nuclear disasters (11 films), and disasters involving some form of transport (36 films).
- Of the 35 natural disaster films 3 involve avalanches, 9 involve some type of fauna (including bees (The Swarm (1978) and dinosaurs (Jurassic Park (1993))), 13 are geological (i.e. volcanoes, earthquakes), 5 involve meteors, and 5 involve some form of extreme weather event from tornadoes to hurricanes to global warming.
- Of the 36 transport disaster films 2 involve buses, 10 involve boats, and 24 feature aircraft disasters.
In summary, the genres of the conspiracy film and the disaster movie a born in the early years of the Cold War and their fortunes broadly coincide as their popularity waxes and wanes – particularly in the 1970s and 1990s. That they should occur together is, I think, due to the shared basis in exploring the our anxiety about the nature of the world and the potential for action in the face of events that exceed our control.
Timothy Melley, ‘Agency Panic and the Culture of Conspiracy,’ in Peter Knight (ed.) Conspiracy Nation: The Politics of Paranoia in Post-war America. New York and London: New York University Press, 2001: 57-81.
Nick Roddick, ‘Only the Stars Survive: Disaster Movies in the Seventies,’ in D. Brady (ed.) Performance and Politics in Popular Drama: Aspects of Popular Entertainment in Theatre, Film, and television 1800-1976. Cambridge: Cambridge University Press, 1980: 243-269.
A version of this post was presented to the CCM Research group at the University of Central Lancashire on 21 February 2007. It is presented here with the first part – a (not entirely satisfactory) discussion of ecological approaches to film theory – missing, and a new introduction. The model in figure 1 now seems incomplete and needs further development, but I think it still has some uses as a basic description of information and perception in the cinema.
Wade and Swanston point out that in order to come to a full understanding of vision it is necessary to ‘include an appreciation of the neurophysical processes that are initiated by the activity of light on the receptors of the eye. These involve the modification of light energy into nerve impulses and their transmission to areas at the back of the brain where they are analysed’ (1991: 59). In my opinion, it is precisely this ‘appreciation of neurophysical processes’ that should form the basis of film theory – not least because we need to be able to account for own experiences of the cinema in formulating hypotheses about it. A cognitive approach to film theory allows theorists to build self-reflexivity into their research. The Society for Cognitive Studies of the Moving Image promotes research in this area, with a particular emphasis on viewer’s emotional experiences. David Bordwell has recently written on the latest research to emerge here and here.
However, I have been repeatedly frustrated by the lack of a model of communication in the cinema that can account for the viewer’s perception of a film in terms of neurophysical processes. For example, Carl Plantinga (1999) has noted that ‘[o]ne of the least explored aspects of film and television is their sensory means of communication’ (239) – but then on the next page he asserts that ‘[c]learly, film directors use the human face to communicate information about the emotions of characters’ (240, original emphasis). These are the only two occasions Plantinga mentions communication – but he goes from a topic that is one of the ‘least explored’ to the self-evident clarity of the assertion that films communicate emotions to the viewer. Plantinga never defines what he means by communication or information, though we may infer that he means the transmission from screen to spectator of meaningful content about the emotional states of a particular character. However, as Thayer has pointed out, such communication is impossible:
The ways in which we traditionally conceive of communication – those being inadequate and untenable – stand as obstacles to more adequate and more potent ways of conceiving of communication … Those preconceptions, our traditional concepts of communication, are often insidious. ‘Communication is the “transfer of meaning”’ has an appealing ring to it. But since none of our receptors is capable of receiving ‘meaning,’ the notion of transfer is a flagrantly untenable one (Thayer 1979: 10).
The work of Paul Bach-y-Rita (2002, 2003) at the University of Wisconsin deserves special mention here. Bach-y-Rita and his fellow researchers have used televisual sensory substitution systems to restore the modality of sight to vision impaired individuals, and he has addressed the impact of restored sight and emotion content:
we found that while experienced blind TVSS subjects could perceive faces and printed images, they were very disappointed when perception was not accompanied by qualia: A Playboy centerfold carried no emotional message, and the face of a girl-friend or a wife created an unpleasant response since it did not convey an affective message. We consider this to be comparable to the lack of emotional contact of curse-words in a language that has been learned as an adult. It is possible that the emotional content could be developed over a long period of usage. On the other hand, a blind infant using a vision substitution system smiles when he recognizes a toy and reaches for it, and a blind 10-year-old child perceiving a flickering candle flame by means of a TVSS is enchanted (Bach-y-Rita et al. 2003: 293).
This quote is very suggestive for cognitive film theorists working in the area of emotion. It raises a fundamental question: what is the nature of communication in the cinema? This paper explores this question through looking at information and the different forms it takes in the viewer’s experience of a motion picture.
Information in the cinema
Francis Crick points out that there ‘is one fact about the brain that is so obvious it is seldom mentioned: it is attached to the rest of the body and communicates with it. The nervous system receives information only from the various transducers in the body’ (1994: 81). This principle has long history and may be traced back to Johannes Müller’s law of specific nerve energies, which states that it no matter how a sensory system is stimulated, the resulting sensation will always be of the type appropriate to that system (Müller 1826). For example, the stimulation of the optic nerve will result in visual sensation regardless of whether that stimulation is by flashing light, by electric shock, or by pressure on the eye (Norrsell et al. 1999); and this sensation is dependent upon the part of the brain in which the sensory pathways terminate and not the stimulus. Thus we ‘see with the brain, not the eyes’ (Bach-y-Rita et al. 2003: 285) as the images that pass through the pupil and are focussed on the retina go no further: ‘The sole source of output from the retina to the rest of the brain is the action potentials arising from the million or so ganglion cells’ (Bear et al. 2007: 300). The brain has no independent reference as to the cause of electro-chemical signals that are transmitted along the optic nerve because the ‘response of a nerve cell does not encode the physical nature of the agents that caused its response. Encoded is only “how much” at this point on my body, but not “what”’ (Foerster  2003: 214).
Images in the cinema are comprised of variable physical properties in the pattern of silver salts of the film’s emulsion (Enticknap 2005: 203), so that light projected through a film and reflected by a screen is energy in the form of electromagnetic radiation and is experienced by the viewer as changes in the intensity of light and colour, and its position in the frame (Read 1998: 1). This light carries no qualitative information about the environment – there is only quantitative data about the energetic properties of the light. As Foerster points out ( 1981: 263): ‘the environment contains no information, the environment is as it is.’ The viewer, as a perceiving system, is capable of receiving light as energy as evolution has led to the development of a visual system that responds to changes in the quantitative properties of a stream of photons (e.g. hue, luminosity) without knowledge of the cause of such properties. The viewer, then, is open to energy (‘how much’) but is closed to information (‘what’) (Ashby 1956).
As sensory systems function by ‘transducing some type of environmental energy into a form that can be analysed by the cells in the central nervous system’ (Wade and Swanston 1991: 59), perception cannot be considered direct – it is mediated by the sensory and neurophysiological processes of the perceiver. Those processes begin with light being focussed onto the retina:
Light emitted by or reflected off objects in space can be imaged by the eye onto the retina. Light energy is first converted into membrane potential charges in the mosaic of photoreceptors. … photoreceptor membrane potential is converted into a chemical signal (the neurotransmitter glutamate), which is again converted into a membrane potential changes in the post-synaptic bi-polar horizontal cells. This process of electrical-chemical-electrical signalling repeats again and again, until the presence of light or dark or colour is finally converted to a charge in the action potential firing frequency of the ganglion cells (Bear et al. 2007: 306).
The light energy reflected by a cinema screen enters the viewer’s eye where it is converted into a pattern of stimulation, and is projected to the magnocellular and parvocellular layers of the lateral geniculate nucleus before being relayed to the visual cortex (Farah 2000). As quantitative information, the pattern of stimulation at the retina is syntactic (Shannon and Weaver 1949), and is transduced into the functional information by the firing of neurons in the visual cortex. There is a non-random correlation between these two types of information (Gulick  1990).
A perceiving system is not aware of this complex process of information transduction, only of the results of this process (Jackendoff 1987). Somehow – and this remains a mystery to philosophers, psychologists, and neuroscientists – a perceiver is conscious of these results and they are meaningful. The semantic information that is the content of consciousness is an emergent product of the mind, and cannot be reduced to syntactic relations: information only becomes meaningful when a perceiver is able to link it to information structures he or she already possesses (Stonier 1997). Such information, then, is highly context-dependent, and from this it must be concluded that the environment does not contain sufficient information to guide a perceiver’s behaviour. The question of meaning comes down to how a perceiving system generates units of experience and relates them to conceptual structures that from the basis for subsequent modes of acting and thinking (Redfern 2004). As semantic information cannot be distinguished from those modes it becomes pragmatic information when it is embedded in a social practice (Zoglauer 1996).
The transduction of light energy to syntactic to functional to semantic/pragmatic information creates a layered hierarchy of information levels in which higher information concepts depend on lower level information concepts but cannot be reduced to them (Zoglauer 1996). This relationship is represented in Figure 1. Though we are conscious only of information at the semantic/pragmatic level of this hierarchy it is essential that we include the lower information levels and do not marginalise the physical inputs to the viewer as a perceiving system or ignore the viewer’s neurophysiological processes of information transduction.
Figure 1 Information transduction in motion picture perception
Representation in the cinema
The concept of representation is central to both cognitive psychology and to the study of all media forms as well as more specifically the cinema. Representation is a term used in a wide variety of senses and can refer to any symbolic description of the world. Thus, from a psychological perspective, representation refers to mental images that correspond to objects that lie beyond a perceiver’s sensory systems; while in film studies, representation is used in reference to the reflection or distortion of the ‘real’ (Dyer 1985). Both these uses of representation are manifest in the application of cognitive models to the cinema: in watching, say, The Scarlet Pimpernel (Harold Young, 1934) the viewer has a mental image of the depiction of the Pimpernel as an English gentleman adventurer (Richards 1997) (see Figure 2).
If the above description of information transduction in motion picture perception is accepted then the concept of representation is of no use in accounting for the viewer’s experience of a film. As the viewer is, in informational terms, organisationally closed his or her conscious experience cannot be said to correspond to anything that exists independently of the viewer. As an organisationally closed system that interacts necessarily with its own states, the viewer has no external point of reference by which to judge the correspondence of mental images to the world: the viewer has no means of establishing a correspondence between his or her perception of The Scarlet Pimpernel and the film itself. If it is accepted that the viewer is organisationally closed then the sensory structure, patterns, or images he or she experiences are the viewer’s own construction, and the notion that they represent an aspect of the world has no empirical foundation. Glasersfeld (1995, 1999) argues that in the place of representation, we should use the term presentation, as this is closer to Kant’s deployment of vorstellung in ‘The Conflict of the Faculties,’ to refer to concepts that are generated by a perceiver: ‘the mind can only create only presentations of its own objects and not of the real things, that is, through these presentations and concepts, things cannot possibly be known as they might be in themselves’ (quoted in Glasersfeld 1995: 39-40). The qualitative aspects of the viewer’s experience are solely determined by the viewer, and his or her experience of The Scarlet Pimpernel is a mental presentation of the viewer that emerges as a result of a complex process of information transduction.
Figure 2 Leslie Howard as The Scarlett Pimpernel (1934)
The concept of constructions that do not correspond to an external reality does not imply epistemological solipsism. Lorenz (1941) argued that evolution has provided us with a perceptual system that allows us to operate in the absence of information about the ‘real world.’ This principle has been developed by evolutionary epistemologists (Campbell 1974) and radical constructivists (Glasersfeld 1995), who argue that constructions are adaptations that provide us with viable ways of thinking and acting in an environment (Sjölander 1999). This principle of adaptation is derived from the work of Jean Piaget (1937), who approached the construction of knowledge as a biologist. For Piaget, adaptation involves two complimentary and simultaneous processes: a cognising organism primarily seeks to organise experience in terms of the psychological structures (schemes) it already possesses, i.e., it seeks to assimilate experience; if the result of this process creates a perturbation the organism attempts to accommodate the error either by modifying an existing scheme or creating a new one. It is this balance between assimilation and accommodation that Piaget describes as adaptation. Knowledge is actively constructed, and is adapted to fit the environmental constraints that act on an organism in order to avoid internal contradictions and achieve equilibrium. Glasersfeld describes the principle of adaptation in radical constructivist thought:
[A]daptation is not an activity but the result of the elimination of all that is not adapted. Consequently, on the biological level, anything that manages to survive is ‘adapted’ to the environment in which it happens to find itself living.… Taken out of the biological context and applied to cognition, this means that ‘to know’ is not to possess true representations of reality, but rather to possess ways and means of acting and thinking that will allow one to attain the goals one happens to have chosen (2001: 39).
The key to evaluating competing knowledge claims, therefore, is not to seek to compare them to the ‘real world’ that cannot be known, but to assess their cognitive viability or functional fitness.
In observing a film, the viewer abstracts regularities from his or her conscious experience and seeks to fit those regularities into pre-existing information structures. In watching The Scarlet Pimpernel, the viewer identifies regularities in their experience of watching a film and links this semantic information to what he or she already knows about ‘Englishness’ and ‘gentlemanly behaviour.’ A viewer who has no understanding of these concepts will be unable to establish such connections, and as a consequence will interpret the film very differently. Perception is, to a significant extent, dependent upon the viewer’s prior experiences and knowledge, and is comprised of a sense-making activity that involves the building up of conceptual structures by linking new information to old. The result of this process cannot be regarded as being representational as the viewer has no means of accessing the film directly.
In adopting an evolutionary-constructivist approach it is possible to develop a model of information transduction in motion picture perception that is non-representational, ecologically viable, and takes as its starting point the viewer as a biological perceiving system. This approach leads to the conclusion that perception is not direct, and the highly detailed, coherent world that such a viewer experiences is an autopoietic construct of the viewer (Maturana and Varela 1980). The ‘impression of reality’ in the cinema may be accounted for as the viewer’s construction of his or her own experiential reality. The viewer does not pickup information from a film because, to paraphrase Foerster, the film contains no information; the film is as it is.
Ashby, W.R. (1956) An Introduction to Cybernetics. London: Chapman and Hall.
Bach-y-Rita, P. (2002) Sensory substitution and qualia, in A. Noe and E. Thompson (eds.) Vision and Mind: Selected Readings in the Philosophy of Perception. Cambridge, MA: MIT Press: 497-514.
Bach-y-Rita, P., M.E. Tyler, and K.A. Kaczmarek (2003) Seeing with the brain, International Journal of Human-Computer Interaction 15 (2): 285-295.
Bear, M.F., B.W. Connors, and M.A. Paradiso (2007) Neuroscience: Exploring the Brain, third edition. Philadelphia: Lippincott, Williams, and Williams.
Campbell, D.T. (1974) Evolutionary epistemology, in P.A Schilpp (ed.) The Philosophy of Karl R. Popper. LaSalle, IL: Open Court: 412-463.
Crick, F. (1994) The Astonishing Hypothesis. London: Simon and Schuster.
Dyer, R. (1985) Taking popular television seriously, in D. Lusted and P. Drummond (eds.) TV and Schooling. London: BFI: 41-46.
Enticknap, L. (2005) Moving Image Technology: From Zoetrope to Digital. London: Wallflower Press.
Farah, M.J. (2000) The Cognitive Neuroscience of Vision. Malden, MA: Blackwell.
Foerster, H. von ( 1981) Notes on an epistemology of living things, Biological Computer Laboratory Report 9.3, University of Illinois; reprinted in Observing Systems. Seaside, CA: Intersystems Publishing: 258-271.
Foerster, H. von ( 2003) On constructing a reality, in F.E. Preiser (ed.) Environmental Research Design, Volume 2. Stroudsburg, Dowden, Hutchinson & Ross: 35-46; reprinted in Understanding Understanding: Essays on Cybernetics and Cognition, New York, Springer: 211-228.
Glasersfeld, E. von (1995) Radical Constructivism: A Way of Knowing and Learning. London: RoutledgeFarmer.
Glasersfeld, E. von (1999) Piaget’s legacy: cognition as adaptive activity, in A. Riegler, M. Peschl, and A. von Stein (eds.) Understanding Representation in the Cognitive Sciences: Does Representation Need Reality? New York/Dordrecht: Kluwer Academic/Plenum Publishers: 283-287.
Glasersfeld, E. von (2001) The radical constructivist view of science, Foundations of Science 6 (1-3): 31-43.
Gordon, I.E. (1997) Theories of Visual Perception, second edition. Chichester: John Wiley & Sons.
Gulick, R. van ( 1990) Functionalism, information, and content, Nature and System 2: 139-162; reprinted in W.G. Lycan (ed.) Mind and Cognition: A Reader. Oxford: Blackwell, 107-129.
Jackendoff, R. (1987) Consciousness and the Computational Mind. Cambridge, MA: MIT Press.
Lorenz, K. (1941) Kants Lehre von Apriorischen im Lichte gegenwärtiger Biologie. Die angeborene Formen möglicher Erfahrung, Blätter für deutsche Philosophie 15: 94-125.
Maturana, H.R., and F. Varela (1980) Autopoiesis and Cognition: The Realisation of the Living. Dordrecht: D. Reidel Publishing Company.
Müller, J. (1826) Zur vergleichenden Physiologie des Gesichtssinnes des Menschen und der Tiere. Leipzig: C. Knobloch.
Norrsell, U., S. Finger, and C. Lajonchere (1999) Cutaneous sensory spots and the ‘law of specific nerve energies:’ history and development of ideas, Brain Research Bulletin 48 (5): 457-465.
Piaget, J. (1937) La construction du réel chez l’enfant. Neuchatel: Delachaux et Niestle.
Plantinga, C. (1999) The scene of empathy and the human face in film, in C. Platinga and G.M. Smith (eds.) Passionate Views: Film, Cognition, and Emotion. Baltimore and London: The Johns Hopkins University Press: 239-255.
Read, R. (1998) The Essence of Communication Theory. London: Prentice Hall.
Redfern, N. (2004) Communication and meaning in the cinema, Constructivism in the Human Sciences 9 (2): 39-48.
Richards, J. (1997) Films and British National Identity: From Dickens to Dad’s Army. Manchester: Manchester University Press.
Shannon, C.E., and W. Weaver (1949) The Mathematical Theory of Communication. Urbana, Il.: University of Illinois Press.
Sjölander, S. (1999) On the evolution of reality – some biological prerequisites and evolutionary stages, Journal of Theoretical Biology 187 (4): 595-600.
Stonier, T. (1997) Information and Meaning: An Evolutionary Perspective. New York: Springer.
Thayer, L. (1979) Communication: sine qua non of the behavioural sciences, in R.W. Budd and B.D. Ruben (eds.) Interdisciplinary Approaches to Human Communication. Rochelle Park, NJ: Hayden: 7-31.
Wade, N.J., and M. Swanston (1991) Visual Perception: An Introduction. London and New York: Routledge.
Zoglauer, T. (1996) Can information be naturalised?, in K. Kornwachs and K. Jacoby (eds.) Information: New Questions to a Multidisciplinary Concept. Berlin: Akademie-Verlag: 187-207.
This piece was originally presented as a paper at the New Nightmares Conference at Manchester Metropolitan University in April 2008. I am currently working on an expanded version which will look more closely at anxiety in The X-Files, while also examining the role of anxiety in some other Hollywood genres.
The mental flow model of the viewer’s experience of the cinema proposed by Torben Grodal (1997, 1999, 2004) takes film genres to be forms constructed in order to evoke characteristic emotions that are intimately connected to generic themes and narrative structures. In this model, the horror film evokes the emotion ‘fear,’ producing autonomic responses of crying, shivering, and screaming in the viewer. Paranoia is frequently cited as a recurrent generic theme of the horror film (e.g., Orr 2000, Pratt 2001); and Kim Newman identifies a sub-genre of the ‘paranoid horror’ film in which ‘the Establishment is a monolithic, all-encompassing Evil’ (1988: 79). Grodal (1997: 172-173, 250-252) also links horror and paranoia. In this paper I argue that characteristic emotion of paranoia is not fear but anxiety, and films that evoke this emotional state in the viewer demonstrate numerous differences from the horror genre. These differences are evident in the type of emotional responses experienced by the viewer and the level of their intensity, as well as in the generic themes, and narrative structure of the paranoid film. The purpose of this paper is to distinguish paranoid films from the horror genre in Hollywood cinema, and I utilise the mental flow model in drawing this distinction. Consequently, this paper is at once an implementation of this model and a refinement of it.
The mental flow model of the film experience
Cognitive approaches to emotion in the cinema reject romantic and psychoanalytical theories of emotion as irrational negations of reality to assert that cognitions and emotions work together in allowing us to evaluate our environment and as a basis for adaptive behaviours. Emotions are action tendencies that require cognition to recognise the cause of emotions and to evaluate appropriate motor responses. Emotions are structured states that consist of physiological changes, feelings, and thinking; and have a particular object as their focus or target (Plantinga and Smith 1999).
For Grodal, the viewer’s experience of a film must be described as a temporal flow that proceeds from perception to (simulated) motor actions and is mediated by innate emotional functions and cognitive schemata. There are three modes of emotional functions: the telic mode consists of voluntary, goal-directed actions and thoughts; the paratelic mode consists of experiences, actions, and thoughts that take place without a specific goal; and the autonomic mode, which consists of non-voluntary emotional responses and is activated when the subject is unable to exert control over his or her situation. The activation of these modes is related to the forward flow of the narrative so that there is ‘a systemic relation between embodied mental processes and configurations activated in a given type of visual fiction and the emotional “tone” and “modal qualities” of the experienced affects, emotions, and feelings in the viewer’ (Grodal 1997: 3). The forward flow of narrative events in the diegetic world of a film guides the viewer through a sequence of emotional reactions, and is structured by canonical narratives that ‘consist of one or several central [characters], a series of emotion-evoking conditions, and a series of actions to alter conditions and to evoke preferred states’ (Grodal 1999: 137). In this model, the main film genres are based on innate features of mind and body, which presuppose specific mental mechanisms as their necessary (but not sufficient) conditions for their existence. Consequently, genres – as prototypical narratives – are constructed in such a way as to evoke a characteristic emotion (see Table 1).
Table 1 Some genres and their characteristic emotions (Grodal 2004)
An example of the cognitive approach to emotion and genre is the horror film, which may be described as a prototypical narrative in which a character (or characters) is confronted with a hypernatural antagonist producing autonomic responses that are transformed into telic emotional states that are the basis for (simulated) motor actions leading to the destruction of the antagonist. The viewer identifies with a character that is initially marked by an inability to act when confronted with a monstrous threat. This antagonist is hypernatural, deviating from norms of behaviour, the laws of physics, the known facts of history, and is usually supernatural or possesses seemingly superhuman qualities. Faced with such a threat, the character – and by identification, the viewer – experiences a state of paralysis, in which the inability to act triggers responses of shivering, trembling, crying, vaso-motor constrictions, breaking out in goose-pimples, and other autonomic responses (Grodal 1997: 172). This inability to act gives way to a telic mode of experience as the character overcomes the cognitive dissonances created by the antagonist to understand the nature of the threat encountered, devises a pan to eliminate that threat, and successfully executes it. The narrative structure of the horror film guides the viewer through a progression from autonomic to telic modes of experience, from the unwanted emotional state of fear to the preferred emotional state of safety. Dracula (Tod Browning, 1931), for instance, is a prototypical horror narrative, in which the heroes are confronted with an immortal vampire capable of changing his physical form and therefore represents a hypernatural threat, and can only be destroyed once the lore of the vampire has been understood and a plan of action based on symbolic-ritual codes of behaviour is carried out.
Paranoia and anxiety
Anxiety as an emotional state can be distinguished from fear – though it is not always easy to do so (Edelmann 1995). Like fear, anxiety is a state characterised by ‘subjective, consciously perceived feelings of tension and apprehension, and heightened autonomic nervous system activity’ (Spielberger et al. 1970: 3); but where fear is a rational response to a specific and identifiable threat producing intense emergency reactions that recede with the removal of the threat, anxiety is ‘the tense anticipation of a threatening but vague event; a feeling of uneasy suspense … In its purest form anxiety is diffuse, objectless, unpleasant, and persistent. … Anxiety is a state of heightened vigilance rather than an emergency reaction’ (Rachman 1998: 2-3). The differences between fear and anxiety are summarised in Table 2.
Table 2 Characteristics of fear and anxiety (Rachman 1998)
Paranoia is ‘a mental disorder characterised by the presence of persistent non-bizarre delusions of persecutory, grandiose, or other self-referential content’ (Scheuer 2001: 1134). This content typically takes the form of a perceived ‘loss of autonomy, the conviction that someone’s actions are being controlled by someone else, or that one has been “constructed” by powerful, external agents’ (Melley 2000: vii), and reflects an uncertainty about the causes of individual action in the face of a persistent but vague threat to the self. Consequently, paranoia is characterised by an emotional state of anxiety.
The Hollywood paranoid film
The distinction between the emotional states of fear and anxiety makes it possible to nuance the mental flow model, and to make a distinction between the horror genre characterised by fear and the paranoid film characterised by anxiety. The differences between the Hollywood paranoid film and the horror genre are evident in the level of intensity of emotional responses experienced by the viewer, the generic themes of the paranoid film, and the structure of the narrative. In the paranoid film, the unwanted emotional state of anxiety is not dispelled, and, unlike the horror film, autonomic responses do not give way to telic modes of experience. These differences are summarised in Table 3. Examples of these differences can be identified in Enemy of the State (1998) and The X-Files (1998).
Table 3 The horror film and the paranoid film
Enemy of the State
In Enemy of the State, Robert Clayton Dean (played by Will Smith), a Washington, D.C., attorney finds himself in the midst of a frantic search for a recording of the assassination of a U.S. congressman (Jason Robbards) he is unaware he possesses. The film presents a vision of how anyone – accidently and innocently – can be become the victim of a conspiracy, and in doing so presents the viewer with a diegetic world in which seemingly everyday objects become sources of anxiety. Tracking and bugging devices are planted inside shoes, pagers, pens, etc. Telephones are tapped and private conversations recorded without the knowledge of the participants. The film has numerous action sequences, but derives its emotional impact from the suggestion that this could – indeed, very well might – happen to the viewer. The film is thus characterised by a pervasive sense of unease, producing apprehension, tension, and autonomic responses albeit without the emotional outbursts of crying, shivering, etc. The emotional response of the viewer is then similar to that experienced in watching a horror film, but has a lower level of intensity.
This lower level of intensity is, in part, generated by Dean’s lack of awareness of the dangerous circumstances in which he finds himself – repeatedly throughout the film he states that he does not know why he is being pursued. The viewer, however, is aware of the nature of the conspiracy, and tensely anticipates the dangers that will befall Dean. It is this tense anticipation that produces a state of anxiety, and is predicated on the viewer’s identification with Dean as the hero and this excess of narrative knowledge. The viewer is thus placed in a position of hypervigilance. This state of anxiety is not transformed into telic modes of action until the very end of the film, and then only in a limited fashion. With the assistance of an anonymous former conspirator, Brill (Gene Hackman), Dean is able to eliminate the immediate threat posed by his pursuers. However, the central issue of the film is not resolved. The threat of the unregulated and nightmarish applications of surveillance technologies and information gathering systems remains. The narrative thus remains unresolved, and the closing shots of the film echo the opening titles with shots of a satellite orbiting the earth, gathering information, and a montage of surveillance images. The threat posed by the conspirators is only temporary, and is only the visible threat – the unseen surveillance technologies (e.g., satellites, computer tracking systems, directional microphones, etc) remain invisible and remain a threat to society. The need for hypervigilance on the part of the viewer continues, but is rendered problematic by the failure of the film to provide a solution to the regulation of surveillance.
In The X-files, FBI Agents Mulder (David Duchovny) and Scully (Gillian Anderson) investigate the apparent cover-up of the existence of extra-terrestrial life on earth. The film presents an alternative history of the Earth in which aliens arrived on the planet millions of years ago, and which challenges the viewer’s knowledge of the history of civilisation. This questioning of accepted knowledge is a theme that is continued throughout the film, as the viewer is forced to reassess the significance of narrative events according to the interpretations put forward by various characters – the destruction of a federal office building, for example, is either an act of domestic terrorism or the cover up of bodies infected with an alien virus. This frustrates the forward flow of the narrative, blocking the narrative drive of the film. In contrast to Enemy of the State, where anxiety is the product of the viewer’s knowledge exceeding that of Dean, in The X-Files it is the product of cognitive dissonances that cannot be fitted into narrative schemata. As Mulder observes at the end of the film, ‘They’ll never believe you – not unless your story can be easily programmed, categorised, or easily referenced.’
The capacity for voluntary modes of behaviour is again transposed to the field of spectacular action – for example, the escape from the research laboratory or Mulder’s rescue of Scully from the alien ship – and not the resolution of the narrative. As in Enemy of the State, the nature of the conspiracy at the heart of the film remains elusive. It is difficult in this film to determine who or what the antagonist is. The aliens are never clearly presented on the screen, and we learn nothing of their nature or their goals – there is no ‘alien lore’ that can form the basis for the heroes’ destruction of the enemy. The conspirators lack personality, and are identified only by vague descriptions (e.g., the cigarette-smoking man, the well-manicured man, etc.), and the film provides only tantalising glimpses that the conspiracy even exists. The viewer is thus left in a state of confusion with their paranoid suspicions intact, but without the possibility of a resolution. Again, the ending to this film – a hearing to determine the reasons behind the destruction of the office building – mirrors an earlier scene, and explicity rejects the narrative of the Agent’s investigation in the absence of ‘hard evidence.’ The conclusion of the film sees Agents Mulder and Scully reassigned to the X-files in order to investigate inexplicable phenomena, and the reconstitution of the conspiracy at a new location, creating a looped and unending narrative structure that is common to paranoid thrillers (e.g., The Parallax View, 1974).
In the early-1970s, Tony Tanner wrote that ‘the possible nightmare of being controlled by unseen agencies and powers is never far away in contemporary American fiction’ (1971: 15, my emphasis). It is certainly the case that paranoia has been a constant element of American popular culture in the post-war era, but while the paranoid scenarios of Enemy of the State and The X-files might be described as nightmarish they should not be seen as a part of the horror genre. Applying the mental flow model to these films, it is possible to identify a genre of the paranoid film in which the characteristic emotion evoked is anxiety, and which shows marked differences from the horror film.
Enemy of the State (Touchstone Pictures\Jerry Bruckheimer Films\Scott Free, 1998) prod. Jerry Bruckheimer, dir. Tony Scott, wr. David Marconi, ph. Daniel Mindel, ed. Chris Lebenzon, m. Harry Gregson-Williams, Trevor Rabin. Cast: Will Smith (Robert Clayton Dean), Gene Hackman (Brill), John Voight (Reynolds), Lisa Bonet (Rachael Banks), Regina King (Carla Dean), Loren Dean (Hicks), Barry Pepper (Pratt), Ian Hart (Bingham), Stuart Wilson (Congressman Albert).
The X-Files (Twentieth Century-Fox, Ten Thirteen Productions, 1998) prod. Chris Carter, Daniel Sackheim, dir. Rob Bowman, wr. Chris Carter, Frank Spotnitz, ph. Ward Russell, ed. Stephen Mark, m. Mark Snow. Cast: David Duchovny (Special Agent Fox Mulder), Gillian Anderson (Special Agent Dana Scully), John Neville (Well-manicured Man), William B. Davis (Cigarette-smoking Man), Martin Landau (Alvin Kurtzweil, MD), Mitch Pileggi (Assistant Director Walter Skinner).
Edelmann, R.J. (1995) Anxiety: Theory, Research, and Intervention in Clinical and Health Psychology. Chichester: John Wiley and Sons.
Grodal, T. (1997) Moving Pictures: A New Theory of Film Genres, Feeling, and Cognition. Oxford: Clarendon Press.
Grodal, T. (1999) Emotions, cognition, and narrative patterns in film, in C. Plantinga and G.M. Smith (eds.) Passionate Views: Film, Cognition, and Emotion. Baltimore and London: Johns Hopkins University Press: 127-145.
Grodal, T. (2004) Frozen flows in von Trier’s oeuvre, in T. Grodal, B. Larsen, and I.T. Laursen (eds.) Visual Authorship: Creativity and Intentionality in Media. Copenhagen: Museum Tusculanum Press\University of Copenhagen: 129-167.
Melley, T. (2000) Empire of Conspiracy: The Culture of Post-war paranoia in Post-war America. Ithaca: Cornell University Press.
Newman, K. (1988) Nightmare Movies: A Critical History of the Horror Movies. New York: Harmony Books.
Orr, J. (2000) The Art and Politics of Film. Edinburgh: Edinburgh University Press.
Plantinga, C. and Smith, G.M. (1999) Introduction, in C. Plantinga and G.M. Smith (eds.) Passionate Views: Film, Cognition, and Emotion. Baltimore and London: The Johns Hopkins University Press: 1-17.
Pratt, R. (2001) Projecting Paranoia: Conspiratorial Visions in American Film. Lawrence, Kan.: University Press of Kansas.
Rachman, S. (1998) Anxiety. Hove: Psychology Press.
Scheuer, A.D. (2001) Paranoia, in The Corsini Encyclopedia of Psychology and Behavioural Science – Volume Three, third edition, edited by W.E. Craighead and C.B. Nemeroff. New York: John Wiley & Sons: 1133-1135.
Spielberger, C.D., Gorush, R.L., and Lushene, R.E. (1970) Manual for State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychology Press.
Tanner, T. (1971) City of Words. London: Jonathan Cape.