Information transduction in the cinema
A version of this post was presented to the CCM Research group at the University of Central Lancashire on 21 February 2007. It is presented here with the first part – a (not entirely satisfactory) discussion of ecological approaches to film theory – missing, and a new introduction. The model in figure 1 now seems incomplete and needs further development, but I think it still has some uses as a basic description of information and perception in the cinema.
Wade and Swanston point out that in order to come to a full understanding of vision it is necessary to ‘include an appreciation of the neurophysical processes that are initiated by the activity of light on the receptors of the eye. These involve the modification of light energy into nerve impulses and their transmission to areas at the back of the brain where they are analysed’ (1991: 59). In my opinion, it is precisely this ‘appreciation of neurophysical processes’ that should form the basis of film theory – not least because we need to be able to account for own experiences of the cinema in formulating hypotheses about it. A cognitive approach to film theory allows theorists to build self-reflexivity into their research. The Society for Cognitive Studies of the Moving Image promotes research in this area, with a particular emphasis on viewer’s emotional experiences. David Bordwell has recently written on the latest research to emerge here and here.
However, I have been repeatedly frustrated by the lack of a model of communication in the cinema that can account for the viewer’s perception of a film in terms of neurophysical processes. For example, Carl Plantinga (1999) has noted that ‘[o]ne of the least explored aspects of film and television is their sensory means of communication’ (239) – but then on the next page he asserts that ‘[c]learly, film directors use the human face to communicate information about the emotions of characters’ (240, original emphasis). These are the only two occasions Plantinga mentions communication – but he goes from a topic that is one of the ‘least explored’ to the self-evident clarity of the assertion that films communicate emotions to the viewer. Plantinga never defines what he means by communication or information, though we may infer that he means the transmission from screen to spectator of meaningful content about the emotional states of a particular character. However, as Thayer has pointed out, such communication is impossible:
The ways in which we traditionally conceive of communication – those being inadequate and untenable – stand as obstacles to more adequate and more potent ways of conceiving of communication … Those preconceptions, our traditional concepts of communication, are often insidious. ‘Communication is the “transfer of meaning”’ has an appealing ring to it. But since none of our receptors is capable of receiving ‘meaning,’ the notion of transfer is a flagrantly untenable one (Thayer 1979: 10).
The work of Paul Bach-y-Rita (2002, 2003) at the University of Wisconsin deserves special mention here. Bach-y-Rita and his fellow researchers have used televisual sensory substitution systems to restore the modality of sight to vision impaired individuals, and he has addressed the impact of restored sight and emotion content:
we found that while experienced blind TVSS subjects could perceive faces and printed images, they were very disappointed when perception was not accompanied by qualia: A Playboy centerfold carried no emotional message, and the face of a girl-friend or a wife created an unpleasant response since it did not convey an affective message. We consider this to be comparable to the lack of emotional contact of curse-words in a language that has been learned as an adult. It is possible that the emotional content could be developed over a long period of usage. On the other hand, a blind infant using a vision substitution system smiles when he recognizes a toy and reaches for it, and a blind 10-year-old child perceiving a flickering candle flame by means of a TVSS is enchanted (Bach-y-Rita et al. 2003: 293).
This quote is very suggestive for cognitive film theorists working in the area of emotion. It raises a fundamental question: what is the nature of communication in the cinema? This paper explores this question through looking at information and the different forms it takes in the viewer’s experience of a motion picture.
Information in the cinema
Francis Crick points out that there ‘is one fact about the brain that is so obvious it is seldom mentioned: it is attached to the rest of the body and communicates with it. The nervous system receives information only from the various transducers in the body’ (1994: 81). This principle has long history and may be traced back to Johannes Müller’s law of specific nerve energies, which states that it no matter how a sensory system is stimulated, the resulting sensation will always be of the type appropriate to that system (Müller 1826). For example, the stimulation of the optic nerve will result in visual sensation regardless of whether that stimulation is by flashing light, by electric shock, or by pressure on the eye (Norrsell et al. 1999); and this sensation is dependent upon the part of the brain in which the sensory pathways terminate and not the stimulus. Thus we ‘see with the brain, not the eyes’ (Bach-y-Rita et al. 2003: 285) as the images that pass through the pupil and are focussed on the retina go no further: ‘The sole source of output from the retina to the rest of the brain is the action potentials arising from the million or so ganglion cells’ (Bear et al. 2007: 300). The brain has no independent reference as to the cause of electro-chemical signals that are transmitted along the optic nerve because the ‘response of a nerve cell does not encode the physical nature of the agents that caused its response. Encoded is only “how much” at this point on my body, but not “what”’ (Foerster  2003: 214).
Images in the cinema are comprised of variable physical properties in the pattern of silver salts of the film’s emulsion (Enticknap 2005: 203), so that light projected through a film and reflected by a screen is energy in the form of electromagnetic radiation and is experienced by the viewer as changes in the intensity of light and colour, and its position in the frame (Read 1998: 1). This light carries no qualitative information about the environment – there is only quantitative data about the energetic properties of the light. As Foerster points out ( 1981: 263): ‘the environment contains no information, the environment is as it is.’ The viewer, as a perceiving system, is capable of receiving light as energy as evolution has led to the development of a visual system that responds to changes in the quantitative properties of a stream of photons (e.g. hue, luminosity) without knowledge of the cause of such properties. The viewer, then, is open to energy (‘how much’) but is closed to information (‘what’) (Ashby 1956).
As sensory systems function by ‘transducing some type of environmental energy into a form that can be analysed by the cells in the central nervous system’ (Wade and Swanston 1991: 59), perception cannot be considered direct – it is mediated by the sensory and neurophysiological processes of the perceiver. Those processes begin with light being focussed onto the retina:
Light emitted by or reflected off objects in space can be imaged by the eye onto the retina. Light energy is first converted into membrane potential charges in the mosaic of photoreceptors. … photoreceptor membrane potential is converted into a chemical signal (the neurotransmitter glutamate), which is again converted into a membrane potential changes in the post-synaptic bi-polar horizontal cells. This process of electrical-chemical-electrical signalling repeats again and again, until the presence of light or dark or colour is finally converted to a charge in the action potential firing frequency of the ganglion cells (Bear et al. 2007: 306).
The light energy reflected by a cinema screen enters the viewer’s eye where it is converted into a pattern of stimulation, and is projected to the magnocellular and parvocellular layers of the lateral geniculate nucleus before being relayed to the visual cortex (Farah 2000). As quantitative information, the pattern of stimulation at the retina is syntactic (Shannon and Weaver 1949), and is transduced into the functional information by the firing of neurons in the visual cortex. There is a non-random correlation between these two types of information (Gulick  1990).
A perceiving system is not aware of this complex process of information transduction, only of the results of this process (Jackendoff 1987). Somehow – and this remains a mystery to philosophers, psychologists, and neuroscientists – a perceiver is conscious of these results and they are meaningful. The semantic information that is the content of consciousness is an emergent product of the mind, and cannot be reduced to syntactic relations: information only becomes meaningful when a perceiver is able to link it to information structures he or she already possesses (Stonier 1997). Such information, then, is highly context-dependent, and from this it must be concluded that the environment does not contain sufficient information to guide a perceiver’s behaviour. The question of meaning comes down to how a perceiving system generates units of experience and relates them to conceptual structures that from the basis for subsequent modes of acting and thinking (Redfern 2004). As semantic information cannot be distinguished from those modes it becomes pragmatic information when it is embedded in a social practice (Zoglauer 1996).
The transduction of light energy to syntactic to functional to semantic/pragmatic information creates a layered hierarchy of information levels in which higher information concepts depend on lower level information concepts but cannot be reduced to them (Zoglauer 1996). This relationship is represented in Figure 1. Though we are conscious only of information at the semantic/pragmatic level of this hierarchy it is essential that we include the lower information levels and do not marginalise the physical inputs to the viewer as a perceiving system or ignore the viewer’s neurophysiological processes of information transduction.
Figure 1 Information transduction in motion picture perception
Representation in the cinema
The concept of representation is central to both cognitive psychology and to the study of all media forms as well as more specifically the cinema. Representation is a term used in a wide variety of senses and can refer to any symbolic description of the world. Thus, from a psychological perspective, representation refers to mental images that correspond to objects that lie beyond a perceiver’s sensory systems; while in film studies, representation is used in reference to the reflection or distortion of the ‘real’ (Dyer 1985). Both these uses of representation are manifest in the application of cognitive models to the cinema: in watching, say, The Scarlet Pimpernel (Harold Young, 1934) the viewer has a mental image of the depiction of the Pimpernel as an English gentleman adventurer (Richards 1997) (see Figure 2).
If the above description of information transduction in motion picture perception is accepted then the concept of representation is of no use in accounting for the viewer’s experience of a film. As the viewer is, in informational terms, organisationally closed his or her conscious experience cannot be said to correspond to anything that exists independently of the viewer. As an organisationally closed system that interacts necessarily with its own states, the viewer has no external point of reference by which to judge the correspondence of mental images to the world: the viewer has no means of establishing a correspondence between his or her perception of The Scarlet Pimpernel and the film itself. If it is accepted that the viewer is organisationally closed then the sensory structure, patterns, or images he or she experiences are the viewer’s own construction, and the notion that they represent an aspect of the world has no empirical foundation. Glasersfeld (1995, 1999) argues that in the place of representation, we should use the term presentation, as this is closer to Kant’s deployment of vorstellung in ‘The Conflict of the Faculties,’ to refer to concepts that are generated by a perceiver: ‘the mind can only create only presentations of its own objects and not of the real things, that is, through these presentations and concepts, things cannot possibly be known as they might be in themselves’ (quoted in Glasersfeld 1995: 39-40). The qualitative aspects of the viewer’s experience are solely determined by the viewer, and his or her experience of The Scarlet Pimpernel is a mental presentation of the viewer that emerges as a result of a complex process of information transduction.
Figure 2 Leslie Howard as The Scarlett Pimpernel (1934)
The concept of constructions that do not correspond to an external reality does not imply epistemological solipsism. Lorenz (1941) argued that evolution has provided us with a perceptual system that allows us to operate in the absence of information about the ‘real world.’ This principle has been developed by evolutionary epistemologists (Campbell 1974) and radical constructivists (Glasersfeld 1995), who argue that constructions are adaptations that provide us with viable ways of thinking and acting in an environment (Sjölander 1999). This principle of adaptation is derived from the work of Jean Piaget (1937), who approached the construction of knowledge as a biologist. For Piaget, adaptation involves two complimentary and simultaneous processes: a cognising organism primarily seeks to organise experience in terms of the psychological structures (schemes) it already possesses, i.e., it seeks to assimilate experience; if the result of this process creates a perturbation the organism attempts to accommodate the error either by modifying an existing scheme or creating a new one. It is this balance between assimilation and accommodation that Piaget describes as adaptation. Knowledge is actively constructed, and is adapted to fit the environmental constraints that act on an organism in order to avoid internal contradictions and achieve equilibrium. Glasersfeld describes the principle of adaptation in radical constructivist thought:
[A]daptation is not an activity but the result of the elimination of all that is not adapted. Consequently, on the biological level, anything that manages to survive is ‘adapted’ to the environment in which it happens to find itself living.… Taken out of the biological context and applied to cognition, this means that ‘to know’ is not to possess true representations of reality, but rather to possess ways and means of acting and thinking that will allow one to attain the goals one happens to have chosen (2001: 39).
The key to evaluating competing knowledge claims, therefore, is not to seek to compare them to the ‘real world’ that cannot be known, but to assess their cognitive viability or functional fitness.
In observing a film, the viewer abstracts regularities from his or her conscious experience and seeks to fit those regularities into pre-existing information structures. In watching The Scarlet Pimpernel, the viewer identifies regularities in their experience of watching a film and links this semantic information to what he or she already knows about ‘Englishness’ and ‘gentlemanly behaviour.’ A viewer who has no understanding of these concepts will be unable to establish such connections, and as a consequence will interpret the film very differently. Perception is, to a significant extent, dependent upon the viewer’s prior experiences and knowledge, and is comprised of a sense-making activity that involves the building up of conceptual structures by linking new information to old. The result of this process cannot be regarded as being representational as the viewer has no means of accessing the film directly.
In adopting an evolutionary-constructivist approach it is possible to develop a model of information transduction in motion picture perception that is non-representational, ecologically viable, and takes as its starting point the viewer as a biological perceiving system. This approach leads to the conclusion that perception is not direct, and the highly detailed, coherent world that such a viewer experiences is an autopoietic construct of the viewer (Maturana and Varela 1980). The ‘impression of reality’ in the cinema may be accounted for as the viewer’s construction of his or her own experiential reality. The viewer does not pickup information from a film because, to paraphrase Foerster, the film contains no information; the film is as it is.
Ashby, W.R. (1956) An Introduction to Cybernetics. London: Chapman and Hall.
Bach-y-Rita, P. (2002) Sensory substitution and qualia, in A. Noe and E. Thompson (eds.) Vision and Mind: Selected Readings in the Philosophy of Perception. Cambridge, MA: MIT Press: 497-514.
Bach-y-Rita, P., M.E. Tyler, and K.A. Kaczmarek (2003) Seeing with the brain, International Journal of Human-Computer Interaction 15 (2): 285-295.
Bear, M.F., B.W. Connors, and M.A. Paradiso (2007) Neuroscience: Exploring the Brain, third edition. Philadelphia: Lippincott, Williams, and Williams.
Campbell, D.T. (1974) Evolutionary epistemology, in P.A Schilpp (ed.) The Philosophy of Karl R. Popper. LaSalle, IL: Open Court: 412-463.
Crick, F. (1994) The Astonishing Hypothesis. London: Simon and Schuster.
Dyer, R. (1985) Taking popular television seriously, in D. Lusted and P. Drummond (eds.) TV and Schooling. London: BFI: 41-46.
Enticknap, L. (2005) Moving Image Technology: From Zoetrope to Digital. London: Wallflower Press.
Farah, M.J. (2000) The Cognitive Neuroscience of Vision. Malden, MA: Blackwell.
Foerster, H. von ( 1981) Notes on an epistemology of living things, Biological Computer Laboratory Report 9.3, University of Illinois; reprinted in Observing Systems. Seaside, CA: Intersystems Publishing: 258-271.
Foerster, H. von ( 2003) On constructing a reality, in F.E. Preiser (ed.) Environmental Research Design, Volume 2. Stroudsburg, Dowden, Hutchinson & Ross: 35-46; reprinted in Understanding Understanding: Essays on Cybernetics and Cognition, New York, Springer: 211-228.
Glasersfeld, E. von (1995) Radical Constructivism: A Way of Knowing and Learning. London: RoutledgeFarmer.
Glasersfeld, E. von (1999) Piaget’s legacy: cognition as adaptive activity, in A. Riegler, M. Peschl, and A. von Stein (eds.) Understanding Representation in the Cognitive Sciences: Does Representation Need Reality? New York/Dordrecht: Kluwer Academic/Plenum Publishers: 283-287.
Glasersfeld, E. von (2001) The radical constructivist view of science, Foundations of Science 6 (1-3): 31-43.
Gordon, I.E. (1997) Theories of Visual Perception, second edition. Chichester: John Wiley & Sons.
Gulick, R. van ( 1990) Functionalism, information, and content, Nature and System 2: 139-162; reprinted in W.G. Lycan (ed.) Mind and Cognition: A Reader. Oxford: Blackwell, 107-129.
Jackendoff, R. (1987) Consciousness and the Computational Mind. Cambridge, MA: MIT Press.
Lorenz, K. (1941) Kants Lehre von Apriorischen im Lichte gegenwärtiger Biologie. Die angeborene Formen möglicher Erfahrung, Blätter für deutsche Philosophie 15: 94-125.
Maturana, H.R., and F. Varela (1980) Autopoiesis and Cognition: The Realisation of the Living. Dordrecht: D. Reidel Publishing Company.
Müller, J. (1826) Zur vergleichenden Physiologie des Gesichtssinnes des Menschen und der Tiere. Leipzig: C. Knobloch.
Norrsell, U., S. Finger, and C. Lajonchere (1999) Cutaneous sensory spots and the ‘law of specific nerve energies:’ history and development of ideas, Brain Research Bulletin 48 (5): 457-465.
Piaget, J. (1937) La construction du réel chez l’enfant. Neuchatel: Delachaux et Niestle.
Plantinga, C. (1999) The scene of empathy and the human face in film, in C. Platinga and G.M. Smith (eds.) Passionate Views: Film, Cognition, and Emotion. Baltimore and London: The Johns Hopkins University Press: 239-255.
Read, R. (1998) The Essence of Communication Theory. London: Prentice Hall.
Redfern, N. (2004) Communication and meaning in the cinema, Constructivism in the Human Sciences 9 (2): 39-48.
Richards, J. (1997) Films and British National Identity: From Dickens to Dad’s Army. Manchester: Manchester University Press.
Shannon, C.E., and W. Weaver (1949) The Mathematical Theory of Communication. Urbana, Il.: University of Illinois Press.
Sjölander, S. (1999) On the evolution of reality – some biological prerequisites and evolutionary stages, Journal of Theoretical Biology 187 (4): 595-600.
Stonier, T. (1997) Information and Meaning: An Evolutionary Perspective. New York: Springer.
Thayer, L. (1979) Communication: sine qua non of the behavioural sciences, in R.W. Budd and B.D. Ruben (eds.) Interdisciplinary Approaches to Human Communication. Rochelle Park, NJ: Hayden: 7-31.
Wade, N.J., and M. Swanston (1991) Visual Perception: An Introduction. London and New York: Routledge.
Zoglauer, T. (1996) Can information be naturalised?, in K. Kornwachs and K. Jacoby (eds.) Information: New Questions to a Multidisciplinary Concept. Berlin: Akademie-Verlag: 187-207.
Posted on June 18, 2009, in Cognitive Film Theory, Emotion, Film Studies and tagged Carl Plantinga, Cognitive Film Theory, David Bordwell, Emotion, Film Studies, Film Theory, Perception. Bookmark the permalink. Leave a comment.