Hitchcock and shot scales

This week a look at shot scales and shot types in the films of Alfred Hitchcock. The pdf can be accessed here: Nick Redfern – Statistical analysis of shot types in the films of Alfred Hitchcock

Statistical analysis of shot types in the films of Alfred Hitchcock

Abstract

This paper analyses the changing use of shot scales and shot types in the films of Alfred Hitchcock from The Pleasure Garden (1925) to The Birds (1963) in the context of the introduction of sound technology to British cinema in 1929 and the director’s move from Britain to Hollywood in 1939. A sample of 42 films was divided into 3 subgroups (British silent films [𝑛 = 9]; British sound films [𝑛 = 14]; and Hollywood films [𝑛 = 19]); and was analysed using linear regression of rank-frequency plots and nonparametric analysis of variance. The results show that all three groups of films are well-fitted by a linear regression model, with no one shot scale dominating these films. Analysis of the different shot scales revealed that there are no significant differences in the use of shot scales between the two groups of British films, but that significant differences did occur between the British and American films for close-ups and medium close-ups, which increase in frequency, and medium long shots and long shots, which became less frequent. The proportion of reverse-angle cuts in the Hollywood films is much greater than in the British films, and this may be due to the use of shot-reverse shot editing patterns in Hollywood cinema. There is no evidence that the number of point-of-view shots or inserts changed, and this may be attributed to the fact that these types of shots are used in specific circumstances as required by the demands of narrative. Overall the results indicate that the introduction of sound technology did not have an impact on Hitchcock’s film style, but that the move to Hollywood did result in specific changes in the style of Hitchcock’s films.

This paper expands and imporoves on the methodology of using rank-frequency plots and ranks to analyse shot scales that I’ve used elsewhere. It also clarifies and updates and earlier discussion of shot scales in Hitchcock’s films, as well as tentatively exploring the relationship between reverse-angle cuts and POV shots. There is, however, much work to be done in this area – especially on Hitchcock’s use of shot-reverse shot editing.

Robust measures of scale for shot length distributions

This week I have written a short paper on robust measures of scale for shot length distributions. The statistical analysis of film style has typcially focussed on questions of location rather than the dispersion of shot lengths in a motion – understanding how the variation in shot lengths has changed is as important as understanding how editing has speeded up or slowed down over time.  Just as we need robust measures of location (e.g. the median shot length) we also need robust measures of dispersion, and in this paper I look at six possible statistics that could be used. The paper can be downloaded here as a pdf file:

Nick Redfern – Robust measures of scale for shot length distributions

The shot length data for the three Laurel and Hardy films that I refer was collected by me as part of a larger study, and when I finally finish it off I will post the draft of my Laurel and Hardy essy along with the complete shot length data for all the films I have looked.

Many of the papers on statistical methodology that I cite can be accessed for free over the internet, and if anyone is interested in the statistical analysis of film style then I recommend reading the papers on robust statistics before proceeding as this will save you a lot of trouble in the long run. The references, with links to online versions of the papers are:

Croux C and Rousseeuw PJ 1992 Time-efficient algorithms for two highly robust estimators of scale, Computational Statistics 1: 411-428.

Daszykowski M, Kaczmarek K, Vander Heyden Y, and Walczak B 2007 Robust statistics in data analysis – a review: basic concepts, Chemometrics and Intelligent Laboratory Systems 85: 203-219.

Gorard S 2004 Revisiting a 90-year-old debate: the advantages of the mean deviation, British Educational Research Association Annual Conference, University of Manchester, 16-18 September 2004: http://www.leeds.ac.uk/educol/documents/00003759.htm, accessed 15 July 2010.

Rousseeuw PJ 1991 Tutorial to robust statistics, Journal of Chemometrics 5: 1-20.

Rousseeuw PJ and Croux C 1993 Alternatives to median absolute deviation, Journal of the American Statistical Association 88: 1273–1283.

Shot length distributions in the films of Alfred Hitchcock, 1927 to 1931

In an earlier post I mentioned I was looking at the distribution of shot lengths in the two versions of Blackmail (1929), along with several of Alfred Hitchcock’s late-silent and early sound films, and today’s post is the first draft of that paper. The abstract is below, and the pdf can be download here: Nick Redfern – Shot length distributions in the films of Alfred Hitchcock, 1927 to 1931.

Shot length distributions in the films of Alfred Hitchcock, 1927 to 1931

The shot length distributions of the silent and sound versions of Blackmail (Alfred Hitchcock, 1929) are analysed in to determine if the introduction of synchronous sound technologies led to a difference in film style. The two versions of this film are then analysed along with four of Hitchcock’s silent films prior to the filming of Blackmail and four of his early sound films. The results show that there is no significant difference in the shot length distributions of the two versions of Blackmail at any point (Kolmogorov-Smirnov: D = 0.07, p = 0.19). Putting Blackmail into the context of Hitchcock’s late silent and early sound films from 1927 to 1931, there is no significant difference in the median shot lengths (Mann Whitney: U = 4.5, p = 0.12), but there is a significant difference in the interquartile ranges (Mann Whitney: U = 0.0, p = 0.01). Closer inspection of this change in the dispersion of shot lengths reveals that there is no significant difference in the lower quartiles of these films (Mann Whitney: U = 10.0, p = 0.66) but there is a significant change in the upper quartiles (Mann Whitney: U = 0.0, p = 0.01). The impact of sound on Hitchcock’s style can, therefore, be seen in the increase in the dispersion of shots above the median rather than in the distribution overall. Comparing the interquartile ranges for the ten films it is possible to sort them into three groups: the late sound films, the two versions of Blackmail, and the early sound films. The results show that there is no difference in the distribution of shot lengths between the two versions of Blackmail due to the unique circumstances of production and that as silent-sound hybrids they represent a transitional phase between Hitchcock’s ‘pure’ silent pictures and his ‘pure’ sound films.

Some notes on Cinemetrics III

This post was originally intended to be part of a thread on the discussion board of the Cinemetrics website, but for some reason it did not upload properly. This post presents my piece in full, but readers should refer to the original thread to get the preceding parts of the discussion.

The difference between the median/mean ratio for two film or two groups of films (e.g. silent films and sound films) can be explained by the presence of outliers in the data and the influence they have on the mean shot length. This can be demonstrated by looking at the shot length distributions for the two versions of Blackmail, The Lights of New York, and Scarlett Empress. (The data for these films can be found in the Cinemetrics database).

Imagine you have two data sets that are identical except for a single value. For example,

A:  1, 2, 3, 4, 5, 6, 7, 8, 9, 10

B:  1, 2, 3, 4, 5, 6, 7, 8, 9, 20

For data set A, the median is 5.5, the mean is 5.5, and so the median/mean ratio is 5.5/5.5 = 1.0. For data set B, the median is 5.5, the mean is 6.5, and the median/mean ratio is 5.5/6.5 = 0.85. The changes in the mean and the median/mean ratio are due to the influence of a single outlying data point, and do not reflect the fact that the two data sets are otherwise identical.

This is precisely what we see when we look at the two versions of Blackmail. In the table below, we have the mean shot length, the median shot length, and the ratio of the median to the mean.

	Blackmail (silent)	Blackmail (sound)
Median shot length (s)	5.6	5.1
Mean shot length (s)	8.1	10.4
Median/mean	0.69	0.49

Looking at the mean, we might think that the impact of sound technology was to lead to a change in style, with an increase in shot lengths (the difference in the means is 2.3 seconds). However, we know that shot length distributions are positively skewed with outlying data points, and that the mean is, therefore, problematic. The difference in the medians is small (only 0.5 seconds), indicating that no such change occurred. This conclusion is supported by a medians test, which shows no significant difference: p = 0.135. A more complete picture may be obtained by looking at the five number summary for each film.

	Blackmail (silent)	Blackmail (sound)
Minimum shot length (s)	0.1	0.1
Lower quartile (s)	2.9	2.5
Median shot length (s)	5.6	5.1
Upper quartile (s)	10.1	11.5
Maximum shot length (s)	104.3	144.6

Looking at this data, we would conclude that the difference between the styles of these two films occurs above the upper quartile – the difference is in the length of the outlying data points away from the mass of the data. The lower quartiles in the silent and sound versions are similar – each film has approximately the same proportion of shots less than or equal to 2.9s (25% and 29%, respectively). This is also the case for the medians: half of the shots in the silent version are less than or equal to 5.6 seconds, while half the shots in the sound version are less than or equal to 5.1 seconds. The difference between the upper quartiles is greater (1.4 seconds) but is still less than the difference between the means – where 75% of the shots in the sound version are less than or equal to 11.5 seconds, this proportion in the silent version is 79% for the same value. In fact by looking at the empirical cumulative distribution functions for both versions of Blackmail (see Figure 1) it is clear that they have almost identical distributions; and a 2-sample Kolmogorov-Smirnov test shows that there is no statistically significant difference for any shot length in the two versions of this film (D = 0.0666, p = 0.1881). (Note that the distribution functions in the graph below are empirical – i.e. they are the actual probability distributions of the shot length data from the Cinemetrics database, and they are not theoretical distributions).

Figure 1 The empirical cumulative distribution functions for the two versions of Blackmail (1929)

The only explanation for the difference in the means, and for the difference in the median/mean ratios, is the influence of the outliers on the mean. Using the mean – or any statistic based on the mean – will lead to incorrect conclusions. Using the mean, we might conclude that the shot lengths in the two versions of Blackmail show a statistically significant increase with the use of sound technology – but this would be wrong. As in the example data sets above, the difference we find in the median/mean ratios reflects the influence of these outlying data points, and does not accurately reflect the distribution of shots in the two versions in the two versions of Blackmail. When we use measures of dispersion that are robust against outliers we do not see the large difference in the dispersion of shot lengths we would expect with the median/mean ratio. The median absolute deviation for the silent version is 3.3 seconds and for the sound version is 3.1 seconds; while the interquartile ranges are 7.2 and 9.0 seconds, respectively.

Let us now look at the shape factors of the two versions of Blackmail. The standard deviation of the logarithms of the shot lengths for the silent version is 0.91; and the equivalent value for the sound version is 1.05. This suggests that the two distributions have different shapes – the sound version being more widely dispersed than the silent version. However, from the five number summary, the empirical cumulative distribution functions, and the robust measures of dispersion we can see that this is not the case. Therefore, we have two versions of the same film with shot length distributions that show no statistically significant difference, but with a large difference in the median/mean ratio and a corresponding difference in the lognormal shape factors. These differences are due to the influence of outlying data points, and do not accurately reflect the nature of the relationship between these two distributions.

A second example can be used to illustrate what happens when we have two films with the same mean but different median shot lengths. The mean shot length, the median shot length, and the ratio of the median to the mean for Lights of New York and Scarlett Empress are presented in the table below.

	Lights of New York	Scarlett Empress
Median shot length (s)	5.1	6.5
Mean shot length (s)	9.9	9.9
Median/mean	0.52	0.66

Looking at the mean shot lengths, we can see that they are identical and we might conclude that these films are cut equally quickly; but looking at the medians we can immediately see that there is a difference of 1.5 seconds, which alerts us to the possibility that Lights of New York is cut quicker than Scarlett Empress. A medians test tells us that there is, in fact, statistically significant difference in the medians of these two films: p = 0.0007.

Now, the median/mean ratio is a crude measure of the dispersion of skewed data set, and the smaller the value of this ratio the more dispersed the data (i.e. the greater the distance between the median and the mean). For symmetrical distributions the median and the mean are equal and the ratio is one; but as shot length distributions are positively skewed the mean will always be greater than the median and the ratio will always, therefore, be less than one. (The ratio is typically given in economic text books as the mean divided by the median [it is used as a measure of income inequality], but this is just the reciprocal of the median/mean ratio). Clearly, the use of the mean as a measure of central tendency will lead us to an incorrect conclusion about the difference in style of Lights of New York and Scarlett Empress; but does it fare any better as an indicator of which film has shot lengths that are more dispersed?

According to the table above, Lights of New York has a smaller median/mean ratio (0.52) and so we would expect the shot lengths for this film to be more dispersed than those of Scarlett Empress (0.66). The standard deviation for Lights of New York is 14.5 seconds, and for Scarlett Empress it is 9.6 seconds – again indicating that the former is more dispersed than the latter. (The lognormal shape factors for these two films are 0.93 and 0.88, respectively). However, when we look at the median absolute deviation and the interquartile ranges we get a different picture.  For both statistics, it is evident that Scarlett Empress is, in fact, more dispersed.

	Lights of New York	Scarlett Empress
Median absolute deviation (s)	2.6	3.5
Interquartile range (s)	7.2	9.3

This can be easily seen when looking at the box plots of these two films (Figure 2). In the box plots note that the interquartile range (the box) for Lights of New York is narrow than that for Scarlett Empress; and that the distance between the minimum shot length (the end of the error bar to the left) and upper inner fence (the error to the right of the box at Q3+(IQR*1.5)) is less for the former (0.9 – 20.9 seconds) than it is for the latter (0.3 – 26.9 seconds). Anything above the greater of these values (i.e. 20.9s and 26.9s is classed as an outlier, while ‘very extreme’ values are defined as Q3+(IQR*3).

Figure 2 Box plots for shot lengths in Lights of New York (1928) and Scarlett Empress (1934)

It is evident, therefore, that (1) shot lengths in Scarlett Empress are more dispersed than those of Lights of New York; and (2) the reason the mean shot length, the median/mean ratio, and the standard deviation give misleading results is because of the influence of the outlying data points in Lights of New York (which account for only 9.76% of this film’s data).

Is the median/mean ratio still useful for estimating the median for these four films? For the silent version of Blackmail the estimated median based on the shape factor given above is 5.3 seconds; and for the sound version is 5.9 seconds. Thus, the first estimate is good with an error of 0.3 seconds (or 4.7%); while the second estimate is less good and is out by 0.8 seconds (16.1%). For Lights of New York, the estimate of the median is 6.4 seconds – an error of 1.3 seconds or 25.1%! The estimate for Scarlett Empress is much better at 6.7 seconds, and is out by only 0.2 seconds (3.7%). We can see, therefore, that estimating the median as the mean/(exp(0.5*(σ^2))) may produce very good estimates but may also produce very bad ones.

The mean is not a robust statistic, and is vulnerable to two factors: the presence of outliers in a data set and the asymmetry of a data set. Unfortunately, these are precisely the characteristics of the distribution of shot lengths in a motion picture. Any value calculated using the mean (e.g. the standard deviation, the median/mean ratio) will not accurately reflect the style of a film due to the impact of outlying data points on the mean. Use of the mean will, therefore, leads us to make a range of incorrect conclusions.

• In the case of the example of Blackmail, we would have incorrectly concluded that there is a difference between the shot lengths of the two versions of this film, when in fact there is no such difference.
• In the example of Lights of New York and Scarlett Empress, we would have incorrectly concluded that there is not a difference between the shot lengths of these two films, when in fact there is such a difference.
• In the example of Lights of New York and Scarlett Empress, we would have incorrectly concluded that the shot lengths of the former are more widely dispersed than in the latter, when in fact the opposite is true.
• Using the mean may produce wither very good estimates of the median or it may produce very bad estimates of the median. Simply relying on this method to lead us to reliable conclusions will not work: if we used the estimate of the median for the sound version of Blackmail in a study we would be basing our analysis on a fundamental error.

The mean shot length is not a reliable statistic of film style. The median/mean ratio suffers from precisely the same problem that has always existed with the mean. It is just a different way of presenting it.

(I’m currently looking at the impact of sound on Hitchcock’s style and Blackmail in more detail, and I’ll put up a post on this subject at a later date. I began working on this piece a couple of months ago and the data I have been using (and the data referred to above) was submitted in 2006 by Isobel Walker. Charles O’Brien has recently submitted new data for the sound version of Blackmail, but I have not looked at this in detail yet).

Power functions and the mean relative frequency of shot scales in motion pictures

UPDATE (26 October 2009): This post has been getting a lot of traffic recently, and I think it is important to point the reader in the direction of a follow up post (here) where I point out that while the use of rank-frequency plots is useful for analysing film style (see here, for example) the power laws approach is not. This is not to say that films or groups of films will not be described by a non-linear power model, but other non-linear models (logarithmic, exponential) are also evident and there does not seem to be any general rule for which model can be applied to specific types of films (e.g. genres, eras, etc.). In general, a power laws approach to the distribution of shot scales is not going to get you anywhere – certainly not to the extent that I suggest in this post. I’ll leave this post here because there is still some useful information, and it’s also nice to see how wrong you can be.

Power functions describe a wide range of social phenomena, from the distributions of city size to the popularity of websites, the citations of academic papers, and the frequency of words in the corpus of a language (Schroeder 1991: 33-38, 103-119; Newman 2005). While power functions have been used for over half a century in analysing language and communication (e.g. Zipf’s law) they have yet to be applied to the empirical analysis of film style. This brief survey looks at the applicability of power functions in describing the distribution of the mean relative frequency of shot scales in the films of two directors – Alfred Hitchcock and Fritz Lang – whose careers encompass both European and Hollywood filmmaking.

Data on the frequency of shot scales was collected from the Cinemetrics database. Seven shot scales were used – big close-up, close-up, medium close-up, medium shot, medium-long shot, long shot, and very long shot (see Salt 2006). The relative frequency of shot scales in a motion picture was calculated by dividing the frequency with which each scale occurred by its normalising value (~500), and this data was then ranked from the event of the highest frequency to the lowest. The average value of the seven ordered relative frequencies was taken to give the mean relative frequency of each shot scale, thereby removing the problem of films that have zero frequency for a particular shot size. This data was then used to calculate predicted values for linear regression (f(x) = ax+b, where a is the slope and b is the intercept of the regression line) and power regression (f(x) = cx^α, where c is a constant equal to the frequency of the most frequently occurring scale and α is the exponent of the distribution such that Σ f(x) = 1). The coefficient of determination (R²) was used as a measure of goodness-of-fit for the predicted mean relative frequencies to the empirically observed values.

Data on shot scales was taken from the Cinemetrics database for 43 films directed by Alfred Hitchcock between 1925 and 1963, of which 23 were produced in the UK between 1925 and 1939; and from 21 films directed by Fritz Lang between 1919 and 1955, of which 11 were produced in Germany between 1919 and 1933. The results are presented in Table 1, and show that power regression provides the better model only for Lang’s German films, while for the other classes the linear model is superior. These results can be seen clearly in Figures 1-4, in which the observed values and the linear and power regression lines are plotted on linear axes. (The power regression line is straight when the log rank is plotted against the log frequency).

Table 1 Linear and power regression for the films of Alfred Hitchcock and Fritz Lang

Table 1 also shows that while the distribution of the mean relative frequencies of shot scales in the films of Alfred Hitchcock are consistent for his British and Hollywood films, there is a change in Lang’s style in his shift from Germany to Hollywood. It is also worth noting the similarity in the figures of R² for both linear and power regression for Hitchcock’s and Lang’s Hollywood films, which suggests that both filmmakers are working within a consistent institutional style (such as classical Hollywood cinema) rather than auteurist idiosyncrasies. The value c is the mean relative frequency with a rank of 1, and this too is similar for Hitchcock and Lang’s Hollywood films, reinforcing the idea of an institutional style. This does not, however, account for why Hitchcock’s British films are so similar to his Hollywood movies. It is possible that German cinema in the 1920s was different from British and Hollywood cinema in general, and that British films style was influenced by Hollywood, but a larger scale study is needed to resolve these questions.

Figure 1 Linear and power regression for the mean relative frequency (MRF) of shot scales in Alfred Hitchcock British films, 1925-1939

Figure 2 Linear and power regression for the mean relative frequency (MRF) of shot scales in Alfred Hitchcock’s Hollywood films, 1940-1963

Figure 3 Linear and power regression for the mean relative frequency (MRF) of shot scales in Fritz Lang’s German films, 1919-1933

Figure 4 Linear and power regression for the mean relative frequency (MRF) of shot scales in Fritz Lang’s Hollywood films, 1936-1956

Table 1 also shows that while the distribution of the mean relative frequencies of shot scales in the films of Alfred Hitchcock are consistent for his British and Hollywood films, there is a change in Lang’s style in his shift from Germany to Hollywood. It is also worth noting the similarity in the figures of R² for both linear and power regression for Hitchcock’s and Lang’s Hollywood films, which suggests that both filmmakers are working within a consistent institutional style (such as classical Hollywood cinema) rather than auteurist idiosyncrasies. The value c is the mean relative frequency with a rank of 1, and this too is similar for Hitchcock and Lang’s Hollywood films, reinforcing the idea of an institutional style. This does not, however, account for why Hitchcock’s British films are so similar to his Hollywood movies. It is possible that German cinema in the 1920s was different from British and Hollywood cinema in general, and that British films style was influenced by Hollywood, but a larger scale study is needed to resolve these questions.

This survey has shown that while power functions can be used to describe the distribution of mean relative frequencies of shot scales in motion pictures, they cannot be applied universally and linear functions may provide a better means of modelling film style. These distributions may be used as a measure of film style in order to distinguish between different groups of films. However, this approach cannot tell us what changes in the use of shot scales have occurred. It is necessary, then, to look more closely at where the continuities and discontinuities lie. As I have shown elsewhere, in there is a shift in the use of particular shot scales in the films in the films of Hitchcock and Lang when they arrive in Hollywood (Redfern 2009, unpublished). For both directors we find that there is a shift from distant shots to closer framing. Armed with the knowledge that different regression models explain the distribution of the mean relative frequencies of shot scales for Hitchcock and Lang prior to their arrival in Hollywood, we can extend this argument to state that: (1) Hitchcock’s Hollywood films feature closer framing than his British films, but there is no change in the distribution of the mean relative frequencies of the scale overall; and, (2) Lang’s Hollywood films feature a large change in the distribution of the mean relative frequencies as well as a shift to closer framing. In Lang’s German films it is the long shot that dominates, while in his Hollywood films there is no single shot scale that determines the films’ style.

References

Newman, M.E.J. (2005) Power laws, Pareto distributions, and Zipf’s law, Contemporary Physics 46 (5): 323-351.

Redfern, N. (2009) Shot scales in the films of Fritz Lang.

Redfern, N. (unpublished) Cinemetric analysis of shot types in the films of Alfred Hitchcock.

Salt, B. (2006) Moving into Pictures: More on Film History, Style, and Analysis. London: Starwood.

Schroeder, M. (1991) Fractals, Chaos, and Power Laws: Minutes from an Infinite Paradise. New York: W.H. Freeman & Co.