Statistical Resources: Free Statistics Lectures

The truly great thing about the internet is the amount of really good stuff you can get for free, and one of the best things if you want to learn something new is the availability of lectures from universities via media sites. YouTube has a large number of statistics lectures available for you to peruse: searching for “statistics lecture” returns 641 hits, and searching for specific topics in statistics and probability will return much more.

Here is a selection of introductory statistics lectures that are freely available on YouTube that you might want to try if you are interested in applying statistical methods in film studies and don’t have easy access to a statistician.

Possibly the best place to start is Daniel Judge’s Statistics Lecture from the Department of Mathematics at East Los Angeles College. This lecture is clearly delivered and starts with a focus on data. A good feature is that unlike some other available, this lecture is broken up into bite size chunks so its much easier to manage. Subsequent lectures in the series look at describing data numerically and graphically, probability theory, and the normal distribution.

Here’s a great introductory lecture that uses baseball to explain (amongst other things) the difference between parameters and statistics  and samples and populations (which I have commented on elsewhere), and which also explains why a batting average isn’t an average. A common problem in the use of statistics in film studies is that statistical terms are used without any proper understanding of what they mean, and this lecture goes to great lengths to explain what is meant by categorical data or relative frequency.

Math Doctor Bob has a whole series of video lectures available covering a very wide range of topics in mathematics, including statistics and probability. I’ve found his lectures on matrix algebra very useful. This is probably not the place to start if you’re a beginner since the lectures cover specific demonstrations of individual topics and often assume some knowledge of maths but they are very clear and easy to follow. Here is a lecture on how to do a two-tailed hypothesis test.

Finally, here is part 1 of Hans Rosling’s BBC programme The Joy of Stats from the Open University which is worth taking some time to watch (even if you only want to know which part of England had the highest rate of bastardy in 1842). The other parts of the programme can be accessed at the OU’s stats playlist here.

Statistical Resources: How to Read a Paper

Statistics abound and you need to be able to understand them – even in film studies (see here and here). There are many textbooks that will tell you how to do statistics, but far less attention is paid to being able to understand statistics as a consumer and – somewhat bafflingly – as a user of statistical methods. There are many good statistical textbooks but understanding of the use of statistics in research rarely features. The result is that learning statistics is a lot like being taught how to write before you have been taught how to read. It would be much easier to things the other way round.

Fortunately, there is a series of articles by Trisha Greenhalgh under the heading ‘How to Read a Paper’ published in the British Medical Journal in 1997 that do precisely this. Even better, they are freely available through Pubmed. If you are thinking of using statistics in research in film studies or if you come across statistics in the research you are reading then it would definitely help to have read these first.

The papers can be accessed at the links below:

Greenhalgh T 1997a Getting your bearings (deciding what the paper is about), British Medical Journal 315 (7102): 243-246.

Greenhalgh T 1997b Assessing the methodological quality of published papers, British Medical Journal 315 (7103): 305-308.

Greenhalgh T 1997c Statistics for the non-statistician: different types of data need different statistical tests, British Medical Journal 315 (7104): 364-366.

Greenhalgh T 1997d, Statistics for the non-statistician II: “significant” relations and their pitfalls, British Medical Journal 315 (7105): 422-425.

It may be necessary to scroll down through the pdf to find the relevant section.

Although these articles are aimed at doctors dealing with medical research the basic principles apply in all areas and are a good place to start if you want to be able understand the use of statistics in research in film studies being able to read the paper will obviously be an advantage.

 

Statistical literacy in film studies II

Previously I have argued that statistical literacy is relevant to film studies because much research on the cinema presents quantitative information in numerical, graphical, and tabular forms, and it is therefore necessary to be statistically literate in order to understand research on film industries, film style, film audiences, and film perception (see here).

This week I want to focus on a further reason for developing statistical literacy in film studies I have touched on briefly before (see here): namely, the ability of film scholars to participate in an evidence–based policymaking process is compromised by a lack of statistical literacy.

‘Evidence-based policymaking’ has become one of the key phrases of the past 15 years, and refers to ‘a policy process that helps planners make better-informed decisions by putting the best available evidence at the centre of the policy process’ (Segone & Pron 2008). Statistics have been described statistics as the ‘eyes’ of policymakers (AbouZahr, Ajei, & Kanchanachitra 2007), while Scott (2005: 40) writes that ‘good policy requires good statistics at different stages of the policymaking process, and that investment in better statistics can generate higher social returns.’ Most people involved in a decision-making process will be using data collected, analysed, and interpreted not by themselves but by professional statisticians, sociologists, market researchers, economists, and so on. It is important to recognise that while we need to be able to understand the information presented to us as part of the making of policy we do not necessarily need to be involved in the research process itself. You can criticise research even if you are not a researcher, and you can criticise statistics in research even if you are not a statistician. It is necessary, therefore, to bear in mind the difference between ‘statistical competence’ and ‘statistical literacy’ I noted in my earlier post.

A distinction can be made between people who are users of statistics and those who are provider of statistics. Whilst it may be unrealistic for professional decision-makers and practitioners to be competent doers of statistics, it is both reasonable and necessary for such people to be able to understand and use statistics in their professional practice. Integrating statistics into practice is a central feature of professions. An increasingly necessary skill for professional policy-makers and practitioners is to know about the different kinds of statistics which are available; how to gain access to them; and, how to critically appraise them. Without such knowledge and understanding it is difficult to see how a strong demand for statistics can be established and, hence, how to enhance its practical application (Segone & Pron 2008).

Participating in a policy making process therefore requires – as a minimum – the ability evaluate research and to understand quantitative information presented in a variety of forms.  The Australian Bureau of Statistics put this very clearly:

The availability of statistical information does not automatically lead to good decision-making. In order to use statistics to make well-informed decisions, it is necessary to be equipped with the skills and knowledge to be able to access, understand, analyse and communicate statistical information. These skills provide the basis for understanding the complex social, economic and environmental dimensions of an issue and transforming data into usable information and evidence based policy decisions.

If you do not understand the information provided to you, the methodologies used, and the pitfalls of both how can you make a sensible decision about which policies have been effective in the past and how can you decide which will provide the best policy for success in the future? Or, as Florence Nightingale wrote, ‘Of what use are statistics if we do not know what to make of them?’

These issues are directly relevant to film studies and its relation to policymaking for film and film education in the UK. The DCMS policy review published in 2012 recognised ‘the need for a strong evidence base for film policy’ and recommended the establishment of a ‘research and knowledge function’ for the BFI in order to

a) collaborate with industry and stakeholders to generate robust information and data on which to base policy interventions, b) assist in the design of BFI policy and funding interventions from the outset to produce learning that can inform future policy, c) actively disseminate results and learning from funding interventions, and d) over time build and maintain a valuable and accessible knowledge base for the benefit of the public, the BFI, Government, industry, academia and all other stakeholders in film.

Evidence-based policymaking has clearly arrived at the BFI, and statistics will inevitably be a part of this process. The BFI’s research outputs already have a substantial statistical component. Obviously, the statistical yearbook is the standout case here, but the Opening Our Eyes report (see here) and the recent policy review both use information presented in numerical, tabular, and graphical forms. These are intended to be used as part of the evidence base for subsequent policy making regarding film education and training (as articulated in the New Horizons document, see here), film distribution, and film production.

Other agencies also produce data-heavy reports. For example, Skillset notes that ‘research provides the evidence, authority and justification for all we do’ and includes large amounts of statistical information in its surveys. There is also much research available from the EU that is loaded with statistics. To these we can add trade publications (Screen International, Variety, etc) and academic research on the cultural economics of film (such as those papers collected together for last week’s post here). Again, this is information that is supposed to provide a basis for decision-making about UK film policy, and all of it containing quantitative information to be used as the desired evidence-base.

The ability to participate in debates is predicated on an assumption that those involved in this process are sufficiently statistically literate to be able to work with the available data and analyses thereof. However, statistical literacy is not a part of the film studies curriculum in the UK at any level. Consequently, film scholars who do not possess the required level of statistical literacy will not be able to fully engage with any evidence-based policy process. Furthermore, film studies courses are not producing graduates with the required skills to participate in debates on film policy in the UK and so this situation will not change. This cuts both ways:

• If you’re not statistically literate, how are you going to know which questions to ask of the information presented to you?
• If you’re not statistically literate, how are you going to communicate your ideas to those with ultimate responsibility for decision-making?

Since the BFI was re-constituted following the abolition the UK Film Council, film studies has to work harder to make its voice heard in the same quarters as industry bodies that have much more experience of lobbying government agencies and are much more effective at it. There is a risk that film studies will be overlooked: for example, in New Horizons ‘education’ tends to be equated with ‘training’ and academic film studies is largely absent, while the panel for the DCMS policy review did not include a single academic working on film in any field let alone film studies. Without taking statistical literacy seriously film studies will find it more difficult to make its voice heard, and risks being reduced to a passive observer of the policymaking process unable to engage in key aspects of the debate because of a lack of relevant skills in understanding the complex and varied dimensions of an issue.

The other side of this coin is that if the BFI is going to produce numerous reports containing large amounts of quantitative information and expects (deep breath) ‘stakeholders’ to participate in an evidence-based policymaking process then it needs to ensure those involved are sufficiently literate to work with statistics. Are film producers statistically literate? Is the Minister for Culture, Communications, and Cultural Industries statistically literate? Is Amanda Nevill statistically literate? The BFI has to take a lead in promoting statistical literacy in order to render consultation processes meaningful, and other film and education bodies have to follow.

The alternative is to have an evidence-based policymaking process in which no-one is able to communicate, understand, and/or challenge the evidence effectively.

References

AbouZahr C, Adjei S, and Kanchanachitra C 2007 From data to policy: good practices and cautionary tales, The Lancet 369 (9566): 1039-1046.

Scott C 2005 Measuring up to the measurement problem: the role of statistics in evidence-based policymaking, in New Challenges for the CBMS: Seeking Opportunities for a More Responsive Role. Proceedings of the 2005 CBMS Network Meeting, Colombo, Sri Lanka, 13-17 June 2005: 35-93.

Segone M and Pron N 2008 The role of statistics in evidence-based policymaking, UNECE Work Session on Statistical Dissemination and Communication, Geneva, 13-15 May 2008.

Statistical literacy in film studies I

UPDATE (21 JULY 2103): A much-revised version of this post has now been published as Film studies and statistical literacy, Media Education Research Journal 4 (1) 2013: 58-71. This article can be accessed here: Nick Redfern – Film Studies and Statistical Literacy.

A theme we will return to over the course of this year’s posts is statistical literacy in film studies.

In the recent film policy review published by the DCMS (here) it was noted that there exists an artificial division between the humanities and the sciences in education in the UK and that this unhealthy for the film industry in particular.

It was noted that some curricula already allow a wider range of subjects easily to be combined but that in general students were driven to either arts and humanities, or science courses. This was not in step with the kinds of skills and talents being sought by cutting edge, creative film companies or in the competitive arena of post-production and special effects.

The Panel recognises that it is vital to the success of the creative industries in the UK that pupils in secondary schools are made aware of the importance of studying arts and science in tandem rather than being pushed to choose between them. The Panel believes it is the synergy between these subjects that is crucial to the development of expertise in many of the creative sectors and especially in film. The Panel would like to see DfE building on proposals in Next Gen, the Review by Ian Livingstone and Alex Hope undertaken for the National Endowment for Science, Technology and the Arts (NESTA) at the request of the Minister for Culture, Communications and Creative Industries.

The NESTA report can be accessed here.

The concerns of the film policy review are focussed on the need to develop a skilled workforce that can continue to make the UK a hub for production and visual effects in the global film industry, but the negative aspects of this separation can be extended to intellectual inquiry in general.

The separation between film studies and statistics can also be viewed as antithetical to the needs of the film industry. The cinema is an industry and as such requires individuals who not only understand how that industry works (which has traditionally fallen within the scope of film studies) but also understand statistics as used in economics, management, and marketing in that industry (which is most definitely not encompassed by the film studies curriculum). Arts and sciences should be taught together, and one way to achieve this in film studies is by developing statistical literacy in film scholars.

Statistics in film studies

The study of film is a diverse field comprising four distinct but related fields of inquiry: film industries, technologies, and film policy; textual analysis; ethnographic research on audiences; and the cognitive-psychological processes of perception and cognition (see here for more detail).

Statistics is relevant to each of these four areas and film students will encounter information presented in the numerical, graphical, and tabular form in whatever aspect of the cinema they choose to study. Statistical summaries feature in many film studies texts, in newspaper and magazine articles on the cinema, and in official reports and statistical yearbooks. Indeed, the DCMS report itself uses many different statistical methods (including some really horrible doughnut graphs). Film scholars will also encounter more advanced methods in research from disciplines such as neuroscience or economics where scientific and/or statistical knowledge  is commonplace.

To illustrate the use of statistics the following provides an example from each of the four areas identified above.

Film industries

Simonton DK 2005 Cinematic creativity and production budgets: does money make the movie?, The Journal of Creative Behavior 39(1): 1-15.

This paper examines the relationship between production budgets and box office success, awards, and critical acclaim, and uses statistical terms and methods including correlation, sample, variables, mean, standard deviation, range, Cronbach’s alpha coefficient, p-values, hypothesis tests, and tables.

Textual analysis

Wang C-W, Cheng W-H, Chen J-C, Yang S-S, and Wu J-L 2007 Film narrative exploration through the analysis of aesthetic elements, in  T-J Cham, J Cai, C Dorai, D Rajan, and T-S Chua (Eds.) Proceedings of the 13th International Conference on Multimedia Modeling – Volume I . Berlin: Springer-Verlag: 606-615.

This paper uses statistical models to reveal the structure of narratives in films by analysing aesthetic features, and uses line charts, tables, flow charts, weighting functions, shape parameters, percentages, and sigma notation.

Audience research

Hardie A 2008 Rollercoasters and reality: a study of big screen documentary audiences 2002-2007, Participations 5 (1).

This paper presents the results of a questionnaire survey of audiences for documentary feature films, and uses a range of statistical methods, including percentages, bar charts, stacked pie charts, (horrible) pie charts, and tables.

Perception in the cinema

Mital PK, Smith TJ, Hill R, and Henderson JM 2011 Clustering of gaze during dynamic scene viewing is predicted by motion, Cognitive Computation 3 (1): 5-24.

This paper studies attention in viewing scenes in motion picture and uses a range of statistical methods and terms (alongside other scientific terms), including range, mean, non-linear statistics, Receiver Operating Characteristic curves, k-means clustering, histograms, line charts, tables, covariance, Gaussian mixture models, time series charts, standard error, and Bayesian Information Criteria.

Clearly understanding research on the cinema requires a relatively high level of statistical literacy, and yet I am not aware of any film studies programme that incorporates statistics as part of its tuition. Many reading the above papers will they have a grasp on what they were intended to achieve and the main results, but this is not the same as understanding why the methods used were chosen or being able to evaluate the design of a study. It is a serious failing in the instruction students receive on film studies degrees that they are expected to deal with numerical and graphical data on a regular basis without the proper training in statistical concepts and methods. For £9000 p.a. – or however much you are paying for your education – I would expect to get more than merely the gist of a piece of research.

Statistical literacy, mathematics, and the liberal arts

Statistical literacy is to statistics as art appreciation is to art

Milo Schield and Cynthia Schuman Schield

The concept of ‘literacy’ has come to mean the ‘idea of being able to find one’s way around some kind of system, and to “know its language” well enough to make sense of it,’ and foregrounds the notion of being able to ‘make meaning’ as either a producer or consumer within that system (Lankshear & Knobel 2003: 15). Education has become focussed on developing a range of literacies, such as scientific literacy, computer literacy, media literacy, and statistical literacy.

Statistical literacy may be defined as

the ability to understand and critically evaluate statistical results that permeate our daily lives – coupled with the ability to appreciate the contributions that statistical thinking can make in public and private, professional and personal decisions (Wallman 1993: 1).

Statistical literacy is directly relevant to the humanities, though it rarely features:

the ability to read and interpret summary statistics in the everyday media: in graphs, tables, statements, surveys and studies. Statistical literacy is needed by data consumers – students in non-quantitative majors: majors with no quantitative requirement such as political science, history, English, primary education, communications, music, art and philosophy. About 40% of all US college students graduating in 2003 had non-quantitative majors (Schield 2010)

One of the problems with introducing statistics into a humanities curriculum is that most students on humanities courses will have limited mathematical skills and/or low confidence in the skills they do possess. Many students may in fact be put off by the fact that film courses have some statistical content because they view it as mathematics. This problem has been widely recognised in the literature on statistical literacy, and although numeracy is a pre-requisite for statistical literacy advocates of statistical literacy stress that it is not the same as mathematics. For example, David S. Moore argues that statistical reasoning is one of the liberal arts because it is a flexible and broadly applicable mode of thinking, and prepares students.

Statistics is a general intellectual method that applies wherever data, variation, and chance appear. It is a fundamental method because data, variation, and chance are omnipresent in modern life. It is an independent discipline with its own core ideas rather than, for example, a branch of mathematics (1998: 1254, original emphasis).

From this perspective, the emphasis in early statistical education should be on statistical thinking rather than on statistical methods, prioritizing conceptual understanding rather than computational recipes. Though it may seem contrary to the goals of teaching statistics, a first course in statistics does not seek to develop statisticians. Rather it seeks to develop a set of skills and attitudes that allow scholars to be able to engage with the information presented to them. A list of goals for students in developing statistical literacy is provided by Gal and Garfield (1997: 3-5), and includes,

• understanding the principles and processes of scientific discovery,
• understanding the role of statistics in scientific discovery,
• understanding the logic of statistical reasoning,
• understanding statistical terms,
• the ability to interpret results presented in tabular, numerical, and graphical form, and to be aware of possible source of variation and bias,
• the ability to communicate using statistical and probabilistic terminology properly,
• developing a critical stance towards research that purports to be based on data,
• developing the confidence and willingness to engage with quantitative research.

The purpose in obtaining these skills is to become a statistical thinker ‘able to critique and evaluate results of a problem solved or statistical study’ (Ben-Zvi & Garfield 2004: 7).

A similar approach is proposed by Milo Schield who argues that statistical thinking is a form of critical thinking:

statistical literacy, critical thinking about statistics as evidence, is an integral component of a liberal education since a key goal of statistical literacy is helping students understand that statistical associations in observational studies are contextual: their numeric value and meaning depends on what is taken into account. The need to deal with context and confounding is ubiquitous to all observational studies whether in business, the physical sciences (e.g., astrophysics), the social sciences, or the humanities (Schield  2004).

By introducing the topic in this way to students who are already (or should be) familiar with critical thinking should make it easier to encourage them to engage with data-based arguments. It is in this context that we understand the epigram that heads this section. Another perspective is to view statistical literacy as quantitative rhetoric (Schmit 2010), which again focuses on ‘critical thinking, analysis of argumentation and persuasion, and an ability to interpret statistics in context.’

A direct parallel may be drawn between statistical literacy and media literacy. ‘Media literacy’ refers to the ability of individuals to access, understand, and create communications in a variety of contexts. It is one of the justifications for film studies and similar fields that it produces media literate citizens. Similarly, courses in statistical literacy aim to produce statistically literate citizens who are able to interpret, evaluate, and use quantitative information when it is presented to them. Since this information often comes to us via the media, statistical literacy and media literacy cannot be separated.

The role of employability in higher education may be defined as ‘equipping individuals to secure their own economic success’ (Denholm et al. 2003: 12) and covers traditional academic skills, personal development skills, and enterprise or business skills (Purcell & Pitcher 1996). Statistical literacy clearly falls within this definition, and selling such courses to students (who are paying a lot of money) needs to stress this dimension. Presenting statistical literacy within film studies in these terms is a direct response to the observations of the DCMS policy review noted above.

Statistical literacy is different from statistical competence, in which individuals function as data producers and analysers in producing original empirical research rather than consumers presented with a completed study. Naturally, we want students to develop the necessary skills that will allow them to produce high quality original research, and it is clear that much research in film studies will require the ability to design studies, collect and manage data, perform statistical analyses, and communicate those results. This depends on statistical literacy – just as you cannot write without being able to read, you cannot become competent in statistical methods without first understanding the role of statistics in empirical research, the ability to communicate ideas in tables, numbers, or graphs, or the willingness to engage with quantitative methods. Every film student needs to be statistically literate, but only those who wish to engage in quantitative research requiring the use of statistical methods need to master procedural skills.

However, I do think that every film studies post-graduate should receive some training in statsitical research methods.

Statistical literacy resources

There is a very large body of literature in the subject of statistical literacy. Fortunately, there are some excellent resource pages that gather this information and some of these are listed here.

• Statlit.org: a good place to start.
• International Statistical Literacy Project
• Journal of Statistics Education: a special issue on statistical reasoning from 2002. The paper by Joan Garfield should be read by anyone interested in statistics in film studies.
• UK Parliament’s summary of statistical literacy
• Milo Schield’s papers on statistical literacy can be accessed here.

The following papers referred to above can also be accessed freely online (other references are given below):

Gal I 2002 Adults’ statistical literacy: meanings, components, responsibilities, International Statistical Review 70 (1): 1-51.

Gal I and Garfield J 1997 Curricular goals and assessment challenges in statistics education, in I Gal and JB Garfield (eds.) The Assessment Challenges in Statistics Education. Amsterdam: IOS Press: 1-13.

Moore DS 1998 Statistics among the liberal arts, Journal of the American Statistical Association 93 (444): 1253-1259.

Schield M 2004 Statistical literacy and liberal education at Augsburg College, Peer Review 6 (4): 16-18.

Schield M 2010 Assessing statistical literacy: take CARE, in P Bidgood, N Hunt, and F Joliffe (eds.) Statistical Education: An International Perspective. Chichester: John Wiley & Sons: 133-152. (Excerpts can be accessed here).

Schmit J 2010 Teaching statistical literacy as a quantitative rhetoric course.

References

Ben-Zvi D and Garfield J 2004 Statistical literacy, reasoning, and thinking: goals, definitions, and challenges, in D Ben-Zvi and J Garfield (eds.) The Challenge of Developing Statistical Literacy, Reasoning, and Thinking. Dordrecht: Kluwer Academic Publishers: 3-15.

Denholm J, McLeod D, Boyes L, and McCormick J 2003 Higher Education: Higher Ambitions? Graduate Employability in Scotland. Edinburgh: Scottish Higher Education Funding Council.

Lankshear C and Knobel M 2003 New Literacies: Changing Knowledge and Classroom Learning. Buckinghamshire: Open University Press.

Purcell K and Pitcher J 1996 Great Expectations: The New Diversity of Graduate Skills and Aspirations. Warwick: Institute for Employment Research.

Wallman KK 1993 Enhancing statistical literacy: enriching our society, Journal of the American Statistical Association 88 (421): 1-8.