From Decoding a Graph to Processing a Multimodal Message: Interacting with data visualisation in the news media
Data publikacji: 18 lut 2020
Zakres stron: 33 - 50
DOI: https://doi.org/10.2478/nor-2020-0004
Słowa kluczowe
© 2020 Martin Engebretsen, published by Sciendo
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Data visualisation (DV) represents a text type growing in prevalence and impact in many cultural domains. A broad variety of graphs, charts, and maps – sometimes in animated and interactive forms – is found in the fields of journalism, business, education, strategic communication, and so forth (Cairo, 2016; Kennedy et al., 2016a). Some DVs are produced in simple and highly conventionalised forms, their codes presumably more or less intuitively perceived by their target users. Other DVs are more complex in form and content, and thus more demanding for users to decode and make sense of (Engebretsen, 2017). Research on DV reception is, however, scarce, and the impact that specific design elements have on the reception process is an issue calling for closer investigation. In this context, “reception” refers to the entire process of individual interaction with the textual product in question: decoding, interpreting, and evaluating the semiotic content, as well as scrolling and navigating the website.
In journalism, DV has become a field of high priority (Engebretsen et al., 2018; Rogers et al., 2017). In most major newsrooms, highly specialised groups consisting of data journalists, developers, and designers construct advanced forms of visual storytelling based on data (Weber et al., 2018). The purposes of the DVs are manifold: to make complex matters more accessible; to add evidence to claims made in the verbal text; or to let the readers explore huge datasets on their own (Engebretsen et al., 2018). DV may be defined as visual representation of data, created to facilitate understanding (Kirk, 2016). Yet, the aesthetic aspect also plays a significant role in the production and consumption of DV (Brinch, forthcoming; Cairo, 2013; McCandless, 2009), potentially conflicting with the desire to mediate facts and patterns (Prince & Archer, 2014).
The language of graphics is normally not taught in school in a systematic manner, for example compared with how pupils learn verbal languages. The “vocabulary” of the graphical “language” – as known to professional DV designers – consists of a large number of DV types, each of them with the capacity to represent specific kinds of relational information, such as development over time, comparison between units, patterns of distribution, networks of connections, and so forth See, for example, The Data Visualisation Catalogue (n.d.), explaining the affordances of 60 different types of DV.
This situation, in which an abstract visual language is slowly evolving in genres of high political and cultural significance, calls for extensive empirical investigations on how citizens interact with data visualisations in their everyday lives. Aiming to contribute to the investigation of this question, this article draws on a qualitative user study that combines observations from reading sessions with ten in-depth interviews. The objective of the study is to nuance our knowledge of how digital DVs are read by young users by revealing patterns and variations in the data collected in a critical case sample. The results from the study serve to identify areas of literacy and practices of reading not well described in the research literature, and they have relevance for DV designers, educators, researchers, and students in visual communication and related fields.
Although this study is data-driven more than theory-driven, it considers the phenomenon in focus from certain perspectives, namely media literacy, multimodal literacy, and – to be even more specific – DV literacy. Starting out from a cognitive perspective, W. James Potter (2004) states that modern media consumption most often leads to a superficial understanding of issues. He advocates the training of a media literacy that involves a “personal construction” of meaning. Potter defines media literacy as “the set of perspectives from which we expose ourselves to the media and interpret the meaning of the messages we encounter” (2004: 58). He underlines that a strong perspective on the media depends on a good set of knowledge structures, developed through complex cognitive processing of information – harvested both from the media and from other real world experiences. These processes include the application of basic cognitive skills, including analysis, synthesis, abstraction, and evaluation. The more we use our knowledge structures when encountering media content, the more able we are to use the media exposure to meet our own goals. However, our knowledge structures are not developed solely by cognitive processes – they are also formed by emotional, aesthetic, and moral elements. Regarding these four types of processes that contribute to the shaping of knowledge structures, Potter (2004: 60) writes that “the combination of any three types helps provide context for the other type”.
Potter's view on media literacy equals in many aspects the view formulated by Gunther Kress (2003); however, Kress focuses more directly on the multimodality of modern media discourse, a topic not accounted for by Potter. In his work, Kress unpacks the concept of “multimodal literacy”, working to support individual agency in a world where visual modes – rather than verbal ones – are becoming the centre of communication. According to Kress, multimodal literacy involves the ability to identify 1) what semiotic work is done by each different mode involved in a text (e.g., words, numbers, images, diagrams, etc.) and 2) what meanings spring from the various interactions of the modes.
According to Tønnessen (forthcoming), multimodal literacy is a key factor in the literature on DV literacy, dealing with the users’ capacity to understand, evaluate, use, and appreciate data visualisations they encounter in their daily life. Tønnessen emphasises the complexity of DV literacy – calling it a “visual-numeric literacy” – as competent use of DVs demands an understanding of not only the graphical resources (lines, bubbles, axes, etc.) and their composition, but also their relations to the data and the statistical values they represent.
A large part of studies within the field of DV reception is related to the field of human-computer interaction and is more concerned with perception and task performance than with interpretation, emotion, and reflection. Studies involving eye-tracking methods have explored reading patterns in detail, often with a focus on the perceived interplay between text elements and visual forms. Several of these studies reveal that the application of data visualisations can strengthen the understanding of a news story, given that the coherence between text and visualisation is evident semantically (due to shared content and focus, see de Haan et al., 2018) as well as compositionally (regarding the position of units, see Holsanova et al., 2009; Norman & Roberts, 2015). In an eye-tracking study including 122 informants reading news stories illustrated with data visualisations, de Haan and colleagues (2018) found that the informants spent much more time reading the verbal text elements than studying the visualisations. They concluded that most readers used the visuals to gain oversight, not to process detailed information. Burmester and colleagues (2010) found that the 14 informants in their study hesitated to click on interactive features, and that narrative elements evoked motivation, but also took attention away from other text elements.
Kennedy and colleagues (2016b) explored the relations between data visualisations and emotions, using focus group interviews and diary keeping. They concluded that emotions can be evoked by a number of factors, including the meanings of the visualised data, the visual style of the DVs, the situation of reading, and readers’ self-confidence regarding their ability to make sense of the graphs and charts.
The empirical literature on DV reception reveals several gaps, including issues important in modern DV design. One of them concerns the use of dynamic, video-based platforms for DV presentation. Another gap concerns the reception of visual metaphors in DVs – many DVs used in public channels or in educational contexts apply forms alluding to well-known physical phenomena like houses, mountains, or vehicles in order to facilitate understanding though metaphoric association (Engebretsen et al., 2018). An interview study with 30 expert informants in ten major Scandinavian newsrooms revealed that producers of DVs in the news media very rarely conduct user studies themselves (Engebretsen et al., 2018). Their knowledge about their readers is mainly based on automatically generated statistics, which do not convey information about how readers interpret and respond emotionally to textual products.
Based on the sample of informants, the literature on media literacy, and the gaps in earlier research, we can now reformulate the research question and break it down into more focused fields of enquiry. The study asks:
What characterises young students’ interaction with DVs found in news media concerning text traversal (the pattern of continuity in the reading process, including any instance of stopping, scrolling up or down a web page, rewinding a video, etc.), interpretation, and emotional response? How do young students interact with DVs presented on a dynamic, video-based platform? How do young students interact with DVs including figurative visual metaphors?
The study starts out from a general theoretical framework of literacy and some knowledge concerning earlier studies; yet, on the whole it is an explorative study with no clear expectations of how young students actually interact with DVs found in the news media. Thus, an approach inspired by grounded theory was chosen in order to develop knowledge representative of the data (Bryman, 2012). The most interesting patterns and variations found in the data are related to earlier research and relevant theory. Implications for designers are also discussed in the final section of the article.
In the study, think-aloud reading sessions were combined with qualitative interviews (cf. Bengtsson, 2018); ten informants were asked to read two texts at their own speed and to comment on their reading process while reading (with minimum interference from the researcher) In this article, both “reading” and “text” are to be understood in the wider sense of the concepts, and include, for example, watching video.
The informants consisted of five men and five women, all except one aged 20–29, and were recruited from a university canteen in Kristiansand, Norway, where students from several educational institutions come to eat and socialise. Although Norwegian students represent a broad and heterogeneous group of young adults, a sample of university students is in this context a critical case sample: all the informants have been through different forms of DV training in high school, they have chosen an educational path demanding a high level of literacy, and they represent the DV users of the future. Problems reported by this group are likely representative of those encountered by the general population. In order to achieve a wide array of scientific backgrounds, students from a range of study programmes were selected.
Individual informant sessions lasted 40–70 minutes and took place in a silent room near the university canteen; the focus of the study was not revealed until the reading session was over. Anonymised information about the informants (age, study background, and interest for the topic) is provided in Appendix 1. Nicknames are used in the summaries below, and I have translated all citations from the informants from Norwegian to English.
Screen-capture software provided a video file for each session, displaying everything that happened on the screen during the reading session (scrolling, video rewinding, etc.) and including all comments uttered by the informants. After finishing the process of data collection, I replayed and closely observed the video files. Every element of action made by the reader (scrolling up, stopping or rewinding the video, making a comment, etc.) was documented in written notes. The captured data from the reading sessions were thus “translated” into written descriptions.
The interview transcriptions were coded following an iterative process of different coding techniques, inspired by grounded theory (Bryman, 2012). In the first step, following the procedures of “open coding”, the transcripts were organised into thematic categories according to their content. In this process, written records of ideas (memos) were produced in order to facilitate the next step, where the categories from the open coding were grouped and linked through a process of “axial coding”. Finally, the most relevant characteristics of the phenomenon under scrutiny became apparent through a process of integration, refinement, and contextualisation of the axial codes – “selective coding” (Bryman, 2012). The two final steps were guided by the reformulated research questions, the memos, the findings from the reading sessions, and the interview guide. The entire process of coding and analysis was done with the software Nvivo.
The number of informants in the study is limited, and the participants are all collected from a population of students. Although the informant group functions as a critical case sample, one must assume that a larger group of participants, more heterogeneous with regards to background and age, would possibly provide different results. Further, the situation of data collection, where the participants read textual material according to the commands of the researcher, is not a natural reading situation. Thus, the situation of use probably had a certain impact on the data collected in the study.
The text material applied in the reading sessions consisted of four text units; two were based on video and two were static with various data visualisations integrated. Hereafter, they are referred to as “videos” and “static texts” (although the static texts also include smaller animated elements). Climate change was chosen as a topic because it is a social and political issue most people can relate to and, at the same time, public communication of numeric data plays a significant role. All the text units are published on the NRK (the Norwegian national broadcaster) website on pages designed with a video on top and corresponding static text directly below.
Since the content of each video is similar to the content of each related static text, they were presented to different groups of informants; half of the informants (group 1) only read the static texts, while the other half (group 2) only watched the videos. Designing the study in this way allowed for investigating the reception of various DV forms – as presented in static and dynamic textual environments respectively – without one format influencing the reception of the other. The intention was not to compare the readings of the two groups through an experiment-like research design, but simply to register responses to two different media forms applied to the same content. A description of the four text units follows.
Video 1 is 3.08 minutes long and includes a series of animated line graphs. The first graph shows the temperature development on earth during the last 135 years (see Figure 1). The following graphs show how different natural and human-made factors (e.g., volcanos and greenhouse gases) have influenced the temperature. Throughout the video, a visible female host functions as a guide to the meaning of the graphs, ending up with the main message of the presentation: It is only when we include the CO2 emissions that the model fits the actual measurements. This point is illustrated at the end of the video, with a canvas showing all the individual graphs simultaneously.
Figure 1
The static text following video 1 contains almost exactly the same information, with only small variations in the verbal formulations. The text begins with a title asking, “What is actually warming up the planet?”, followed by a short introduction offering information concerning the data sources and origin of the design idea. The graphs that follow are the same as those in video 1, now integrated at relevant spots in the linear flow of written text. The graphs are animated, with graph lines in different colours automatically moving quickly from left to right when the reader has scrolled down to the relevant position in the text flow. At the bottom of the page, there is an interactive element where the reader can choose which of the graphs shall be highlighted in a picture where all graphs meet (see Figure 2).
Figure 2
Video 2 is also about climate change and CO2 emissions; however, the use of graphics is quite different. Here, the designers applied circular balls in the shape of a metaphoric mountain, representing CO2 emissions almost burying Oslo (see Figure 3). Similar balls are also used in the form of bar graphs (see Figure 4). The video lasts for 1.38 minutes, and the presentation is driven by a narrating voice.
Figure 3
Figure 4
The static text following video 2 is more different from the video than in the first pair. It is more focused on the data sources behind the video and how the data was extracted and processed, and several of the visualisations are complemented with numeric tables. Yet, it repeats all the visualisations and main points made in video 2 (only in a static text-image format), again applying the circular balls both in the mountain and in the bar graphs to illustrate CO2 emissions (see Figures 3 and 4). To a large degree, it functions as an autonomous text, and presumably, many readers visiting the web page will scroll down to the text without watching the video first.
The below analysis is presented in two parts; the first is concerned with the screen-capture data from the reading session and the second with the interview data. Both parts contribute to the results discussed in the last sections of the article.
The screen-capture data offers insights into the reading behaviour of the informants, and to the problems they were faced with while interacting with the (static and video-based) online texts.
The time used to read and process the text – including scrolling, clicking, and re-reading – varied between the five informants in group 1, who read the two static texts. The reading time ranged from Anna, who used a total of 18 minutes on both texts, to Erik, who used only 5 and a half minutes. The group's average reading time was 12 minutes, 5 seconds. The screen-capture recordings show that the informants reading the static texts quite often, though to a varying degree, scrolled upwards in order to re-read the content of earlier text elements – the five informants watching the videos were much more reluctant to stop and re-read. There were a few stops of video 1, while no one stopped the progress of video 2, although most of the group 2 informants reported after the reading session that it offered too much information in too short time.
Re-reading can sometimes indicate a lack of focus and concentration; however, re-reading can also indicate a strong motivation to gain better understanding. This was illustrated by William in group 2, who suddenly stopped the video he was watching and said, “Hey, this graph was a bit too quick. I will rewind it just to take a closer look at the impact of those greenhouse gases on the temperature”.
A core aspect of literacy is the ability to draw inferences, creating coherence where the textual cohesion may seem weak – in other words, “bridging gaps” in the text (e.g., Kispal, 2008). Weak cohesion may occur between sections of a given text, or between semiotic modes (cf. Halliday & Hasan, 2013/1976; van Leeuwen, 2005). Holsanova and colleagues (2009) emphasise that the positioning of the visual and verbal elements in a DV is crucial to the reader's establishment of coherence. Several findings in the data material – both the screen-capture data and the interviews – relate to this aspect of literacy.
Anna reported that while reading – like several others – she had a problem deciding whether the circles used in the bar graphs and the figurative mountain referred to the same thing. She said: “I do not understand whether the balls in the columns mean the same as the bubbles of the mountain”. When Dorte spoke of a similar problem, it became clear that the problem was related to both multimodal cohesion and sequential cohesion. She said, studying the DV displayed in Figure 4: “This was hard to understand. Fifty-four of what? Did it say, anywhere? Is it tons, perhaps?” It is necessary to closely read the surrounding text elements in static text 2 to understand that the balls sometimes mean tons and sometimes 1 million tons, depending on whether they refer to individual persons’ emissions or emissions from the nation as a whole. However, both Anna and Dorte revealed a good understanding of the main message of the text; thus, their struggle on the detail level did not mean that they had problems understanding the text as a whole. On the contrary, when they sometimes struggled with getting the details right, it indicated that they kept a strong focus throughout the reading process, making good observations of textual unclarities.
An important aspect of media literacy is to synthesise and evaluate the information received in the reading process by activating earlier knowledge and beliefs (cf. Potter, 2004). Most of the informants in the study did not evaluate the DVs until they were asked to do so in the interview following the reading session – yet, some did. Erik was the fastest reader, but still took the time to utter a critical evaluation of the information conveyed by the graphs. After reading the last, summarising graph, he said: “Sun and volcanoes... I don’t believe that, really. Greenhouse gases… well, maybe. But the aerosoles… Ok, so, less car driving means higher temperature”. His comment indicates that he engaged critically in the reading, involving his former knowledge and beliefs in evaluating the text. This process led him to question the text's main claim and instead focus on an “alternative” point made in one of the graphs that fitted his existing beliefs better – that car driving produces dust (aerosoles), which shadows the sun and makes the temperature fall. This illustrates what we may call “selective interpretation”, or what is known as “motivated reasoning” in the field of cognitive psychology Motivated reasoning refers to the cognitive strategy of forming conclusions aimed at supporting one's existing beliefs and thus mitigating cognitive dissonance (cf. Kunda, 1990).
Another important aspect of media literacy is the ability to use technological features in order to explore and engage with the material offered through the media interface (Jewitt, 2006). In the study, all informants engaged with the basic interactive features of scrolling up and down the static texts (group 1) or starting and stopping the videos (group 2). But when it came to activating links in order to retrieve supplemental information or manipulating the visual appearance (colour saturation) of a multiline graph, the level of activity was much lower. No one in group 1 clicked on any links embedded in the text in order to explore, for example, the concepts of “solflekker” [sunspots] or “klimagasser” [greenhouse gases]. And when arriving at the last, multi-line graph, only one of the five informants (Erik, the fastest reader) took the time to click 8–9 times on the buttons (which is necessary in order to investigate the different patterns and combinations thoroughly). Two of the five informants did not click at all. One of them was Anna, who said after the reading session that she did not understand that these were clickable buttons. This reluctancy to explore the entirety of a textual universe by following electronic links is documented in several studies (e.g., Burmester et al., 2010).
This part of the study revealed that the informants applied different strategies when approaching a digital, journalistic text involving DVs. The differences were most prominent among the informants in group 1, who read the static texts. Erik demonstrated a strategy of reading quickly through the entire text – including a quick exploration of interactive features – and thereafter evaluating how the text relates to former beliefs. Anna demonstrated a very different strategy, taking more time to read and re-read the text, stopping where the relations between graphs and co-text seemed unclear, and focusing so strongly on the linear text flow that no detours in the form of clicking on interactive features were taken – or even identified. The other three group 1 informants demonstrated varying degrees of both these strategies. The reading session also revealed that the informants in group 2, who watched the videos, spent less time on processing the content of the DVs because they tended to follow the progress of the video without pausing or rewinding it.
In this section, we will first look at how the two groups of informants responded to the line graphs and their connections to the written or oral verbal texts, referring to static text 1 and video 1, respectively, as well as their general impressions. Thereafter, we will look at how the total group of ten informants responded to the figurative, metaphoric elements – the balls and the mountain – in static text 2 and video 2.
The general impression in group 1 was that static text 1 was interesting and clear, and that the line graphs represented a necessary element in the totality; however, they perceived the role of the graphs differently. Anna said that the graphs made her use more time on the text, making her think more consciously about what she was reading. Tor said that the graphs provoked more feelings in him than what words alone would have done. Frank was concerned with the visual attractiveness of the graphs, and said: “Once you add some colours to an otherwise rather dead article, of course it catches your interest”.
All group 1 informants except Frank used most of the time reading the verbal text and less on exploring the graphs. This finding corresponds with results reported by de Haan and colleagues (2018). It should be noted that the main points made by the graphs are summarised in the verbal text; yet, the graphs offer more detail on how the different environmental factors have developed during the reported time period. Some of the informants reported that they could recognise a specific pattern in the interplay between the verbal text and the DVs. Frank stated: “I got the sense of a strong thread running through the text, that the verbal elements somehow made certain claims, which then was documented in the following graph”.
Several of the informants in group 1 responded that the graphs needed more information on the axes, especially the x-axis. The x-axis represents the timeline 1880–2015; however, it does not show any numbers between the two endpoints. Anna suggested that this lack of year numbers weakened the cohesion between the verbal text and the DV: “Sometimes the text mentioned specific events, like volcano outbreaks, and I could not relate that clearly to the graphs, just going up and down”.
The informants expressed diverging opinions with regard to the animation effect of the graphs presented in static text 1. For Erik, a static graph would have worked just as well; Tor and Dorte reported that the animation captured their eye, but it moved too quickly for them to really follow. They commented that the animation made the graphs stand out in the text composition, but it did not convey any additional information. Tor said: “I think animation often works well to show some kind of development over time. But in this case, I did not have time to think it over, really”.
Similar to the informants in group 1, the participants in group 2 generally reacted positively to the application of line graphs in video 1. This general response correlates with findings in a study based on interviews on the production side, where designers of journalistic DVs stated that simple graphs were most often the most effective ones, even in a time where more creative charts flourish (Engebretsen et al., 2018). One of the informants, Oda, expressed that the video managed to create an element of suspense, because the individual graphs did not fit with the measured temperature, building up to a climax where all the graph lines together made a result corresponding better to the measured temperature. When asked to explain the main point of video 1, three of the five informants explained it correctly: the factors influencing temperature variations cannot explain the measured temperatures unless CO2 emissions are also included. The other two informants had different interpretations, apparently focusing more on the details than the totality. Mona said: “The video shows that everything is connected, that you cannot keep the different factors apart. I think that was the main message”. Oda said: “Ok, so the climate changes in a sort of cycle, it all varies, whether we have emissions or not”.
Group 2 informants were overall positive to the personalised presentation of the data in video 1, where a visible narrator guides the viewer through the entire presentation. Stine gave spontaneous responses about the narrative qualities of the video medium: “It felt different. There was a constant movement forward, and her explanations made it easier for me to understand what was going on in the graphs”. Yet, three of the five informants in group 2 found it quite hard to follow due to the speed and content density. So, why did they not stop it? Mona stated that she did not rewind as much as she needed due to the flow created by the narrator: “I hesitated to stop and rewind, because the narrator made me a more passive listener”. William said that his way of using such a resource is very context dependent: “I think I would have stopped it if it was related to some school activity, or if we were going to have a test. But I would not have stopped it if I just were looking at it at home”.
This section focuses on the reception of the metaphoric mountain of CO2 balls and the application of similar circular forms applied in bar graphs (see Figures 3 & 4). As these forms are used in both static text 2 and video 2, we will look at the ten informants as one group. Both the bar graphs and the metaphoric mountain have circular figures – looking like balls – as their basic visual unit. These balls are used in very different DV types – bar graphs and a metaphoric mountain – but in both cases each ball represents a certain quantity of CO2.
To some of the informants, the use of balls made more sense in the bar graph than in the mountain, for various reasons related to accuracy or concreteness. Geir said: “I felt [the balls] gave me more information in the bar graph. There, I could read the values of the emission quite exact. That was not equally easy when they were just piled up [in a mountain]”. Likewise, Oda and Stine found the use of balls helpful, especially in the bar graphs. They reported that – knowing the meaning of each ball – the bars were perceived as something more concrete than what abstract rectangles would have been. Stine said: “When you get to know these balls, they give more meaning to me than just one column being a little bit shorter than the other”.
Others saw it differently. Mona looked at the bar graphs just as piles of balls: “I did not compare the different countries much, as I usually would with regular, straight bar charts. I think I focused too much that these just were some kind of balls. What I noticed was that Norway only had two balls on top of the rack...”
Some of the informants expressed that they liked the impression of a mountain of CO2 balls almost covering a big city. Frank said: “I think it is a good idea to make something very abstract into something so concrete. Even a child can understand the point being made”. They also felt that the mountain made an emotional impression. Anna did not think of the mountain as a DV per se but saw it mainly as a metaphor indicating something surprisingly big: “I liked the mountain, it was cool. But all those dots, well, I did not reflect much on those, although they were the reason that it looked the way it did”.
In contrast, several of the informants found the mountain problematic. Erik felt that the mountain metaphor was too emotionally manipulative: “This is a more journalistic take on it, almost more propaganda than information. It says, ‘here is a big mountain, see how much we emit!’”. William had problems understanding what the mountain actually told him: “The mountain made it seems like a really huge amount of CO2. But I do not understand clearly how much it actually is, or how serious that amount is”. Thus, he pointed to the same problem as several other participants: the figurative and metaphoric design of a CO2 mountain may provoke an emotional reaction due to the size of it, yet the implications remain vague, as it does not offer any values for comparison (what bar graphs normally do). Oda reported that she liked the mountain, but the associations it gave her indicate the risk of over-interpretation and strayed association that visual metaphors may cause (cf. Sperber & Wilson, 1995). She said: “This is really good. If all this CO2 was falling over Oslo, everyone would have died”. Thus, she associated the almost buried city with a sudden natural disaster, quite different from the slow processes of CO2 emissions and climate change.
When it comes to emotional impact, it seems clear that the mountain DV had the strongest effect on the participants. For some of them, the emotional effect included a sense of responsibility. Dorte said that the mountain made her think that “ten of those balls are actually mine”. Others confirmed an emotional response, but doubted that it would last long. Geir said the mountain made him think, “Oh, I must be more careful about this”, yet he added that this emotional reaction would quickly fade, because the claims made in the text were perceived as too general and superficial. He said: “Numbers and facts can be fine. But it is more important to understand what is actually happening. What is actually destroyed. What the specific consequences of our emissions actually are”.
This study brings attention to certain potentials and challenges related to the reception of DVs in a digital news media context. The challenges the ten informants faced illustrate various aspects of media literacy. Some of these concern media literacy on a basic level – more specifically, the learning and practicing of the reading conventions involved in DV (cf. Kennedy et al., 2016a). These conventions are related to the graphic “vocabulary” (the visualisation types); the multimodal interplay between graphics, verbal text elements, and numbers; and the options for interactivity offered by the technology involved.
The informants in group 1, reading only the static texts, had a higher tendency to stop and re-read, compared with the informants in group 2, who very rarely stopped the videos. Thus, the time spent on processing the multimodal information was considerably shorter for the video viewers than for the readers of static versions, and the feeling of information density and complexity correspondingly stronger. Two of the five video viewers were not able to retell the main message of the line graphs correctly, though the dramaturgic and sensoric effects of the video medium were perceived as adding attractiveness and encouraging more attentiveness to the presentation.
Comparing the reception of the line graph with the reception of the figurative and metaphoric mountain of balls, the study revealed that the line graphs to a high degree were perceived intuitively and correctly by the ten informants, especially when they displayed only a single line. The metaphorical DV evoked a stronger emotional response, but the exact meaning of the visual elements remained unsettled for several informants. Among the most interesting findings in the study is the variation in the participants’ responses to the figurative, metaphorical forms of DV. Some informants gave their attention to the symbolic value of the figurative shapes (e.g., the “bigness” of a mountain) without losing focus on the abstract, non-naturalistic coding orientation of the expression Coding orientation is a term referring to the semiotic nature of a class of expressions. A photo is interpreted according to a naturalistic coding orientation, while a graph is interpreted according to an abstract coding orientation (cf. van Leeuwen, 2005).
The broad spectrum of interpretations evoked by the visual metaphors in this study can be explained by the semantic openness that characterises most metaphors, whether presented in verbal or visual form. A core issue in metaphor theory is the capacity of metaphors to bring abstract phenomena closer to our everyday world through comparison. However, metaphors are ambiguous by nature, and thus demand active interpretation from the reader in order to add relevant information to the issue in question (Sperber & Wilson, 1995). In the field of DV, figurative visual metaphors are used in order to connect both rationally and emotionally with a broad audience (Cruz, 2015). Cruz states that such visual metaphors add a layer of non-data elements to the visualisation, carrying the weight of subjective authorial intent. Further, they may cause “confusion in the viewers when distinguishing what is data related from what is not” (Cruz, 2015: 20). In the context of statistics and DV, we might thus say that visual metaphors represent an element of aesthetification, in the sense that they open up a space for engagement and emotion and at the same time room for subjective association and reflection – beyond the control of the text producer (cf. Freeman et al., 2018).
The second aspect of media literacy highlighted by the study, following Potter (2004), concerns the processing of information on a more advanced level: connecting it to earlier knowledge and evaluating its value for personal opinion and action. On this level, Potter also points to the role of emotional, aesthetic, and moral aspects of media literacy and interaction. Several of the informants expressed a feeling of surprise, triggered mainly by the metaphoric representation of CO2 emissions in the form of a mountain almost covering a big city. One informant, Erik, felt that the mountain of CO2 was almost more propaganda than information, revealing that he was able to spot an authorial presence behind the aesthetic style of the presentation. According to Potter (2004), such an ability, where cognitive and aesthetic processing and evaluation is combined, indicates a high level of visual literacy. Erik's responses to the aerosoles indicate, however, that a high level of visual literacy does not “protect” a reader from mis-interpreting a message due to motivated reasoning (cf. endnote 4).
Some informants said that the mountain of CO2 triggered a feeling of responsibility and awareness of their own habits. This response resonates with findings reported by Allen (2018) after a similar experiment with focus groups responding to visualisations of scientific data. He found that surprise is an emotion closely connected to the motivation for further engagement with the text and the subject. The present study suggests, however, that surprise must be connected to substantial knowledge about the subject matter in order to provoke a durable engagement; some informants said that their emotional response probably would not last long, because it was not anchored in substantial knowledge about the seriousness or implications of the situation depicted.
As an explorative, qualitative study based on think-aloud reading sessions and in-depth interviews, this study offers some nuances to our knowledge of DV reception and media literacy that quantitative methods could not provide, and further, they are not commented on in the general literature on media literacy. To gain further insights into the potentials and challenges of DV reception in the digital age, we certainly need more studies investigating the conditions under which specific visual forms enhance the comprehension, application, and appreciation of statistical information. Yet, some of the present findings may serve to support certain views relevant to designers producing DVs for a general audience: 1) the cohesive links between the verbal and visual elements must be easy to identify; 2) graphs should be kept simple, but not at the expense of relevant information on the axes; 3) interactive options should be explicitly marked; 4) figurative metaphors should have clear signals for which aspects are data-driven and which are not; and 5) in video-based media, the speed of progress and information density in DVs should be considered carefully.