Images hold certain qualities that make them powerful in communicating issues like climate change: making visible, tangible, and relatable what is otherwise complex, abstract, and distant (O’Neill, 2013; Rose, 2016). However, while seemingly presenting the real-world one-to-one, images are neither objective nor neutral, but play an important role in framing an issue. While this has made climate images a subject of growing scholarly interest, most studies have focused on iconographic motifs (Manzo, 2010; O’Neill & Hulme, 2009), as well as images from established news media (Ahchong & Dodds, 2012; Kangas, 2019; O’Neill, 2013), nongovernmental organisations (Doyle, 2007), and scientific publications (Mahony & Hulme, 2012). Climate images circulating on social media platforms have, with few exceptions, so far been largely overlooked. This is curious, since images shared on social media have a dialectic relation to larger sociopolitical events in society, simultaneously reflecting broader trends and serving as a powerful tool for the public's engagement in these events. They impose particular ways of “seeing the world” (Rose, 2016) and engage audiences by evoking emotions, facilitating memory, and transmitting cultural meaning (O’Neill & Nicholson-Cole, 2009).
While some visual studies of climate change have examined social media images (Hopke & Hestres, 2018), they have done so mostly from a single-platform approach, while looking at a snapshot moment in time. With rare exceptions (Niederer & Colombo, 2019; Pearce et al., 2020), this leaves both a gap of cross-platform research on how climate change is visually debated on different social media platforms, and a gap of temporal frameworks that study how the visual social media debate develops over time. Moreover, a majority of earlier studies have used qualitative, small-sample, or manual approaches that are not well equipped to navigate the large number of digital images that are produced and shared online. This has been problematised by, for instance, visual scholar Gillian Rose (2016), who calls for development of novel methodologies that incorporate the computational into visual studies.
Meanwhile, scholars in digital research have for years pioneered the use of digital tools to study large-scale patterns in climate change communication on social media. A valuable example can be found in Munk (2014), who uses Facebook data to map how climate disputes about wind energy unfold online. This study is inspirational in showcasing the potentials of using digital methods together with network analysis for “controversy mapping” (Latour, 2005a; Marres, 2015; Rogers, 2013). Another project by Marres and Gerlitz (2016) studies the liveliness of the climate debate on Twitter, demonstrating the potential of using digital tools to capture its changing dynamics. Both of these studies – along with others from digital social science – show great potential for using computational tools to study online climate communication. Meanwhile, these and other digital studies (Jang & Hart, 2015; Veltri & Atanasova, 2017) have primarily analysed the climate debate through social media texts, since tools for processing images – compared with natural language processing – have taken longer to refine. This has created a bias towards Twitter studies and textual data, and left visual platforms like Instagram critically understudied, as highlighted by Highfield and Leaver (2016). The lack of big-scale studies of visual content from social media, we argue, makes it crucial to start bridging the computational turn and visual turn of social science, exploring how tools like image recognition can expand digital social science to include visual data.
The article at hand takes up this challenge of developing a digital-visual methodology that opens up opportunities for the study of large-scale visual data with cross-platform and temporal sensibilities. We develop and explore the application of such a methodology through a two-fold case study. First, we set out to generate insights on what visual motifs are mobilised in climate images on Twitter and Instagram, focusing on how the human–nature relation is visually depicted on the two platforms, following recurring scholarly attention to this divide in shaping the climate issue (Latour, 2004; Morton, 2007). Second, we explore temporal patterns in how the debate has changed on Twitter from 2015 (when the Paris Agreement was established), through 2016 (when former President Trump was elected and popularised the notion of “climate hoax”), until 2017 (when Trump announced the US withdrawal from the Paris Agreement). It is imperative, we think, to ask how online climate communication has responded to and changed in light of these significant political events.
The methodological anatomy of this project is an exploratory and descriptive, rather than explanatory, social science, following Latour's (2005b) call for shifting focus away from underlying causes and all-explaining theories. Taking a methodological path, this project situates itself within digital methods, an emerging field in social science concerned with re-purposing digital platforms for studying the social through the growing availability of large-scale digital data (Marres & Gerlitz, 2016; Rogers, 2013). As a framework, digital methods does not prescribe a certain tool or method, but implies redirecting focus away from methodological separations of the qualitative and quantitative, which have been heavily problematised within empirical social science (Savage & Burrows, 2007). Instead, digital methods urge us to leverage advances in computational tools and the granularity and scalability of digital data to take a “quali-quantitative” approach that moves between micro and macro levels in data, reconciling the quantitative dimensions of large-scale data with the qualitative sensibilities needed to understand it (Lindgren, 2020; Ruppert et al., 2013; Venturini & Latour, 2010). Adhering to this, we assemble a combination of digital tools to collect, code, and visualise data in a way that empowers us to map cross-platform and temporal dynamics in climate images.
Data for the project was collected in December 2018. Netlytic (Gruzd, 2016) was used to capture Instagram data and Twitter Capture Analysis Tool (TCAT) (Borra & Rieder, 2014) to collect from Twitter,
For a longer debate on public/private in post-API Internet research, we refer to Freelon (2018) and Perriam and colleagues (2019), but rely here on the fact that at the time of collection the data used in this project was made public both by the users and platforms through open APIs.
To enable a temporal analysis of Twitter images in relation to COP21 (United Nations Climate Change Conference) in 2015, COP22 and the simultaneous election of Trump in 2016, and the US withdrawal from the Paris Agreement in 2017, we collected data from Twitter in 30-day periods around these events (see the exact dates in Table 1). Data collection was done in collaboration with Digital Methods Initiative at University of Amsterdam, who kindly gave access to their TCAT server with historic Twitter data on the climate debate. To enable a cross-platform comparison of the 2017 debate, we collected a 2017 sample from Instagram. Due to Instagram's application programming interface (API) not allowing historic data collection, the two platform samples are not from the same dates. While this reduces the 1:1 comparability of our data samples from Twitter and Instagram, we still find them useful for our purpose, since they are both collected in a 30-day period, and both from periods
Overview of data sample from Twitter and Instagram
2015 | 29 November–28 December | COP21: 30 November–12 December | 169,220 | |
2016 | 29 October–27 November | COP22: 7–18 November | 31,931 | |
2017 | 18 May–16 June | US withdrawal from Paris Agreement: 1 June | 46,567 | |
2017 | 4 October–2 November | US withdrawal from Paris Agreement: 1 June | 14,375 |
To analyse the content of these images, we used the image recognition software of Google Vision AI to annotate all images. Specifically, we used the AI's object detection feature to identify visual content (Google Cloud Platform, 2018), returning a list of objects detected for each image. For simplicity, we call these “image-objects” and reference them with the “
With inspiration from scholars like Ricci and colleagues (2017) and Niederer and Colombo (2019), we use visual network analysis (Venturini et al., 2019) to map how image-objects appear together in climate images, leveraging network analysis both as a heuristic tool to get an overview of our datasets, and to offer an illustration of network analysis findings (Bastian et al., 2009; Jacomy et al., 2014). With this method, we operationalise a re-orientation of visual analysis away from looking at individual images as stand-alone, confined entities (Rose, 2016), and on to seeing images as assemblages of visually related objects. Rather than analysing visual content of images one by one, we look across all our images and analyse how visual objects appear across them. Figure 1 exemplifies how we use network analysis to compute how image-objects can be seen as related to each other if they appear in the same images.
This relational network approach is key to the first part of the case study, where we explore the following question: How are motifs of humans and natural environments depicted together in Instagram and Twitter images? In the second part, we use streamgraphs to map a temporal question: How have visual genres of climate images changed on Twitter from 2015 to 2017?
In the case study, we situate image recognition within a quali-quantitative approach (Venturini & Latour, 2010), which entails a fluid analytical movement between charting large-scale patterns and quantified data visualisations and zooming in on examples of visuals with specific image-objects. Specifically, we display 71 images through the analyses. The selection of images is based on a combined quali-quantitative logic: First, we use network analysis to quantify occurrence and relations of the visual content and identify image-objects that spark analytical curiosity. Then, we qualitatively investigate how the given image-object appears in images, looking at all or up to 100 images in which the Vision AI has identified the image-object of interest. From the sample, we select a handful to display in the analysis, with the same logic that a researcher selects quotes from interviews to represent a common or important viewpoint in the interview.
More specifically, the selection of images is throughout the project based on qualitatively exploring a multiplicity of images that have been annotated with certain image-object of interest to our analysis. The process involves us looking at the networks in Figure 2 and Figure 3 and finding image-objects that seem central to a visual theme. In the Nature cluster on both Twitter and Instagram networks, we for instance see that
This project only uses photos already made public by the users themselves, meaning that we do not publish any information that is not already public.
In the first part of the case study, we investigate how image recognition and a network approach can be used as a way of analysing and comparing visual content across social media platforms. We compare 14,375 images from Instagram and 46,567 from Twitter, exploring how the human–nature relation was depicted in the climate debate after the US announcement of withdrawal from the Paris Agreement in 2017. To do so, we built two network graphs – computed with Gephi (Bastian et al., 2009) – that display as nodes the image-objects detected by the AI, while drawing connections between objects if they co-appear in images. The layout is spatialised with the ForceAtlas2 algorithm (Jacomy et al., 2014), which pulls image-objects that often appear together in images closer to each other in the map, hereby giving analytical meaning to node positions and topology of the networks. Node sizes represent the quantified frequency of image-objects: how often an image-object is featured in the overall sample. Finally, a modularity class algorithm is run on both networks, dividing it into clusters of nodes that are more strongly connected to each other, and hence represent image-objects that more often appear together in images. Figures 2 and 3 show the networks for each platform dataset (high resolution images are provided in the supplementary file).
Looking at the overall structure of the networks, the 1,582 image-objects on Twitter and 1,609 on Instagram are subdivided into seven clusters in each network. When investigating the image-objects in each cluster, we find that the seven clusters identified in the Twitter network are fairly similar to the ones identified in the Instagram network, both containing clusters of visual objects related to people, natural environments, urban environments, fauna, flora, food, and graphic elements. Figure 4 shows an interpretative diagram of the networks, outlining the clusters that we name Human, Nature, Urban, Fauna, Food, Flora, and Graphic.
The networks show that Fauna, Flora, and Food are clusters of visual content important to users on both platforms when discussing climate change. Of interest to our study, however, Figure 4 reveals that the three biggest clusters in both networks are Human, Nature, and Urban, indicating that these visual themes are central to the debate on both platforms. Since this methodological experiment focuses on exploring the human-nature relation, we zoom in on these three clusters: Interestingly, we see that Human is positioned opposite of Nature and Urban, preliminarily suggesting that on both platforms, visual motifs of humans do not often co-appear with motifs of natural environments.
While the thematic similarity of clusters in the two networks initially suggests that the visual debates on Instagram and Twitter are similar in content, a closer investigation reveals how several image-objects only appear on one platform, or appear very often on one but rarely on the other. Looking at the Human cluster, we find that some of the most frequent image-objects in the cluster on Twitter are
Twitter images thus frame human engagement in the climate issue as a primarily political one, while Instagram images frames human engagement more in social terms, with more imagery tied to everyday life. This confirms what others have shown in describing the identity of these platforms (Hu et al., 2014; Mislove et al., 2011). A common feature across platforms, however, is that people are rarely depicted in direct relation to nature or to the climate causes or consequences, thus communicating climate change as a seemingly remote issue.
Twitter and Instagram network clusters
Twitter images annotated with
Unfolding the differences between the platforms further, we zoom in on the Urban and Nature clusters in both networks. The Nature cluster makes up 21 per cent of the Instagram network and 19 per cent of the Twitter network. The largest image-object in the Nature cluster in both networks is
Twitter images (left) and Instagram images (right) annotated with
On Twitter, users mobilise frames of a polluted nature, putting
Twitter images annotated with
It is noteworthy that in the disaster-images from Twitter, humans are very rarely depicted. Instead, it is the built environment which is shown in relation to climate destruction. As such, we find two parallel visual framings of the relation between nature and urban; on one hand, the build environment is causing climate change through pollution, on the other, it is also urban environments – and not humans themselves – that are visually depicted in relation to the consequences of climate change, for example, with floods, fires, and earthquakes. The urban hereby acts as a stand-in symbol of people in depictions of the human-nature relationship.
Zooming in further on image-objects at the border between the Urban and Nature clusters in the two networks, we identify a motif of
Instagram and Twitter images annotated with
The images of
Summarising this analysis, our approach has produced multiple insights on the human–nature relation that demonstrate the potentials of the methodology to open up opportunities for visual cross-platform analysis: First, the macro structure of the Instagram and Twitter networks showed the Human cluster positioned on the opposite side of the map from Nature, indicating that image-objects from these clusters are not often depicted together. Second, a look into Human cluster images revealed that while different frames of human agency proliferate on the two platforms, humans are rarely depicted in direct relation to climate causes and consequences on either of them. Similarly, explorations of Nature cluster images revealed platform-specific differences in an idyllic (Instagram) versus dystopian (Twitter) frame of nature, meanwhile revealing a consistent frame across platforms of an alienated human–nature relationship, with few people displayed in images of nature or images depicting climate causes and consequences. The method has thus made platform differences legible, while revealing a consistent cross-platform visual frame where climate change is depicted as a remote issue. Finally, tourism-images from both Instagram and Twitter confirmed this further by showing humans as visitors in nature, sustaining an alienated imaginary.
These findings add to existing discussions of the human–nature divide that can be found in a wide range of literature, such as Latour (2004, 2012), Ricci and colleagues (2017), Morton (2007), and many others. For the agenda on climate communication specifically, these findings could add to studies of problematising climate visuals, where authors like O’Neill and Nicholson-Cole (2009) have shown that although fearful images attract attention to climate change, fear leaves audiences overwhelmed by the issue, concluding that climate images can
In the second part of the case study, we explore how image recognition can be used to trace specific image-objects over time and provide a temporal perspective on the climate debate. From interpreting the Twitter network in Figure 3, we were particularly curious about the Graphic cluster, which contained image-objects such as
Twitter data samples divided in eight periods, 2015, 2016, and 2017
Divided into eight periods (A–H), this data allows us to investigate the visual debate before, during, and after each political event. We analyse four selected image-objects connected to the Graphic cluster –
Streamgraph of four genre-related image-objects on Twitter, 30-day intervals 2015–2017
What is immediately noticeable in the graph is that
Twitter images annotated with
The images which the algorithm characterises as
In 2015, we also see the prominence of climate imagery labelled with
Twitter images annotated with
Figure 12 exemplifies what the uptake of cartoons and downfall of diagrams means for how the visual genre has changed on Twitter from 2015 to 2017. Where scientific diagrams are earlier used to communicate complex numbers and graphs that are hard to read, cartoons are used to communicate in a more political, emotional, and satirical way that is easy to understand. Also, the subject of these two forms of communication is different, with diagrams typically debating climate-specific science, while the favourite subject of cartoons is political figures such as Donald Trump and Barack Obama.
Similar to cartoons, the image-object
Twitter images annotated with
Interestingly, we see that the
In summary, this analysis has highlighted a shift in how climate change is visually debated on Twitter, where, from 2015 to 2017, the scientific visual genre of technical and fact-oriented diagrams lost its dominant position. The shift towards the use of cartoons suggests a change in the dominant types of engagement in the climate issue: where diagrams visually construct climate change as a scientific issue that can be technically measured and rationally debated, cartoons frame climate change as an emotional and political issue: as something you can believe in, make fun of, or have subjective opinions on. In showing this, our findings contribute to a lot of ongoing discussion in both the public and academic literature, where attention is placed on investigating the rise of right-wing populism, spread of fake news, and nature of political participation on social media (as discussed in, e.g., Effing et al., 2011; Rogers, 2018). Most pressingly, perhaps, our findings raise questions about what the consequences are for public participation and deliberative democracy, when scientific visuals lose territory in the online climate debate: how can people, organisations, or societies come to consensus or mutual understanding on the climate issue if the very concept of scientific facts is destabilised as a communication form? While cartoons have historically been effective in opening up controversies, as seen with the 2006 Muhammed drawings, they are equally inefficient in closing conflicts down (Müller et al., 2009), calling for further investigation of how cartoons shape the climate debate.
In empirically opening up opportunities for such research trajectories, the method applied demonstrates a tangible way of tracing how visual content changes over time, and it provides a tool for making legible how genres of visual communication dynamically lose and gain superiority in online debates. Again, we suggest that the quali-quantitative approach was key to fruitfully bridging the visual and computational capabilities of social science: While Google Vision AI helps quantify occurrence of visual genres, the AI-powered annotation is not self-explanatory but demands contextualisation from attentive researchers, as we saw with the
To conclude, our study makes contributions on both the empirical and methodological level. Empirically, the study delivered a cross-platform analysis of Instagram and Twitter climate images from 2017, showing both platform differences and similarities, especially revealing a consistent dichotomist and alienated depiction of the human–nature relation across both platforms that frames climate causes and consequences as a remote, distant issue. Second, a temporal analysis of Twitter showed how the dominant genres of visuals has changed from 2015 to 2017, where diagrams are substituted with cartoons as the most-used type of visual genre, replacing scientific images with a more politicised, sarcastic, and emotional genre. To advance these results further, a priority could be to extend the cross-platform approach by including other platforms, while it would also be useful to track recent developments of the climate debate from 2018 to the present moment, where, among other events, the US has re-joined to the Paris Agreement, continuously altering the debate.
Methodologically, we first argued that to capture the liveliness of complex visual debates, we need a method that can map and combine both cross-platform and temporal dynamics in visual data. Second, we used the two-fold empirical case study to demonstrate how such a method might be assembled as a digital methods approach, where the combination of image recognition and network analysis within a quali-quantitative lens opened up opportunities to study a quarter-million images from two different platforms and three different years. This, we hope, showcases some of the promises in bridging the computational and visual turn in social science, while many more should also be explored.
On the technological level, this study has only just scratched the surface of what is possible to do with image recognition. Digital and media scholars would do well to experiment more with training and building image recognitions, as well as examining the biases and built-in epistemologies of existing algorithms to show how these visual technologies shape our epistemic processes (Ihde, 2000) – experiments which have been beyond the scope of this article. Other analytical potentials should also be explored, including methods for studying relations between