A Novel Approach to Teaching a General Education Course on Astrobiology

Typical general education science courses for non-STEM majors suffer from lack of motivation, interest, or engagement of many students, in addition to having a student population of underprepared students, specifically students with little background in either science or mathematics. In addition, many students in such courses exhibit high levels of science and mathematics anxiety (Udo et al., 2004). Students in such courses come from a diverse range of academic majors and typically do not need to learn specific subject matter. Instead, a typical goal for such courses is that students would become science literate.

Roberts (2007) listed two visions for what science literacy can achieve. Vision I includes students’ mastery of “the canon of orthodox natural science, that is, the products and processes of science itself.” Vision II includes students’ familiarity with “situations with a scientific component, situations that students are likely to encounter as citizens.” We may subdivide Vision II according to Shen (1975) into three parts: (1) practical science literacy, which helps ordinary people make real-life decisions; (2) civic science literacy, which enables citizens “to arrive at considered decisions” about science-related policy issues; and (3) cultural science literacy, the appreciation of science and of what scientists do. Feinstein (2011) emphasizes science literacy as the public engagement with science:

“[S]cience literate people are competent outsiders with respect to science: people who have learned to recognize the moments when science has some bearing on their needs and interests and to interact with sources of scientific expertise in ways that help them achieve their own goals. It follows from this definition that the pursuit of science literacy is not incidentally but fundamentally about identifying relevance: learning to see how science is or could be significant to the things you care about most.”

The education of an outsider does not need to be identical to that of a future insider (Feinstein et al., 2013) in subject matter, goals, or methods.

Motivated particularly by Roberts’ Vision II and Feinstein's interpretation of science literacy, we developed a course for the general education requirement of non-STEM majors. Many of the students have never had a college-level science course before, and many will never have another science course after such a course. For many of the students such a course is the last opportunity to learn about science. One may therefore view such a course as a one-time opportunity to promote the development of citizens engaged with science.

We have chosen to develop a unique course on astrobiology for three reasons: (1) astrobiology is an interdisciplinary and a multidisciplinary topic, such that students may be exposed to major principles of a number of scientific disciplines; (2) astrobiology addresses some of the “big questions” that an educated person may be curious about, such as questions about the nature of life, the origin and fate of humanity and of Earth, and the place of humanity in the Universe; (3) astrobiology and related scientific areas are often discussed in the media, including both the news media and popular media (such as sci-fi movies or books).

We have chosen to support student motivation, interest, and engagement by selecting a novel as the primary text of the course instead of a traditional textbook (Burko, 2016a). The latter tends to be informative and comprehensive but at the same time lacks a compelling storyline that connects scientific principles and facts. A novel may promote student motivation because a compelling storyline—intimately related to science—could lead to greater student interest in the science. To paraphrase Talmud Bavli (Sanhedrin 105b), when one engages in science not for its own sake, eventually one may come to be engaged in science for its own sake. That is, students may be captivated by the story of the novel and then be motivated to learn the associated science underlying the story; ideally, they would become more scientifically literate so that they could apply the science they have learned to their everyday lives and interact with science. Indeed, one student wrote, “it makes learning the content more interesting which in the end, makes it easier to remember.”

Not just any novel would be a good choice. A good choice would be a novel that (1) includes sufficient science content; (2) contains correct science content; (3) allows for appropriate lab experiments; (4) allows for a complete coverage of the subject matter as it is taught in typical courses on similar topics; and (5) revolves around a topic that can generate students’ interest.

We have chosen Carl Sagan's Contact (Sagan, 1984). Given that Contact was first published nearly four decades ago, we feel compelled to justify this choice. After all, science in general and astrobiology in particular have developed significantly since the peak of the Cold War. We have found no particular problem with using Contact in terms of its science content. In fact, in a number of cases, Sagan was able to predict then future developments, such as the discovery of an exoplanet in the habitable zone in the Alpha Centauri system (Anglada-Escudé et al., 2016), or a planet-sized radio telescope to image the black hole at the galactic center (Akiyama et al., 2019). Other advantages of Contact are that in addition to being a book that tells a great story and the inclusion of much science (not just astrobiology—see below), Contact has 24 chapters of roughly equal length, which make it ideal for a semester-length course when roughly two chapters are studied each week (additional time is needed for experiments and class projects). Moreover, it is rich in cultural, intellectual, historical, and societal issues, such as the role of women and members of underrepresented groups in science, nuclear disarmament and arms reduction, and the international nature of science, which may optionally be used for enrichment if the instructor so wishes. Arguably, at least some of these topics should anyway be included in courses where science literacy is an important goal.

Teaching a science course with a novel is not free of challenges, since novels are not written for the purpose of serving as textbooks: (1) the topic ordering is not conventional; (2) topics recur in different chapters; (3) the science content is presented at an uneven pace; (4) some topics are not typical for an astrobiology course; (5) some topics require some ingenuity in order to relate them to the text; (6) some science content is merely mentioned in passing or with insufficient detail, and; (7) consequently, supplemental materials are required.

While the story line of Contact certainly revolves around the search for extraterrestrial intelligence (SETI), the science presented in it is not exclusively or even predominantly SETI. A course based on Contact does not have to be a course on SETI but can be an astrobiology course. In fact, we did not identify a single topic that is traditionally included in similar astrobiology courses that we could not connect to Contact. As an example, for traditional textbooks on astrobiology, we compared Contact with Life in the Universe (Bennet and Shostak, 2016).

We have tried to turn each of the aforementioned challenges into an advantage. For example, while plate tectonics is not discussed explicitly in Contact, it can be connected to Palmer Joss's body art. Similarly, the Viking experiments to discover evidence for life on Mars are not mentioned explicitly, but Devi's field of expertise is mentioned in passing, and from the timeline it can be related to the Viking experiments and then to the more recent discovery of perchlorate (ClO₄⁻) on Mars (Hecht, 2009; Quinn, 2013) and its relation to the results of the Viking experiments (Navarro-Gonzalez, 2010).

We have created supplementary materials, such as short stories that illustrate certain science topics. For example, Contact discusses radioactivity, and to provide the students with more information than discussed in Contact we wrote a short story in which the essentials of radioactivity and radiation sickness are discussed through the story (Burko, 2016b). We supplement the discussion of radioactivity with a standard experiment in which the students use a Geiger-Müller counter to find the properties of the three types of radioactivity. Other supplemental teaching materials that we wrote for this course include, inter alia, a sci-fi story about a future lunar mission and space weather (Burko, 2021a), and dialogues about the definition of life (Burko, 2021b), as well as about the origin of life. We also use other supplemental materials, such as a short story about the Big Rip and the fate of the Universe (Baxter, 2007) and a novella about communication with visiting aliens and the nature of determinism (Chiang, 2016). Subject ordering is certainly a challenge. As an illustration, we list here the topics discussed in the first four chapters of Contact. These include: –

Chapter 1: Interaction of civilizations at much different technological levels; the number π (irrational and transcendental). The question whether π is normal becomes pivotal to the ending of Contact, the epistemology of science, the apparent rotation of the celestial sphere and the Earth's spin, and ancient Greek astronomy.

The remaining chapters include similarly eclectic collections of scientific, societal, intellectual, historical, and cultural topics. Overall, all topics that are traditionally included in an introductory course on astrobiology are discussed in Contact, either directly or indirectly, so that the instructor has the opportunity to discuss the entire traditional subject matter without having to make any sacrifices. A full list of science topics in all 24 Contact chapters appears in Table 3.

Certain topics are repeated in more than one chapter. We view this as an advantage, as it allows us to revisit important topics, remind students of ideas they have seen before, and expand the discussion in different directions. For example, the question of the longevity of young technological civilizations is alluded to in several places throughout Contact. We supplement the typical discussion, connected with the Drake equation, with discussion of the two incidents that have brought humanity to the brink of a global thermonuclear war, specifically, the 1962 Cuban missile crisis and the 1983 Soviet false nuclear alarm incident. The Cold War background of Contact is an opportunity to have this discussion in the proper context. Students are typically amazed at how close we got to the brink of nuclear war and how fortuitous humanity and the Earth were in these incidents.

This discussion allows us to review the time scale from the onset of modern science (say, in 1543, the publication of Copernicus's De Revolutionibus Orbium Coelestium) to Maxwell's theory (say, in 1865, the publication of Maxwell's A Dynamical Theory of The Electromagnetic Field), to Hertz's wireless experiment (in 1887), to Jansky's invention of the radio telescope (in 1932), to the first atomic weapon at the end of WWII (in 1945), and to the first (in 1962) and then the second time (in 1983) we were on the brink of nuclear destruction. This timeline emphasizes the developments in electromagnetism, which connect the invention of the radio telescope (the way astrobiologists often define advanced civilizations) to weapons of mass destruction. One could also include the nuclear timeline, including the discovery of radioactivity by Becquerel (in 1896), the first splitting of the atom by Rutherford (in 1917), the first nuclear fission of uranium by Meitner, Hahn, and Frisch (in 1939—an opportunity to revisit women's participation in science and the recognition thereof!), and the first sustained chained reaction by Fermi (in 1942) leading to Hiroshima and Nagasaki (in 1945).

A related idea that recurs is that the chances that an alien civilization would be comparable in development to ours are very small and that any alien civilization we might be in contact with would in all likelihood be much more advanced than we are. Contact is not free of incorrect information; for example, Ellie says that “[n]o even number is prime,” which is of course not true, because 2 is both even and prime. A correct statement would be that no even number other than 2 is prime (Contact does get it right elsewhere though, shortly after this mistake is made). A more serious misrepresentation, in our opinion, is the representation of the Aryan Invasion Theory as incontrovertible, while it is now widely appreciated (and was also when Contact was written) that there is a fierce debate regarding its validity, related to the scarcity of evidence in its support from physical, cultural, or biological anthropology (Bryant 2001). We use this misrepresentation as an opportunity to revisit the scientific method and science as a self-correcting process, a topic we discuss several weeks earlier in the context of the development of Greek astronomy and periodically during the semester when hypotheses made by characters in the novel related to the Contact storyline need to be revisited in light of new evidence.

In other cases, Contact is anachronistic: some of the plot events happen after the beginning of the millennium, long past the Soviet Union when the Soviet Union collapsed, and yet the novel presents the Soviet Union as still being in existence. Also, Contact presents weekly news magazines as a go-to source for news commentary, whereas the internet was already in wide use at the time of the storyline. It is not hard to excuse Sagan's inability to prognosticate, as few could predict such developments when Contact was written. Indeed, Sagan attacks prognostication in Contact directly. These anachronisms are an opportunity to give examples for the difficulty in prognostication and why pseudoscience, e.g., astrology or numerology, is meritless. However, Contact is mostly accurate about developments in science even long after it was written and is easy to use in an up-to-date class on astrobiology. When Contact was written, we had no evidence for exoplanets, gravitational waves, or imaging galactic centers. These and other developments are discussed or mentioned in Contact and are all established science now.

Contact presents multiple opportunities to discuss cultural and societal topics that are not directly related to astrobiology but can still serve the instruction goals for beginning college students in general education courses and can promote the goal of science literacy. Indeed, Schneider and Schoenberg (1998) argue that the goals of a general education course include “acquiring intellectual skills or capacities, … understanding multiple modes of inquiry and approaches to knowledge, … [and] developing societal, civic, and global knowledge.” Examples include literary works, specifically epics (the text mentions or alludes to Gilgamesh, the Iliad, and the Odyssey, and it can also be connected to The Divine Comedy), mythology (e.g., the Tanabata festival of Japan, the singing of the Sirens), history, political science, geography, music, and sociology of science in addition to its rich science content. The instructor may, of course, decide how much time if any is desired to devote to such topics.

An example for a music topic is Sagan's mention of “full and melodic mirror fugues: The counterpoint would be the theme written backwards.” Although we have mirror fugues (e.g., Contrapunktus XII in Bach's Art of the Fugue) and counterpoints where the theme is played backwards (e.g., Bach's Crab canon), we do not have a piece where both exist at the same time. Students were fascinated listening to the Bach's Canon 3 from a collection of 14 canons (BWV 1087) and seeing how it has the topology of the Möbius strip (Altschuler and Phillips, 2015).

Students were also impressed that they can understand notoriously hard mathematics problems, such as Fermat's Last Theorem, the Goldbach Conjecture, or the question of the normality of π (the last two are as yet unsolved open mathematical problems), even though their own mathematical skills (and also their perception of having mathematical skills) are not very advanced.

The typical structure of the course is that students are assigned assignments before the lecture, which include a reading assignment from Contact and supplementary materials if any as needed and are also given the discussion questions for the next class. The students are tasked with answering these questions, many of which require online or library searches. In class we discuss the questions and do science demos; students do experiments (see below) and other group activities, such as activities from Prather et al. (2007). Classroom discussions are typically in the style of the Socratic dialogue (Overholster, 1992). After-class activities include writing up lab reports, finishing other class activities, and writing up individualized term papers. Students were to submit two topics to write their term paper about, the second intended to be an alternate topic in case the first one is not approved. The papers were to be written in the style of a journal article, and students were given assistance in how to write in such a style.

We assigned grades to students based on a participation assessment, work turned in (such as lab reports and term papers), and a final exam. For classroom participation grades full credit was awarded to any student who demonstrated having read the chapter assignment from Contact and a reasonable attempt to do the assignment before the lesson. Our impression was that the large majority of students read the assigned chapter regularly.

The science supported by a course based on Contact goes beyond the classic topics of the typical introductory course of astrobiology. Some instructors may want to choose the experiments done by the students to be restricted to the typical astrobiology course experiments, while others may wish to have the students perform science experiments belonging to a wider spectrum of experiments. We chose the latter approach. Laboratory experiments that are directly related to the science content of Contact and that we used include the following: –

Experimental measurement of π (measuring diameter and circumference of spherical objects and calculating the ratio; emphasis is placed on measuring multiple times and finding the mean and standard error).

Other experiments that are more traditionally included in introductory astrobiology courses can be found in Robinson et al. (n.d.) and in Prather et al (2007).

We also put emphasis on critical thinking skills. There have been previous attempts to use astrobiology courses to develop critical thinking skills (Foster and Lemus, 2015). We focused specifically on two such skills: first, the separation of factual information from inferences that might be used to interpret those facts and second, identifying inappropriate conclusions (Stein et al., 2007). We wrote two activities specifically to practice these critical thinking skills, specifically an activity on the strong signal detected in 2015 from the direction of HD164595, which some have suggested may be evidence of intelligent extraterrestrial life (Bursov et al., 2016). We also focused on the mystery of magnetic moon rocks, which we were fortuitously discussing at the same time that the (failed) Israeli lunar lander Beresheet was attempting to collect data to answer this question. Over the semester we emphasized critical thinking skills at many other opportunities as part of the regular teaching materials without writing specific teaching materials for that purpose other than the two discussed above. We also emphasized science vs. nonscience, including religion and pseudo-science. Contact discusses in depth many aspects of the relationship of science and religion, which we were able to revisit several times during the course. We also emphasized the distinction of science (e.g., Darwinian evolution) from pseudo-science (e.g., creationism, or the so-called Intelligent Design) by emphasizing science as a method and process over its subject matter.

We taught the four-credit hour Physical Science with Laboratory course and collected data over five semesters starting in the summer semester, 2017, and ending in spring semester, 2019, at Georgia Gwinnett College, a four-year liberal art college, which is an access school, that is, student admission is nonselective. Class size was capped at 24 students per section, and the number of students in a section varied in practice between 12 and 24. Of the student body the largest ethnic group is Black/African American (32.1%), followed by White students (30.5%), Hispanic students (21.2%), and Asian students (11.0%). Also, 3.8% of the students identified as multi-ethnic, 0.2% as Pacific Islanders, and 0.1% as Native Americans. The student body is 57% female and 43% male. Approximately 40% of each entering cohort is made up of first-generation students.

This study received Institutional Review Board approval number 17060. We allowed students to opt in after explaining to them the nature of the study and the risks involved. Students received extra credit for completing the study questionnaires whether they consented to participate in the study or not. In the latter case, we excluded their questionnaires from the analysis. All the 23 students participating in this study were freshmen belonging to two different sections during the same semester and exposed to the same teaching materials and activities. Of these students, nine were business majors, four were cinema/media majors, two were human development and aging services majors, two were psychology majors, one was an elementary education major, one was a political science major, and four had undeclared majors. The students had had no previous science courses at the collegiate level and had varying high school exposure to science and mathematics.

Data collected from previous offerings of the course were used formatively to improve results that had been obtained but were not included in the data presented here. For example, after seeing that students’ gains in understanding the concept of the Snowball Earth model were minimal when the main instruction of the concept was done with a reading assignment and was mentioned only in passing in class, we decided to discuss Snowball Earth in class directly. Students’ gains increased dramatically from 0.07 to 0.45 (see Table 1). We inferred from this example that many students probably did not do the additional reading assignments (in addition to Contact) very well. A closely related point will be discussed below when we analyze our results. Because the course was changed between semesters (based on data collected and analyzed) we did not conflate data from different semesters when the course was offered; rather, in this study we included results only from the two sections of the last semester of the study. This way we did not dilute the results obtained from different teaching materials and activities.

We assessed the objective student learning of our course following the method of Foster and Drew (2009). Specifically, we used the 15 key knowledge areas (Foster and Drew, 2009) and administered pre- and post-tests, which were in the form of multiple-choice questions. We then calculated the normalized gains (Hake, 1998), g=x¯post−x¯pre1−x¯pre g = {{{{\bar x}_{{\rm{post}}}} - {{\bar x}_{{\rm{pre}}}}} \over {1 - {{\bar x}_{{\rm{pre}}}}}} and the effect size f=|x¯post−x¯pre|σpost2+σpre2 f = {{\left| {{{\bar x}_{{\rm{post}}}} - {{\bar x}_{{\rm{pre}}}}} \right|} \over {\sqrt {\sigma _{{\rm{post}}}^2 + \sigma _{{\rm{pre}}}^2} }} where σ is the standard error and an overbar represents the mean. Our results are shown in Table 1.

We have also performed an attitudinal study (Foster and Drew, 2009), which includes three groups of questions: student learning self-assessment (Group 1: questions 1–4 in the attitudinal study; see Table 2), student self-assessment of scientific writing and reading of primary literature skills (Group 2, questions 5–7), and student self-assessment of long-term interest in a scientific research career (Group 3, questions 8–9). The data are included in Table 2. Table 2 shows that while there were higher levels for self-assessment after instruction than for self-assessment before instruction for all questions except for Question 9 (for which no change was measured), the effects were very different among the three groups of questions. The greatest change was in Group 1 (before 3.9 ± 0.3, after 2.4 ± 0.2, g = 0.51, f = 3.7, p-value < 1×10⁻³), followed by Group 2 (before 2.5 ± 0.2, after 2.2 ± 0.2, g = 0.24, f = 1.2, p-value < 1×10⁻³), and finally, Group 3 (before 3.7 ± 0.2, after 3.45 ± 0.04, g = 0.08, f = 0.8, p-value < 1×10⁻³). The p-values were calculated by matched-pair t-tests for the pre- and post-test results, where the null hypothesis was that there was no difference between the pre- and post-test results and the alternative hypothesis was one-tailed. The modest increase in Group 3 questions is likely related to the students’ majors, their career goals, interests, and also the perception of their aptitude to become scientists. Nevertheless, as questions from Group 1 and Group 2 show, the students’ self-assessment in either the science content knowledge or the understanding of science communication has increased significantly.

We see substantial differences between our results and those of Foster and Drew (2009). Direct comparison is hard to make because of differences not just in the teaching method, in the teaching materials, and the instructors, but also in student populations. In Foster and Drew (2009) the students were graduates of two introductory science courses before taking an intermediate astrobiology course, a very different population from the one we had, and in addition, the student body at the University of Florida in 2007 was very different from the student body at Georgia Gwinnett College in 2019. Indeed, we can appreciate the differences by comparing the pre-test scores: the overall fraction of correct answers in our pre-test was 0.34 ± 0.05 and in Foster and Drew (2009) it was 0.42 ± 0.06. These results are 1.02 standard deviations apart. This difference between the pre-test scores of the two populations could also explain, at least in part, the differences in the normalized gains. Indeed, it was argued that normalized gains are correlated with the pre-test scores (Coletta and Phillips, 2005). See, however, Von Korff et al (2016). We were not able to find relevant student gain results in the literature for a student population similar to the one we had. Direct comparisons are therefore difficult to make.

We were interested in knowing what students thought about our teaching method. Comments made by students were mostly favorable. One student wrote,

“Professor Burko obtains a teaching method that makes it easier for students to grasp the materials. Rather than a two-hour lecture with power points, he provided us with a novel to read throughout the semester that ties into the curriculum we are required to learn. We read the chapter before class time, and answer the questions from the chapter. It's not just questions on the book, its content questions as well that are mentioned in the novel. As we have these questions answered and come to class prepared, we discuss each question and the correct answer to it. What helps out most about it is how we are already aware of the content after answering the pre-lecture questions, so discussing them in more depth with the professor enhances the knowledge we already obtained before coming to class. Another great thing about this teaching method is how it makes learning the content more interesting, which in the end, makes it easier to remember. I hope to see more professors use this method in the future and bring students great success in the class as it did for me and many others.”

Another student wrote,

“This is my first semester at [Georgia Gwinnett College], and I am going to school to become a chiropractor. Dr. Burko's approach to teaching science is different than any other science class I have ever been in, most science teachers throw random facts at you and expect you to pick them up and memorize them. But not Dr B, he uses the book “Contact” by Carl Sagan to introduce college level science to freshman by giving us real world context to what he is teaching in a way that is as captivating as science can be for those who are not so interested in the subject... I have learned a lot from this course, and it has made me less instant [sic] in the subject of physical science.”

A third student wrote,

“It was amazing taking [Physical Science] - Astrobiology with Dr. Burko this semester at Georgia Gwinnett College. He engaged us and made sure we understood all that was needed to succeed. His lecturing style was unique, introducing epic novel into the course made it more interesting and understandable. The handouts he provided on D2L [learning management system] were very engaging and easy to comprehend. Interestingly, he authored some of them,”

And a fourth student wrote,

“The class was required to read a chapter of Carl Sagan's Contact each class period, and then, the class had an interactive lecture. This was a simple way to structure concepts, and it made them easier for the audience to remember. The pre class assignments were research based, as well as thought provoking. The pre lecture let me make my own ideas, and opinions of the material, before Burko expanded upon them.”

On the other hand, not all students liked everything about this class, as can be seen from comments written by a fifth student,

“I am usually not a huge fan of science, and honestly figured I wouldn’t like the class at first, but I was happily surprised by it… I was able, as someone who doesn’t consider their strong suit to be in science, to understand it and take away lessons from the class… [T]he book that was required for us to read for the class, Contact by Carl Sagan, it was good in the sense of helping us answer the questions related to the field of study. Reading the book helped me to understand the class but, if I can speak for some of the other students in the class with me, it was more of a hindrance on them. Whether or not the book should be added in with the next Physical Science course is debatable, but I can see either side of the argument.”

In an attempt to address the problem of student motivation, interest, and engagement in general education science courses for non-STEM majors, we created a novel course based on teaching science with a novel as the main text instead of a standard textbook. Our objective was to create a context-rich course that discusses without omissions all the topics that traditional introductory courses on astrobiology do. Our teaching method was based on students’ doing an assignment before the lecture, and then class time was devoted to in-depth discussion related with the assignment.

The subject matter for the course followed all the major topics discussed in a traditional textbook such as Life in the Universe (Bennett and Shostak, 2016), but we did not make direct use of any textbook. Instead, the students read the novel Contact (Sagan, 1985), as well as additional handouts to support, enhance, and update the scientific content of Contact. Our context-rich course also includes tangential topics that support the general education purpose of the course, although they are not directly related to astrobiology; also, it allows students to see how science can relate to a wide spectrum of topics and experiences. Of particular interest for us was the inclusion of societal issues such as women's participation in science and nuclear disarmament.

We assessed our course by using surveys before and after the course, in which, following Foster and Drew (2009), we measured both objective and self-assessed student learning in 15 knowledge areas and areas of students’ perceptions. The surveys were identical multiple-choice question surveys, each question having four possible answers, only one of which is correct. We endeavored to write distracters that would likely be chosen by students with only cursory knowledge. Random choice therefore accounts for 0.25 in Table 1. Fractions lower than 0.25 do not indicate any actual knowledge and may be attributed, at least in part, to random guessing. Indeed, Miller et al. (2010) found that, among low performing students, losses are fairly common and ask, “Do these ‘losses’ represent actual conceptual losses, or do they result from correct guesses on the pre-test that, by chance, became incorrect on the post-test?” a question to which no conclusive answer is offered. To the hypothesis offered by Miller et al. (2010), we add the conjecture that the explanation is not entirely provided by random chance, but also by random guessing on the pre-test and a bona fide, but failed, attempt actually to answer the question on the post-test. As pointed out by Miller et al. (2010), these possibilities require further study.

We saw gains in both student knowledge and in student self-assessment of content and perceptions of science. However, we have not seen gains in student desire to become scientists or participate in science research in this population of non-STEM majors. In fact, the attitudinal questions with two of the three lowest (or no) gains where the questions about wanting to pursue a career in science research and participating in astrobiology research. The student population was made up of nonscience majors. It is not surprising that few of them would be interested in careers in science. The other question with very low gain was about the comfort level in reading primary research literature. Since this was an introductory course for nonmajors, the highly technical nature of the primary literature makes this discomfort unsurprising. (We did make use of primary literature in the course; however, we limited its use to the introductory and discussion sections but not to the highly technical parts.)

Examination of Table 1 shows that several key knowledge areas showed significant gains, whereas other areas did not. Table 1 shows the normalized gains, the effect sizes, and the p-values for a matched-pair t-test for the difference between the results before and after the course. In most key areas the p-values are small enough to allow us to reject the null hypothesis according to which there were no gains. In only four key areas are we unable to reject the null hypothesis in favor of a one-tailed alternative hypothesis according to which there were gains. While we do not expect gains to be uniform across the 15 key knowledge areas, this great variability needs to be explained. It is remarkable that areas with small gains are those for which pre-test results are high. One would be tempted to suggest that student knowledge was already high in those areas. But we do not believe so. The reason is that classroom discussions suggest to us that student knowledge before the course was actually rather limited, despite the high pre-test results. Perhaps the example that specifically demonstrates this point most clearly is the fact that not even a single student, during class discussions, knew what the parallax effect was. Notwithstanding this, 43% ± 11% of the students answered the question correctly on key knowledge area 3. This result cannot be explained solely by the random nature of chance. A similar situation occurred with key knowledge area 11. Not even a single student, during class discussion, knew what a chemoautotroph was. And still, the exact same percentage of students as for key knowledge area 3 also answered correctly here. We witnessed a similar situation also on the other cases for which gains were low when prior knowledge was apparently high. One suggestion for future study is that the pre- and post-tests could include a larger number of questions on each key knowledge area, so that the chances for statistical flukes are much smaller.

The data shown in Table 1 were analyzed in two ways. First, we posed the null hypothesis that there is no difference between the pre- and post-test results for the full list of 15 areas, H₀: post-test - pre-test = 0, and the alternative hypothesis that the post-test results were higher than the pre-test counterparts, H_A: post-test - pre-test > 0, that is, we perform a one-tailed paired t-test for the difference in course results. We choose the significance level α = 0.05. With 14 degrees of freedom, out test statistics is t = 3.6622, which leads to a p-value of 1.281×10⁻³. The Cohen D effect size is 0.95. The p-value for normality is 0.5068, and the post-hoc power is 0.9666. Alternatively, we consider the normalized gains in Table 1, and pose the null hypothesis H₀: normalized gains = 0 and the alternative hypothesis H_A: normalized gains > 0, that is we perform a one-sample t-test for the normalized gains with significance level α = 0.05. With 14 degrees of freedom, our test statistics is t = 4.3086, which leads to a p-value of 3.607×10⁻⁴. The Cohen D effect size is 1.11. The normality p-value is 0.8978 and the post hoc power is 0.9927. In both analyses the p-values are much smaller than the α levels such that we are 95% confident that we may reject the null hypotheses. The Cohen D effect size for both analyses are large.

Our sample is reasonably random, as choosing the class where the study was done was a random cluster sample of the various physical science sections offered at Georgia Gwinnett College in the relevant semester. Having large p-values for normality, we cannot reject the hypothesis that the data were approximately normally distributed. In addition, the sample data do not show strong skews or outliers. Therefore, the conditions for the significance tests are reasonably satisfied and we may adopt their conclusions.

We find in both our study and in Foster and Drew (2009) that the pre-test results in key knowledge area 7 on Snowball Earth were exactly 0, that is, not even a single student answered the question in this key knowledge area correctly on the pretest. While we do believe that no student indeed knew the Snowball Earth model during the pre-test, it is statistically surprising that not even a single student randomly guessed the right answer correctly. Indeed, the probability that the result in our study could be explained by the random nature of chance alone is 0.002.

The cohort of students showed even greater lack of pretest knowledge compared with Foster and Drew (2009). Not even a single student indicated strong agreement with the statement on student familiarity with three subfields of astrobiology (such as astronomy, biology, geology, chemistry, and planetary science), and only one student declared he or she somewhat agreed. Knowledge areas that showed the least normalized gains where those that we devoted less class discussion to and ones for which we relied mostly on student self-learning through the assignments before the lecture and other individual or group activities. We suggest that when this class is taught, more class time should be devoted to discussing these topics (e.g., the Cambrian explosion, age of the universe, types of rocks, the carbon dioxide cycle), and we would be interested in measuring whether student learning gains increase. Indeed, as discussed above, gains for knowing the Snowball Earth model improved dramatically when students transitioned from self-learning to direct classroom instruction. We therefore expect a similar effect also for the topics discussed here.

Our results show that the approach of basing a STEM course on an appropriate, carefully chosen novel can result in substantial learning gains for students and at the same time improve student self-assessment and perceptions of science while not compromising absolute learning gains. Specifically, it is possible to teach a successful content-rich astrobiology course based on Contact.