Who Gets to Wield Academic Mjolnir?: On Worthiness, Knowledge Curation, and Using the Power of the People to Diversify OER

1 In many academic fields Western/white/male/cishetero 2 /abled perspectives are often centered, while other perspectives are presented as “other.” Implicitly, this sends messages to students that success looks like one type of person, knowledge is generated in one kind of way, and their background is not worth being centered. While open educational resources (OER) are often marketed as a tool for social justice, due to their ability to neutralize class-based differences (e.g., Okamoto 2013), there is no evidence that OER are any better than commercial texts at addressing issues of diversity, equity, and inclusion (see Mishra 2017 for a discussion of context). However, OER do present a good opportunity for diversification, due to the relatively simple nature of updating the content. This project takes a crowdsourcing approach to diversify OpenStax Psychology (OpenStax College 2014), an OER for Introductory Psychology courses. Contributors were asked to read areas of the textbook they were comfortable with and make suggestions to diversify the content. The author then used some of the suggestions to create modified chapters and conducted a study investigating the impact of the revisions. Participants read either the original chapter or the diversified chapter and completed a questionnaire assessing their sense of belongingness in the classroom/on campus. Overall, first-generation students had a reduced sense of belonging related to their financial circumstances. However, this effect was ameliorated for first-generation students who read the diversified chapter, compared to those who read the original chapter.


Introduction
In terms of numbers, diversity in college student populations in the USA continues to grow. In 2016 students of color comprised 45.2% of college students, compared to 29.6% in 1996 (Espinosa et al. 2019). There have also been increases in the percentages of low-income and very lowincome students (Chen & Nunnery 2019), students identifying within the LGBTQ* community (American College Health Association 2018, 2000, and disabled students (Newman et al. 2010). At a glance, these numbers suggest that higher education is a place where opportunities are available to all. However, it is also true that educational outcomes are typically lower for students who are marginalized in some way. Black students have the highest college dropout rates and student loan burdens (American Council on Education 2019), disabled students complete post-secondary degrees at lower rates than their abled peers (Sanford et al. 2011), and first-generation students drop out at rates more than twice that of students with college-educated parents (Cataldi, Bennett, & Chen 2018).
One hypothesized cause of these educational attainment gaps is lower levels of belonginess on campus for marginalized students. Having a sense of belonging to the university community has been identified as a key factor in student retention for all types of students (O'Keeffe 2013; Davis et al. 2019) and is predictive of achievement for students of color (Murphy & Zirkel 2015). Unfortunately, it is often more difficult for marginalized students, from various groups, to feel like they belong on their campuses. First-generation students report that they have more difficulty fitting in and making friends compared to their continuing-generation peers (Pratt et al. 2017) and firstgeneration, Black, and Latinx students all report a lower sense of belonging among their peers as compared to white students (Ribera, Miller, & Dumford 2017). Thus, it should be a goal for universities to enhance the sense of belongingness on campus for marginalized students in order to increase their retention and graduation rates (and more generally improve the academic experience of these underserved students).
While this belongingness gap is complex, one seeminglysimple way to help increase students' sense-of-belonging (and thus their educational attainment) is to diversify the educational materials used in the classroom and help students see themselves reflected in their classes. In many academic fields, upper-class white men predominate in the textbooks, despite all fields having rich histories including people of color, white women, and other marginalized groups (Apple & Christian-Smith 2017). Diverse individuals tend not to be depicted as scientists (Ceglie & Olivares 2012), LGBTQ* issues are underrepresented in history textbooks and ' othered' in human sexuality texts (Höhne & Heerdegen 2018;Myerson et al. 2007), and light-skinned individuals are overrepresented in medical textbooks (Louie & Wilkes 2018). Even when marginalized groups are represented, they tend to be ' othered' or presented in a context where they are seen as a problem (Niehaus 2018). Overall, white abled cishetero men tend to be overrepresented in educational materials, regardless of the specific discipline.
This phenomenon also impacts how students view their career opportunities. For example, the "Draw a Scientist" paradigm asks participants to draw what they imagine a scientist looks like. Across nearly all gender, age, socioeconomic, and racial/ethnic groups, the majority of people draw a white man when asked to do this task (Finson 2002). As the saying goes, "You can't be what you can't see," meaning students who do not see themselves represented in the scientific enterprise will not pursue careers in that domain. The stereotypical presentation of white men as scientists also affects how people comprehend the material in textbooks. Good and colleagues (2010) found that girls who read chemistry textbooks with women scientists depicted in the images were better able to comprehend the material than girls who read text excerpts where the images were of male scientists. Together, these findings suggest that adding more diverse representations of professionals to course materials could both increase the number of marginalized students who want to work in those fields and increase their ability to succeed by improving their knowledge and comprehension in those fields.
This "white-washing" (to use a colloquial term, though the erasing of diversity is not limited to race/ethnicity) of educational materials can lead to a lack of student belonginess for marginalized students and a narrowing of their career prospects. However, despite its importance, it can be difficult to modify textbooks. Most traditional textbooks go through a closed revision process, where people (typically experienced professionals in the field) must be invited to the process in order to contribute. Given that most academic fields are not particularly diverse at their upper levels (77.3% of full-time faculty in U.S. higher education institutions are white; Smith, Tovar, & García 2012), the limited number of people who can contribute to the textbook creation and modification process does not reflect the diversity of the students who will be using those materials, where nearly half are students of color (American Council on Education, 2019). Beyond the authors, it is also the case that many repositories of photos which may be included in textbooks are heavily skewed towards white people (for a discussion, see Blicher 2018), resulting in textbooks where photographs are overwhelmingly of white men (e.g., Bush & Mattox 2020). This also applies to other academic materials such as journals, where women are only 15% of corresponding authors and underrepresented in photographs such as those used in advertisements and stock photography (Loverock & Hart 2018).
Beyond these problems, the restrictive copyright licenses on these commercial texts do not allow for individual teachers to make modifications to the materials. While it is possible for instructors to add material when needed, it cannot be presented as a cohesive text or as a part of the core text. That is, it is clear to students when materials are added, which only further reinforces the idea that they are "other" if they are not of the majority. In this regard, open educational resources (OER) can offer a path-to-modification for instructors who are interested in effectively representing the diversity in their fields. OER are "teaching, learning and research materials in any medium -digital or otherwise -that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions" (United Nations Educational, Scientific and Cultural Organization 2012). This means that any instructor who is using these texts can modify them to fit their local context, by adding examples, pictures, questions, and/or content that best reflect their students.
Due to their free availability, and thus power to equalize access to course materials, OER are often marketed as and assumed to be a tool for social justice (e.g., Okamoto 2013). It is indeed notable that the people operating in the open educational space tend to be cognizant of issues of equity and inclusion due to their consideration of student financial problems, and there has been discussion of this in the literature. For example, Willems and Bossu (2012) discuss in-depth how OER may not be equitable as they relate to access to technology (i.e., not all students can readily access a computer to access their OER) and those learning in languages other than English. Bossu et al. (2019) further describe how OER can and should emphasize diversity, saying, "Diversity as a value in OER is an intentional and active embracement of difference" (p. 2). Thus in principle, OER should be more diverse than their commercial peers.
However, there is no concrete evidence that OER are any better than commercial texts at addressing issues of diversity, equity, and inclusion (see Mishra 2017 for a discussion of context). That is, while OER at their core are more democratic in nature than their lock-and-key commercial counterparts, they are not a magic bullet for the diversity problem of educational materials. Bossu and colleagues (2019) suggest that open education as a field may have been so caught up in the inherent "goodness" of OER that practitioners and educators neglected to critically evaluate their true contribution to equity. They say, "Is it possible that in the hype and promise surrounding OER practices and projects the aspects of diversity, inclusion and equity are not carefully interrogated and considered?" (p. 1). It would appear that the power of OER lies not in the inherently independent nature of their creation, as this creation process can and often does continue to reinforce structural inequalities that exist in the wider educational world. However, there is power in the ability for individual people to modify the content in thoughtful ways to achieve diversity and inclusion goals.
Beyond the power of OER to facilitate diverse textbooks as an outcome, there is something inherently powerful about the process of democratizing textbook creation. The current state of commercial textbooks is such that a select few powerful people make decisions on what information is worthy of inclusion in the textbooks from which thousands learn. Put another way, influential textbook writers hold the power to curate knowledge for entire generations of learners. Further, textbooks that are sold by commercial publishers will always be subject to the monetary interests of said publishers, providing an incentive to stick with the status quo. For example, why risk a discussion of queer issues in sexual education if you know that fundamentalist instructors will cease to use your book and cost you money? Using OER to expand textbook writing/editing opportunities serves to isolate knowledge from power and allows educators and students to seize the means of textbook production.
Despite the power of textbooks to enhance educational diversity and the power of OER to democratize knowledge curation, to-date the author is not aware of any published papers looking at how people go about the diversification of OER, nor any evaluations of how this diversification might affect students. The goal of this paper is two-fold. First, it outlines a project the author comanaged with OpenStax, a key purveyor of OER, that had the ultimate goal of crowd-sourcing the diversification of the OpenStax Psychology text (OpenStax College 2014). Then, it describes a study which compared the standard and modified text and their effects on participants' ratings of the books and their sense-of-belongingness on campus. This explored the hypothesis that all participants would have higher quality ratings for the modified text compared to the standard text. A further hypothesis was that participants who were marginalized by either first-generation status or under-represented minority (URM) status would experience a heightened sense of belonging related to their social class and race (respectively) after reading the modified text, compared to participants reading the standard text.

Textbook Diversification -Study Context
In order to facilitate a crowd-sourced approach to diversifying OER, a system was needed that would be simple for the general public to use. A Pressbooks copy of Open-Stax Psychology was created with Hypothesis enabled (as shown in Figure 1). Hypothesis is a web annotation tool that allows for straightforward communication about specific aspects of the text. The integration of Pressbooks with Hypothesis meant that people did not have to download a Hypothesis extension or otherwise go through steps to access the platform, other than creating a log-in. In the instructions, contributors were told that they did not have to use their name and were able to remain anonymous if they wished to do so.
Once they reached the main landing page, contributors could navigate to the section of the book that reflected their expertise or lived experience. They could then highlight sections of the text and propose additions, modifications, or general comments. Two examples of what this looked like are included in Figure 2.
In order to encourage people to participate with the project, several outreach strategies were engaged. First, reaching out to leadership on several discipline-specific teaching and diversity initiatives in the author's own field (e.g., the Society for the Psychology of Sexual Orientation and Gender Diversity, the Society for the Psychological Study of Culture, Ethnicity, and Race). Then, reaching out to people who could be located as in charge of Introductory Psychology on their campuses. Finally, work was undertaken with OpenStax to contact instructors who had selfidentified as using the textbook in their classes. For more general outreach, the author also wrote and published a Medium post (https://medium.com/@anusbaum8/ourstudents-deserve-better-b8d5ea1b8890) that could be shared on social media and in other, private conversations. It should be noted that while this particular project targeted outreach at instructors and graduate students in the field, these kinds of projects can and have been done with student populations as well (e.g., Howard, Nusbaum, & Van Allen 2019). Indeed, involving students in this kind of work may further emphasize the democratization of knowledge that OER introduces. Students are not only told that OER expand opportunities to contribute to the field, they themselves contribute to that knowledge. Instead of being passive consumers of content, open pedagogy projects such as this one give students a chance to be actively engaged in the knowledge curation process -they get to be worthy of participating.
Overall, 59 annotations were collected on 22 different sections of the book. While the annotations were focused on areas of the text that are more "amenable" to suggestions around diversity (such as Social Psychology and Sexual Behavior), there were also suggestions in sections such as Stressors, Mental Health Treatment, and Motivation. This wide variety of suggestions exemplifies why projects of this sort should not be limited based on preconceived notions of where "diverse" topics may arise. In order to explore whether this approach was worthwhile, a study was then conducted using some of the proposed modifications. The research questions focused on whether these diversified materials would lead to increased quality ratings for the text, in addition to a higher sense of belongingness for marginalized students (defined by first generation status and racial/ethnic minority status).

Procedure
All procedures were approved by the Washington State University Institutional Review Board in accordance with the Declaration of Helsinki and were conducted using Qualtrics (Provo, UT). After giving informed consent, participants were given a set of readings on social psychology and sexual behavior. They were randomly assigned to either a Standard condition, where the reading was directly from the OpenStax Psychology book as it currently exists, or a Modified condition, where the readings were supplemented by contributors who edited the book with diversity in mind.
Following the readings, participants completed a set of survey questions assessing the readings (modified version of the Textbook Assessment and Usage Scale; Gurung & Martin 2011) and another set of questions regarding a hypothetical instructor who might use these readings in their class. They were then asked a set of questions regarding their sense of belonginess on campus and a set of demographic questions. Based on these demographic questions, participants were classified as first-generation or continuing generation subject to the educational attainment of their parents and classified as White or an Underrepresented Minority (URM) based on whether they self-identified as white. Many academic institutions and governmental entities classify students as URM based on whether they are underrepresented at the institution compared to local and national demographics (e.g., Page et al. 2013; National Institutes of Health 2020) -at this university non-white students are in the minority (Washington State University 2019). This is also true for non-white people in the state of Washington more broadly (Washington 2018).

Participants
The sample consisted of students currently enrolled in psychology classes at Washington State University. Participants (n = 422) completed the study in exchange for course credit in one of their classes. All participants had the option to complete an alternative assignment if they did not want to participate and all opted-in to participating in this study. Participants (n = 16) who completed the survey in less than three minutes (a time deemed too fast to read through the text and answer all questions appropriately) were removed from the sample, as was one participant who reported not knowing whether they were a first-generation student, leaving 405 participants in the final sample. Of these, 39.4% reported being a first-generation college student. The sample was 69.9% women and 66.6% white, with 12.8% of participants not reporting their race/ethnicity. The rest of the sample was Latinx (14.4%), Asian (9.9%), Black (4.8%), Native American/Native Alaskan/Pacific Islander (2.8%), or self-reported as another race/ethnicity (1.4%). Neither of these two variables differed based on their assignment to groups. The average age of the sample was 20.84 (SD = 4.57). Age varied as a function of group assignment and number of credits, hours worked, and percent of students with loans varied as a function of first-generation status. Based on these differences, these variables were included as covariates in all further analyses.

Readings
The textbook passages that were used originated from the OpenStax Psychology book (OpenStax College 2014), specifically from the Prejudice and Discrimination and Sexual Behavior sections of the text. The Standard condition passage was the exact version of the text that can be accessed at: https://openstax.org/details/books/psychology. The Modified condition passage included edits and additions focused on enhancing the diversity and inclusivity of the book. The textbook passages are available at: https://osf.io/tz5nc/.

Questions about readings
Participants were asked questions about the text passage that stemmed from the Textbook Assessment and Usage Scale (Gurung & Martin 2011). This scale was modified to reflect the fact that participants were only reading the textbook for the purposes of this study, as opposed to using it in a course for an entire term. The questions asked students about the research examples used, the everyday examples used, and the general writing quality on a scale from 1 (not at all) to 7 (very much so). They were additionally asked questions about a hypothetical instructor who would use the text passage in their class. Participants rated the hypothetical instructor on a scale from 1 (strongly disagree) to 5 (strongly agree) for descriptors such as professional, engaging, approachable, and caring, among other things. Both questionnaires are available at: https://osf.io/gkqc6/.

Questions about belongingness
Participants completed two sets of questionnaires regarding whether their 1) financial circumstances and 2) racial group affect how they fit in on campus (Ingram 2012). Prompts were answered on a scale from 1 (much more difficult) to 5 (much easier), with lower scores indicating more difficulties. The individual statements included, "being taken seriously by professors," "finding the academic support you need to do well," and "finding like-minded friends," among others.
A second MANCOVA was used to examine the effects of condition (Standard, Modified), first-generation status (Continuing Generation, First Generation), and race/ethnicity (White, URM) on perceptions of the hypothetical instructor who was using these textbooks. There was no significant multivariate effect of condition

Questions about belongingness
A MANCOVA was used to examine the effects of condition (Standard, Modified), first-generation status (Continuing Generation, First Generation), and race/ethnicity (White, URM) on belongingness related to financial circumstances. There was a significant multivariate effect of race/ethnicity (F [7, 314] = 2.644, p = 0.011, η p 2 = 0.056). The univariate analyses showed that there was a significant difference on one statement (Feeling comfortable socially on campus; F [1, 320] = 4.323, p = 0.038, η p 2 = 0.013) such that those who are URM reported feeling less comfortable socially on campus due to their financial circumstances (M = 3.23, SD = 1.119) as compared to white participants (M = 3.59, SD = 0.978).

Discussion
There are several key findings to note. The first is that participants in the modified condition, regardless of any recorded demographic variables, rated the textbook passages higher than those in the standard condition. Second, both first-generation and URM participants reported a reduced sense of belongingness based on their financial circumstances, and URM participants reported the same reduction based on their racial group. Third, and perhaps most importantly, this reduced sense of belongingness was ameliorated for first-generation participants who read the modified text passages. That is, first-generation stu-  dents were indistinguishable from continuing-generation students on this measure after they read the modified text, as compared to those who read the standard text and reported lower levels of belongingness based on financial circumstances. Together, these findings suggest that textbook modifications of this kind may be a way for institutions and instructors to help increase belongingness for marginalized students. While many universities have small grant programs to fund the development of OER generally, to the author's knowledge there are no grant programs specifically designed to help instructors make their open texts, activities, and other resources more diverse. 3 These findings suggest that such programs would be a good idea, and could in fact lead to a much larger savings return to the university. That is, if diverse textbooks help marginalized students feel like they belong more and thus are more likely to stay at the university (O'Keeffe 2013; Davis et al. 2019), the money that is put towards grant programs could come back to the university in the form of higher student retention rates. For individual instructors, these findings should be thought of as overwhelmingly positive -they suggest that each of us can enact important changes within our own classrooms. At its core, these findings show that the materials a student is reading for one class can have an impact on the extent to which they feel they belong on our campuses. This should leave individual instructors feeling empowered and motivated to make changes to the materials they are using.
There are several limitations to the findings presented here. While tightly controlled experiments are a good option for establishing the legitimacy of an intervention, the lack of external validity in these findings is a key limitation. It is quite possible that these modifications must be made in the context of an actual class for their full impact to be felt. It is one thing for participants in a research study to read material that suggests a hypothetical instructor might care about marginalized groups. It is a different thing altogether for students to be in a classroom setting where an instructor they know is using a textbook that reinforces the idea that they care about marginalized groups. Course materials cannot be separated from the classroom context, and any changes to the materials need to be accompanied by a classroom environment that supports what is contained in the materials. Future studies should try to examine how a modified textbook affects students in an actual classroom setting. For example, measuring students' sense of belonging before-and-after a class where a modified textbook is used would be a more ecologically valid way of approaching this question.
It is also possible that the extent of the benefits to this kind of approach were not fully captured by the measures used in this study. As noted in the introduction, past work has demonstrated that people tend to view scientists as white men (Finson 2002) and tend to comprehend material less when the text does not represent themselves (Good, Woodzicka, & Wingfield 2010). This study did not attempt to assess whether a modified textbook of this sort could elicit changes in how participants imagine scientists or their ability to learn the material. It would be beneficial for future studies to try to assess whether these modifications could lead to these additional benefits. It would also be ideal if future studies could examine more targeted modifications. That is, if we make modifications that specifically target trans* inclusivity or Indigenous issues or disabled representation, would we see benefits specifically for those students? Given that the findings reported here are most promising for first-generation students, who themselves are quite varied in background, one would imagine that this effect is not highly specified. But, that is an empirical question to be tested in future work.
Overall, these results both provide evidence that marginalized students feel like they belong less on campus and demonstrate that there is a student-centered benefit to diversifying our textbooks. Some of our most marginalized students will read these texts and feel like they belong more on our campuses. While certainly not an end-all-beall solution to wider issues of systematic inequality, this crowd-sourcing approach to diversifying course materials is one method that can be used in making progress towards educational equity.

Notes
1 Mjolnir is the magic hammer wielded by Thor in the Marvel Cinematic Universe. In order to hold/use Mjolnir, it must deem you "worthy." 2 Cishetero refers to people who are both cisgendered (their assigned sex at birth matches their current gender) and heterosexual/romantic (sexual and romantic attraction to people of the "opposite" gender). 3 Note that this does not mean people are not thinking about these ideas -see https://www.cccoer. org/2018/10/09/on-equity-diversity-inclusion-andopen-education/ for an excellent discussion and https://open.ed.ac.uk/openness-equality-and-inclusion/ for examples of resources.