Learning the Lessons of Openness

The Open Educational Resources (OER) movement has built up a record of experience and achievements since it was formed 10 years ago as an identifiable approach to sharing online learning materials. In its initial phase, much activity was driven by ideals and interest in finding new ways to release content, with less direct research and reflection on the process. It is now important to consider the impact of OER and the types of evidence that are being generated across initiatives, organisations and individuals. Drawing on the work of OLnet (http://olnet.org) in bringing people together through fellowships, research projects and supporting collective intelligence about OER, we discuss the key challenges facing the OER movement. We go on to consider these challenges in the context of another project, Bridge to Success (http://b2s.aacc.edu), identifying the services which can support open education in the future.

The challenges are described below, together with examples drawn from OLnet and from the actions that are taking place more broadly. Open approaches are advancing rapidly, so we don't see these challenges as static (nor of purely academic interest). We examine a practical example drawn from the Bridge to Success project, which needed to provide open resources into a new context in a short period of time. Reflecting on the experience of applying open approaches leads though to a description of the type of services that may need to be put in place to meet the key challenges in a diverse variety of contexts.
The Key Challenges of OER One of the significant achievements of the OLnet project in its final year has been the identification and ratification of a new set of key challenges for the OER movement through the OER Evidence Hub (OER Evidence Hub, 2012;De Liddo et al., 2012). Earlier work (including analysis of reports from previous recipients of funding from the Hewlett Foundation under their OER programme) had identified key areas and themes which were drawn upon in the creation of the collective intelligence data model. In turn, the seeded content on the Hub -literature studies, news articles, journal papers, presentations, anecdotal evidence -was analyzed and classified with relevance to ten key questions. This list was circulated back to the OER community for comment and refinement (De Liddo, 2011) leading to an extended list of twelve key challenges. In addition to validating the OER Hub as a tool for collecting and making sense of research data, this list of key challenges provides a useful way of framing achievements and future challenges.

Creating new appropriate assessment/evaluation models and practices for OER
One of the challenges facing the OER movement is finding an effective way of integrating learning analytics into assessment (Lovett et al., 2008). Tackling this challenge could enable may of the beneficial aspects of Open Learning by offering alternative ways to demonstrate learning that are independent of particular sources or methods. Wiley (2011) suggests that a bank of assessment (sometimes termed Open Assessment Resources -OAR) could be established that is at such scale and range that individual challenges can be proposed to suit almost any situation. Steps toward this may be to build different pedagogical patterns of assessment tasks that encourage individuals to take control of their own learning. An automated approach to marking assessments risks over simplification and may miss the actual challenge of learning, or alternatively require an abundance of worked answers so the difficult but essential learning process of facing the challenge is missed out. A long history of tutor-based assessment has shown that good feedback is more important to the student than the raw mark, and that such feedback is typically time consuming to produce and hard to support.
In assessing and evaluating learning the issue is not so much the production of "open" content, but how to connect the wide range of existing content through to learning activities. Learning is a complex process, and the acquisition of new knowledge and skills is often challenging. The gratification of the learner may be delayed until a solid grounding for building further knowledge is available. Assessment driven learning may be an artefact of existing requirements for proof of learning rather than the learning that meets authentic needs, particularly in informal contexts. However, the addition of external assessment can act as a catalyst to turn intentions into motivations and structure them into learning activity and in the concept of badges (Mozilla, 2012) a hybrid of activity and reward is starting to appear.

2.
Technologies & infrastructure needed/in place to help the OER movement Many technology-driven solutions now present themselves to the aspiring OER educator, including tools for improving discoverability through search engine optimization and metadata; for publishing content and assessing learning. Broadly, we distinguish specific solutions that are designed to support OER from wide-access systems designed for other purposes that have been appropriated for use in education and learning. In the first category of specific solutions to the OER challenge examples include OpenLearn (2012), Connexions (2012), OERGlue (2012), P2PU (2012) and OpenStudy (2012). OpenLearn's LabSpace provides its facilities to all users and allows downloading of content, uploading content, setting up learning clubs, building paths, journals, forums, video-conferencing and more as a fairly comprehensive approach to the needs identified by OpenLearn during 2006-2008. Since then it has proven its value in supporting other projects that need such a space, but has too much complexity for individual users. OpenStudy focuses on solving one problem: where can learners talk to other learners about topics raised by OER. OpenStudy offers other sites the opportunity to embed or link in to a unified place for discussion and for learners it gives the critical mass of people talking about the subject in which you are interested (Ram, Ram & Sprague, 2012). In the second category of accidental OER software can be found such services as Slideshare, scribd, YouTube, iTunesU, and Flickr. These were developed to meet other needs but have emerged as good places for sharing and with a strong community of educational users. Some of these also now help to spread the openness message through their support for Creative Commons (CC) licences.
Ideally, the platform for the providers of OER would be to offer of multiple content input and multiple content output formats, supported by clear licensing, tracking all use of content, providing easy tools for customisation and sharing back, enabling very easy resource discovery, revealing the options for how the resources are intended to be used and how they actually are used. One of the key requirements for OER for the user is its "invisibility" as part of the range of resources. This means that OER needs to be flexible and seamless across relevant content and assessment as required, integrated into both curriculum and the learning experience.

Institutional policies for the promotion of OER
As take up of OER becomes more widespread then the decisions made to support them and share the ways forward need to be shared and understood. The "policy" level can be a very efficient way forward by setting an agenda that openness works towards. At the institutional level this can be important to help cross the chasm between isolated innovation and the mainstreaming of innovative approaches.
The last year has seen some important policy developments for OER, with a number of national and federal bodies moving to legislate in support of OER. Notable examples include the OER K-12 Bill in the USA, the São Paulo Department of Education's mandate for BYNC-SA licences on educational materials, the Indonesian Ministry of National Education's mandate for open textbooks, and the OER movement in South Africa. Political changes like these reflect the growing momentum behind the view that "all publicly funded resources are openly licensed resources" (Wiley, Green & Soares, 2011).
In broad terms, policymaking that changes goals and metrics can have an important scaling effect. However, it should be recognised that policies are limited as tools for promoting innovation. Indeed, those who innovate may well be those who do not feel bound to follow restrictions of policy and find ways around it. Policy can also be very bound to contexts, and so the sharing of the policies themselves of more limited value than might be hoped. OER communities are diverse, and policies need to reflect different cultures and needs just as repackaged OER must be appropriate to context. Policy is also susceptible to being reversed by changes to that policy, such as with the TAACCCT Federal Grant Program (Keller, 2011). While some policymakers are contributing to frameworks that support OER as it goes mainstream, others may attempt to ameliorate changes to existing business models. Copyright remains a contentious issue, and anti-piracy bills similar to the proposed Stop Online Piracy Act (SOPA, 2011) could still have an adverse effect on sharing.

What evidence is there of use (and re-use) of OER?
While finding evidence about the use of OER remains a challenge, it can be argued that a clearer picture of the world of OER is coming into focus. The OER Evidence Hub is pulling together data from a range of sources to support the arguments of the OER movement. Although by its very nature OER use is often difficult to analyze, OER projects need to do a better job of recording successes and providing evidence about reuse and reappropriation.
Any lack of reliable evidence might be viewed in one of three ways: that there is in fact little reuse; that there is reuse but it is not visible; or that the accepted definition of reuse is not a useful one and we should focus on value to the user rather than be concerned with labelling particular instances of activity. We need to encourage the use of learning materials which allow for attribution when content is remixed or repurposed. Tools like OER Glue have shown how digital platforms can support the process of creating, evaluating and linking OER into course structures. But designing courses for re-use requires a culture of sharing and collaboration (Ossiannilsson & Creelman, 2011). Technological solutions alone will not be sufficient; educators need to adopt a more positive outlook to using and sharing educational resources for OER to become truly mainstream.

What can be done to improve OER sustainability?
Governmental bodies are now beginning to tentatively fund OER on the grounds that the public should have access to research and educational materials which they have funded through taxes. This is a big step forward from a policy perspective, but there remains a danger that this kind of funding will be reduced as OER curricula are fleshed out and legacy OER grow. While OER advocates may be winning arguments about the best way to spend public money on educational materials, the long-term sustainability of OER remains the focus of research. The majority of OER are still produced by philanthropists, colleges themselves, and the efforts of faculty (Hampson, 2011). Dependence on philanthropy is unsustainable, and runs the risk of affording donors too much influence over curriculum production. As financial pressure on (especially higher) education increases, faculty may feel that the extra efforts of producing OER are unwarranted while educational institutions are likely to reallocate funding for OER production from other areas.
It's important to distinguish issues of sustainability from questions about business models. If we treat sustainability purely as a problem raised by the "free" element then we overlook the fact that sustainability often depends on recognising those benefits brought to other parts of a business or indeed broader benefits to the overall ecosystem of education.
6. Copyright and licensing A range of Creative Commons licences have been firmly associated with openness and OER for some time now, and in light of recent policy successes it might be tempting to think that the licensing problem is solved. Indeed, in many contexts and scenarios, highly effective licensing arrangements are already in place. It is important to bear in mind that the CC-BY license does not restrict the commercialization of "open" content (Green, 2011). Commercial providers (including Apple and Amazon) have shown interest in the e-book and textbook markets, using the savings made through digital textbooks to preserve market share. Advocates need to continue to make the argument that e-textbooks and open textbooks are fundamentally different, and that subtle differences in licensing agreements can have profound implications.

What are the costs and benefits of using OER in teaching?
One of 2011's most visible interventions in the world of OER was the impact on school and college textbooks in the USA. Through initiatives like Utah Open Textbook, Students PIRGs Textbook Rebellion and the $5 Textbook, college students were able to experience the significant cost savings offered by OER. Textbooks normally costing hundreds of dollars can be provided for free online or between $5 and $30 for physical copies, resulting in widening student participation and improving access.
Further research is needed into the ways in which the shift to OER can support deeper learning while contributing to cost savings. OER also has the potential to change the learning experience itself, especially in terms of supporting formal, institutional learning and informal, often self-directed learning Open material designed for open learning, such as that from The Open University's courses published on the OpenLearn website, can be used to support the broad spectrum of subjects taught at undergraduate. The OERuniversity (Witthaus, 2012) has proposed a collaborative approach to providing accreditation for such learning at much lower cost to the institution and to the learner.
8. Promoting and advocating educational methods which use OERs 2011 was a successful year for OER advocacy, with important breakthroughs in a number of areas (particularly policy). The successes of the OER Advocacy Coalition are in part due to a diverse team of advocates working effectively across political and geographical borders to build communities, co-ordinating and sharing their activities (Google, 2012).
While this advocacy movement has raised awareness of OER and made a significant impact on policymakers, it should be noted that commercial publishers and other interested parties continue to make attempts to ameliorate legislation which supports OER. There remains a crucial role for individual acts of advocacy which can spread the OER message into new areas of application in teaching, learning and research. However, few staff feel incentivised to contribute to OER. (This can result from seeing the demands of OER production as extra work or from reluctance to share intellectual property other than in accordance with traditional forms of dissemination.) Institutions need to take a lead with developing skills in instructional design and educational technology among staff in all faculties, though there remain questions around how best to engage and incentivise.

How do we ensure OER is of high quality?
All educational materials must meet accepted quality standards, and the so-called "quality" problem is not unique to OER; rather, OER partakes of it on account of being educational. Furthermore, the uptake in OER use is indicative of a growing acceptance of the idea that OERs are not necessarily of a poorer quality than commercial equivalents.
There is also evidence to suggest that OER are challenging accepted notions of quality through developing and implementing resources which are more relevant to the way that learners will engage with curricula in the future. Traditionally, the production of educational resources was restricted in terms of both production and consumption. OER, by contrast, can be produced through frameworks in which "various types of stakeholders are able to interact, collaborate, create and use materials and processes" (Kanwar, Balasubramanian and Umar, 2010). Hence, under the open model it is not only scholars who assess the quality of OER.

Creating the right culture of teaching and learning to improve OER adoption
In some ways this challenge is the mirror image of the issue of advocacy, since it concerns the attitudes and values of educators in situ rather than at the removed level of policymaking. Many state education agencies now have offices devoted to identifying and using OERs and other digital resources in their states. To help states, districts, teachers, and other users determine the degree of alignment of OER to the Common Core State Standards, and to determine aspects of quality of OER, Achieve has developed eight rubrics in collaboration with leaders from the OER community (Achieve, 2011). In Europe, the OERTest project has provided a series of briefing papers for OER assessment and good practice (OERtest, 2011). P2PU is currently developing a model which wraps assessment around the content it provides, effectively embedding it within the OER itself. The P2PU model also provides volunteer tutor support to learners in a cohort (P2PU, 2012).

Improving the value and impact of OER research
Can OERs find a place within existing academic cultures of research, teaching and publication, or must they forge new networks and processes for disseminating knowledge? While there remains a considerable numbers of scholars who are sceptical about the value of open research, open systems of peer review and open access publishing are increasingly becoming accepted, with many academics expressing frustration with existing models (see Boyd, 2008). Although open educational practices can disrupt established patterns of action, an ascendant "culture of openness" is promoting cross-fertilization of ideas between different stakeholders and opening up new opportunities for research collaboration (Nielsen, 2011). Research on openness can thus itself be a catalyst for change.
The OLnet project has acted as an exemplar for a culture of networking and openness towards OER adoption, supporting a number of fellowship schemes and building closer links between institutions and individual educators in discovering new ways to network and research in an open world.

Improving access to OER
Widening participation in education remains a core driver of the OER movement, and each of the other challenges can be understood as attempt to improve access. There have been encouraging policy developments, and considerable progress has been made in the USA with student textbooks over the last year. There remain, of course, significant barriers to OER, including discoverability, publishing models, technical standards and lack of relevant skills. Nonetheless, around the world there is a growing recognition that OER can make a real difference to access. Within the context of OLnet, Teacher Education in Sub-Saharan Africa (TESSA, 2012) brought together teachers and teacher educators from across Africa, offering a range of OER materials in four languages to support school based teacher education and training. It provides a good illustration of how OER itself can be a route to improving participation and widening access by crossing cultural or geographical boundaries.

Meeting the Challenges: a Case Study
The Bridge to Success project (B2S, 2012) offers a good example of how many of these challenges arise in practice. By reflecting on the services that need to be provided to such a project we can help share experiences and prepare for the needs of similar OER projects. The B2S project aimed to introduce content and approaches already applied at The Open University to the US Community College context . The courses were designed to help learners prepare to enter degree level courses and had been demonstrated to be effective in improving attainment for students who lacked the standard qualifications for higher education study. Two courses were selected and these are now available in new versions as "Learning to Learn" and "Succeed with Math". The courses were not previously fully available as OER and indeed were designed around print and telephone support models to meet the original target audience.
The challenges can usefully be divided into four categories, each containing three challenges. First, there are the challenges of preparation: copyright, technology and access. These have potential solutions and so can primarily be addressed through a process of selection, and will be discussed in more detail below. Second, there are three common issues for learning: quality, sustainability, and reuse. These are a hybrid between applying existing processes, such as in B2S the use of the Quality Matters framework already in place in the partner Community Colleges, and of considering the specialised concerns of openness. The third group are areas for current research: cost/benefit, impact of the research and policy. These become the focus of the reflection and approach to evidence within the project itself. The fourth category includes the emerging areas of advocacy, culture and assessment. These are not so much the focus of B2S itself but are very much part of the rationale for the OER movement of which it is a component. B2S contributes through take-up, demonstrating ways in which openness aids flexibility and take-up, crossing cultural boundaries internationally and between learning sectors, and opening up a more flexible approach to assessment and attainment.

Preparing for Openness
Each of these categories could be examined in more detail. We will concentrate here on the three challenges within the first group (preparation) and look at how they indicate a way forward based on the services and support needed for working on open projects.
Copyright: for B2S we have mandated use of the CC-BY licence (Creative Commons, 2012). This is now the most popular licence for OER as it allows wide use without additional clarification while retaining reference back to the originator but not insisting on imposing a "sharealike" condition that could inhibit remixing with other material with a more restrictive licence. For some cases the non-commercial (NC) licence is preferred as it extends a message of free of cost use and avoids potentially misleading representation of open content. The particular challenge for us as a partnership was to match the existing preference of NC by The Open University to the use of CC-BY across the funded projects. This example serves to highlight accepted choices where issues still need to be understood.
Technology: B2S was prepared to consider other platform choices but selected the existing LabSpace provided as part of OpenLearn from The Open University (a Moodle-based system). This has proved to be strong in supporting multiple versions of content through a shared editing approach and direct support for learners. While the platform was developed some time ago, it proved well suited to the B2S case which required reuse, both cohorts of learners and open learning, and the ability to provide tracking data.
Access: in B2S there are access challenges of discoverability and accessibility. Discoverability is addressed by siting content with other OER and by identifying and working with appropriate pilots. Accessibility is particularly important in the context of material that is not only open for use but part of an offering to identified students. Specialist workshops considered approaches while a process of developmental testing by the accessibility team attached to The Open University's Institute of Educational Technology identified issues. An important side effect was to reconsider the features of the underlying LabSpace platform and recognise that revisions could be prioritised and then implemented. Making platform rather than content changes to improve usability and accessibility has brought benefits for all users not just those working with B2S content.

Services for Open Education
Open Education clearly has its challenges, some of which we are starting to know how to overcome, others of which are emerging as the field progresses and increases its ambitions for change. In working across OLnet and applying what we have learnt in B2S one can identify the range of services that are needed, and the collective way in which they might be met. A tentative list of such services and the way they interoperate is given below. 1 Supporting the practicalities of OER. To provide expertise or routes to expertise for technical, management and processes associated with the successful conversion of course material to OER, and the routes to adopt and make use of them.

2
A technology base for OER. There is recognition that there is a need for underlying technology and shared platforms. Existing work needs to be revisited and enhanced to provide an immediate answer to project needs.

3
OER courses and web presence: draw on the experience of projects such as Bridge to Success to provide best practice and advice on running pilots, surveying instructors, students etc. Support through development of the instruments and technology needed but also by packaging up knowledge into courses e.g. within the School of Open.

4
A Fellowships "Plus" programme: the fellowship approach has been very successful both in OLnet and the UK-based SCORE. Extending the concept to offer greater persistence through the development of research nodes that link back to shared support and then reach out to the local environment.

5
Collective action on a regional or sector basis. This may operate as an extension of the fellowship model but at an institutional scale. The open collaboration that is possible around OER means that connections do not all need the same funding source, or the same motivations, to work together.

6
The OER Evidence Hub: the hub developed by OLnet shows the potential and interest in a shared research base which links practical outcomes and data from a range of projects and initiatives. Evidence about OER needs to demonstrate validity through collective intelligence, curation and peer review, while remaining open to contributions and use by all.
As decisions are made to adopt OER practical guidance is needed and key factors can be identified with reasonable confidence. The findings highlight the potential for impact of OER on policy and on practice in education but also confirm weaknesses in the evidence base. In order to help new projects and initiatives to make good choices as they work with Open Educational Practices, the way forward is to accept some of these partial pieces of evidence while making their basis clear and understanding the contexts in which they can apply.