• \
    Start Submission Become a Reviewer

    Reading: Equipping the Next Generation for Responsible Research and Innovation with Open Educational ...

    Download

     A- A+
    Alt. Display
    Special Collection: Open Education in Latin America

    Articles

    Equipping the Next Generation for Responsible Research and Innovation with Open Educational Resources, Open Courses, Open Communities and Open Schooling: An Impact Case Study in Brazil

    Authors:

    Alexandra Okada ,

    The Open University UK, Twitter, GB
    X close

    Tony Sherborne

    Mastery Science, GB
    X close

    Abstract

    There has been an increasing number of projects and institutions promoting open education at scale through Open Educational Resources (OER) and Massive Open Online Courses (MOOC) to broaden learning opportunities for all. However, there are still many challenges in relation to sustainability, effective implementation and evidence-based impact to support educational policies. To explore this gap, this paper focuses on an integrated model that combines OER, MOOC, Communities of Practice (CoP) and Open Schooling to promote open education and foster inquiry skills for Responsible Research and Innovation (RRI), a key approach coined by the European Commission. This study focuses on the ENGAGE Project, with 14 partners in Europe who produced more than 300 OER, 60 MOOC in ten languages and supported 27 CoP with more than 17,000 members in the world including more than 2,000 from Brazil. Through a novel framework on impact assessment of OER for RRI underpinned by a mixed method approach, this study examines the influence of open education on academic and non-academic groups and the correlation between the outputs developed in the project with the outcomes reported by the Brazilian communities. Qualitative and quantitative data from the ENGAGE platform, journal articles produced by the Brazilian participants and interviews with authors were analysed. Findings report the different ways that the community developed open schooling projects, the changes in their practices to foster digital scientific literacy, and outcomes with implications for society.

    Keywords: Responsible Research and Innovation, Open Education, OER, MOOC, CoP, Open Schooling, impact case study, inquiry skills, teachers’ continuing professional development
    How to Cite: Okada, A. and Sherborne, T., 2018. Equipping the Next Generation for Responsible Research and Innovation with Open Educational Resources, Open Courses, Open Communities and Open Schooling: An Impact Case Study in Brazil. Journal of Interactive Media in Education, 2018(1), p.18. DOI: http://doi.org/10.5334/jime.482
    562
    Views
    163
    Downloads
    2
    Citations
    5
    Twitter
      Published on 14 Dec 2018
    Peer Reviewed
     CC BY 4.0
     Accepted on 16 Sep 2018            Submitted on 03 May 2018

    Introduction

    Innovations from science and technology are vital to Europe’s future (Hodson, 2011, 2014). To ensure that the process and products of research are acceptable to society, these endeavours now fall under the framework of Responsible Research and Innovation (RRI). RRI sets out six keys for how all societal actors should work together: Science Education, Public Engagement, Open Access, Ethics, Gender and Governance (EC, 2012, 2013). For citizens to participate in the processes of RRI, they will need to be sufficiently literate about how science works and understand, among other things, the benefits and risks of technology, and ethical thinking, in order to participate in debates and make informed choices (Owen et al., 2012; Ryan, 2015).

    The international initiatives in science education which are emerging this decade pay particular attention to fostering new skills and knowledge for teachers, centred on the use of Inquiry-Based Science Education (IBSE) for RRI (Okada et al., 2015; Alcaraz-Domínguez et al., 2015; Bayram-Jacobs, 2015; Mikroyannidis et al., 2016). This approach to teaching science has major potentials to raise students’ interest in science, to increase knowledge of how science works, and to develop competencies such as critical and creative thinking, vital for understanding socio-scientific issues. Science teachers typically focus on delivering the canon of scientific knowledge, transferring knowledge. RRI-based teaching, by contrast, focuses on how we know what we know, i.e. the nature of science, and the effects of that knowledge, that is its social impact. Such approach requires the adoption of different pedagogies such as inquiry (Okada, 2014; Blonder et al., 2016; Bardone et al., 2017; Gorghiu et al., 2016). Inquiry-based teaching aims at developing the skills of scientific thinking, so that learners will be able to interpret evidence, weigh up technologies, make informed judgements, and argue their views (AAAS, 1993). The literature demonstrates a poor record of success for attempts to build an RRI teaching force, and transformations generally occur with only small numbers (Gorghiu et al., 2015; Kiki-Papadakis and Chaimala, 2016; Okada et al., 2016; Bayram-Jacobs et al., 2017; Peciuliauskiene, 2017; Blonder et al., 2017).

    To explore this gap, this study focuses on the project ENGAGE – equipping the next generation of teachers and students for RRI through a novel model that combined Open Educational Resources (OER), Massive Open Online Courses (MOOC), Communities of Practice (CoP) and Open Schooling to bridge informal, non-formal with formal learning for RRI. This approach was implemented during three years from 2014 to 2017 and linked with the national curriculum of 11 countries mostly in Europe: UK, France, Germany, Spain, Switzerland, Norway, Greece, Cyprus, Romania, Lithuania and Israel. The purpose of ENGAGE was to support the EU’s ambition by shifting the practice of science teachers on a massive scale by engaging 12,000 teachers (approximately 1,090 teachers per country) to foster scientific knowledge and inquiry skills for RRI.

    Background

    The key to engaging the next generation to be aware of and participate in socio-scientific issues is to change how science is taught. Traditionally, students gain an image of science as a body of content. Teaching about RRI (Burget et al., 2017; de Vocht et al., 2017) however, focuses more on areas of emerging science and technology whose applications and implications are uncertain, and where values and debate are as important as established facts (Owen et al., 2012; Von Schomberg, 2013). The shift is hugely challenging. High stakes education systems marginalise teaching about the nature of science. The greater challenge is to help teachers develop the beliefs, knowledge and classroom practice for RRI teaching in formal education (Kiki-Papadakis and Chaimala, 2016) and non-formal learning (Petrescu et al., 2015). This requires adopting a more inquiry-based methodology, which gives students opportunities for self-expression and responsibility for coming to informed decisions.

    While primarily a teacher-based intervention, ENGAGE targets all three components of student scientific literacy – motivation (care), knowledge (know), and skills and attitude (do) – for preparing students to engage with issues around emerging technologies (Okada, 2016a). ENGAGE shows how learning science concepts can be set within the context of its implications to society.

    RRI Curriculum

    Our curriculum materials invited students on a journey to the future. Showing possible future scenarios means making the pros and cons of technology more concrete; and help all students to think through the current and emerging issues in more depth. Making science more relevant to students’ concerns – which are known to be future orientated – will increase the likelihood that they can apply what they have learned outside school and respond to societal challenges (Sherborne and Okada, 2013).

    ENGAGE has specified its ‘RRI curriculum’ (Figure 1), synthesised content from the European Commission about RRI (Owen et al., 2013) and US Next Generation Science Standards curriculum framework (NGSS Lead States, 2013) based on key components:

    1. Science Media: Much of our scientific information is interpreted by the media, who may give an unbalanced, biased, black and white or sensationalised account. The source of information needs to be assessed in terms of its purpose, scientific credentials (reliable and valid) and currency (Jarman and McClune, 2007).
    2. Technology Impact: Technological and scientific developments are the basis for a better future. These advancements must be planned carefully in order to maximise the benefits and reduce risks, particularly any harmful impact (Gott and Duggan, 1996).
    3. Values Thinking: In emerging science and technology, there are often uncertain issues with unclear implications that require socio-ethical thinking. Decisions should be made by taking into account the views and concerns about social, economy and environment, science-related values and actors in societies (Fullick and Ratcliffe, 1996).
    4. Big Science: Science is no longer an individual search for knowledge, but a collaborative complex enterprise, done with participatory research with and for society. Funded largely by corporations and governments and politically determined, it favours practical applications to address societal needs in accordance with societal values with evidence-based scientific argumentation (Osborne, 2010).
    Figure 1 

    RRI Curriculum.

    This set of areas supported the development of ten inquiry skills models (Figure 2) during the project, which were covered through its Materials and Courses (Okada, 2016a):

    1. Devise questions: Define a clear scientific question which investigates cause or correlation relationships between different factors.
    2. Interrogate media: Question different sources and assess their validity by judging the reliability of the source, check for bias and evidence for claim.
    3. Examine consequences: Evaluate the merit of a solution by identifying consequences regarding economy, society, and environment.
    4. Estimate risks: Measure risks and benefits by assessing its probability, weighing up and combining its probability and the scale of its impact.
    5. Analyse patterns: Interpret observations and data in a variety of forms to identify patterns and trends by making inferences and drawing conclusions.
    6. Draw conclusions: Decide whether the claim made by a piece of research is supported by sufficient data.
    7. Critique claims: Check strength (quality accuracy and sufficiency) of evidence provided and identify lack of clarity of justification.
    8. Justify opinions: Synthesise scientific knowledge, implications, and value perspectives into an informed opinion with evidence-based arguments.
    9. Use ethics: Understand and use ethical thinking to make informed decisions and explain why different people may have different viewpoints.
    10. Communicate ideas: Describe opinions and accomplishments using the major features of scientific writing and speaking.
    Figure 2 

    Inquiry skills for RRI (authors, 2016).

    This set of skills was designed to maximise understanding of the nature of science and decision making, particularly through collaborative inquiry-based learning projects. Yet the shift towards RRI-based or ‘humanistic’ science teaching (Aikenhead, 2006), and inquiry-based pedagogy (Bardone et al., 2017; Okada and Bayram-Jacobs, 2016) is challenging.

    RRI CPD: adopt-adapt-transform

    Aikenhead (2006) underscores the lack of success of continuing professional development (CPD) programmes to transform teachers with a list of reasons for failure. The notion that any event or short-term set of workshops produces lasting change has been discredited. Real transformation such as the one required to move to RRI-based teaching is a long term and rather complex process. The ENGAGE programme synthesised contemporary models of CPD with the curriculum development expertise into a framework for supporting teachers through this process of transformation, which combined two models (Sherborne and Okada, 2013).

    The first model is a continuing professional development based on three stages, adopt-adapt-transform (Figure 3), to represent an RRI learning and teaching pathway integrated to teacher’s CPD (Dwyer et al., 1991; Rogers, 2003).

    • Stage 1 Adopt: Minor change – extending topics already taught through an OER with a socio-scientific issue, which presents little RRI content for motivational purposes to be applied in short lessons.
    • Stage 2 Adapt: Significant changes – teaching inquiry processes with problem-solving lessons. There is a casual infusion of more RRI content but with no explicit purpose.
    • Stage 3 Transform: Major changes – teaching science content with a scenario-based project.
    Figure 3 

    Transformational CPD Model (authors, 2016).

    The implications of the model influenced the design of ENGAGE:

    • Each step involves much work for teachers, and so only a proportion are expected to progress. To enable large numbers to reach ‘Transform’, large numbers must be attracted onto the first step. Therefore, our Adopt strategies must be very easy to spread and to take on-board, with easy-to-use materials, massive appeal and achieving massive take-up.
    • The key to success is to understand how teachers move up. A significant number of researchers (e.g. Hoban, 2002; Clarke and Hollingsworth, 2002) believe teacher learning is like a complex system. It requires a highly interrelated set of conditions working together over a long period, to overcome the tendency to revert to the status quo.

    We use a second model (Figure 4), which represents these conditions and relationships as an ‘inquiry cycle of learning’ (Guskey and Huberman, 1995). It involves experimentation, feedback, reflection and input of new ideas. The teacher inquiry cycle is shown in the diagram below.

    There are major advantages of structuring ENGAGE’s CPD as a teacher inquiry:

    • It is the ideal preparation to help them use Inquiry Based Science Education (IBSE) in their own classroom (Loucks-Horsley et al., 1998).
    • They will come to see that the principles apply equally to themselves and those they teach.
    • It is an intensive learning experience for teachers, a key factor in successful CPD (Supovitz and Turner, 2000).
    Figure 4 

    Inquiry learning cycle for teaching staff’s professional development adopted by ENGAGE universities.

    RRI Open Education strategies

    ENGAGE targeted each part of the cycle, with these key strategies (project outputs):

    • Materials (OER): Teachers’ first classroom experiences are pivotal. They have to see positive outcomes on their students to continue experimentation and begin the process of reflection (why did that work?). ENGAGE Materials combined relevant, topical contexts, and ease of use, as well as good curriculum coverage with educative pedagogies embedded (Bruner, 1960). Davis and Krajcik (2005) argue that materials should be ‘both effective and efficient’ in the way they can communicate a rationale for new content and pedagogy, and help teachers deal with implementation problems. Materials in our ‘teacher inquiry cycle’ were designed to facilitate the first process of ‘classroom experimentation’. They were published as ‘Open Educational Resources’ (OER) on the Knowledge Hub (website), to encourage their free use, modification, and re-publishing by teachers, under a Creative Commons license.
    • Community (open CoP): The ENGAGE online professional learning community focused on the use of the materials and creation of open schooling projects supported by a brokering system platform for partnerships. It is designed to foster reflective dialogue on the inquiry questions with knowledgeable ‘facilitators’ to guide the learning in the form of ‘expert RRI-teachers’ and ‘RRI professionals’ to support teachers learning to use RRI. Also known as a ‘community of practice’ (CoP), this is a group of people working towards a common goal, sharing practices, and interacting regularly (Lave and Wenger, 1991; DuFour and Eaker, 1998). CoP creates the conditions for new members to engage with more experienced members, developing knowledge integrated with practice, and becoming more expert (Wideman, 2010). This fitted with our ‘teacher inquiry cycle’ model where we aim to facilitate the process of ‘personal reflection and peer exchange’. Having used our Materials, we were aiming to stimulate the reflection on why and how they work, which will lead towards a) more effective usage and b) towards changes in teachers’ beliefs, i.e. a change in practice, and c) confidence to support open schooling projects. There are existing spaces where teachers could interact online. The reason for creating a new community is that it needs to act as glue between other ENGAGE strategies: Materials, Courses and Projects. Time is their most precious commodity. Teachers will be more likely to seek advice from ‘expert RRI teachers’ on a ‘particular inquiry-based technique’ if they can see the forum on the same page as they downloaded the Materials. The choice of an online community is also based on the CPD criterion of long-term teacher involvement, the scalability to large numbers, and the practical efficiency of ‘anytime, anywhere’ learning.
    • Courses (MOOC): Our short, highly interactive online courses provided ‘just-in-time’ input (Kop et al., 2011) from training professionals and expert RRI-teachers. These were based on best practice formal and informal learning, and created the conditions to support teacher knowledge development. ENGAGE participants were interested in pedagogies for RRI teaching. The pedagogies were embedded within the Materials, and the function of the Community content is to help teachers reflect on these before and after using the Materials. To do this the Knowledge Hub provided layers of information and coaching, which guided teachers to look at different aspects of the pedagogy. This kind of content was designed for what we call ‘just-in-time’ learning where ideas are presented at the time they are needed. This is in contrast to the traditional professional development format of workshops where ideas are presented out of context, often a long time before they are ever used in the classroom. To encourage teachers to use our just-in time content, questions and articles were constantly visible and accessible from the same web page as the resource that was downloaded.
    • Open Schooling Projects: Collaborative projects focused on real-life challenges and innovations, including associated ethical and social and economic issues. They were promoted by ENGAGE though materials, courses, CoP and a brokering system to facilitate the partnerships among teachers, local communities, enterprises and families. This approach was highlighted by the European Commission report Science Education for Responsible Citizenship (Ryan, 2015) for bridging formal, non-formal and informal learning to ensure relevant participation and meaningful engagement of society with science. It aims to motivate students to learn science and increase the uptake of science studies and science-based careers for improving employability and competitiveness.

    Research questions

    This study focuses on the impact of the ENGAGE open education for RRI beyond Europe. The meaning of impact refers to the influence and effect of an Open Education project to promote RRI on academic and non-academic groups. This work examines the integrated model that combines OER, MOOC, CoP and Open Schooling to foster inquiry skills for RRI in Brazil. With this aim in mind, we investigate the following research questions:

    1. Did ENGAGE have influence on academic and non-academic groups for promoting RRI in Brazil? In which ways?
    2. Is there any correlation between the outputs (Open Education strategies) developed in the project with the outcomes reported by participants? In which ways?
    3. Has ENGAGE enabled or supported prosperity in communities, enterprise/s or other sector/s? In which ways?

    Methodology

    Framework to assess impact

    Two key references were used to design a framework (Figure 5) on impact assessment of OER for RRI. The first document provided some quantitative ‘indicators for promoting and monitoring RRI’ (EC, 2015) grouped in six key components: science education, public engagement, open access, gender equality, ethics and governance.

    The second document provided some qualitative ‘indicators for assessing impact on society and prosperity’ (OU-UK, 2018) which were grouped also in six key components: professional development, participatory research, policy change, social prosperity, business prosperity and sustainability.

    Figure 5 

    Framework to identify the influence of an Open Education project for RRI.

    Data collection

    Quantitative and qualitative data were produced by the communities from the ENGAGE portal (WordPress), OER (SlideShare), MOOC (Open edX) questionnaires, interviews and webinars (Hangouts), video library (YouTube), collective dialogue maps (LiteMap), and social media (WhatsApp, Facebook and Twitter) during three years of the project. Participants signed the consent form and became aware of ethical procedures including data protection and privacy, as well as transparency and openness of research procedures, key issues for open science (David, 2007) and RRI (EC, 2012). Anonymised data were available for Brazilian communities who developed their own studies, supported by the European ENGAGE team. Six Brazilian groups who authored the peer-reviewed articles were also interviewed (Table 3).

    Participants

    At the end of three years of the European project ENGAGE, there were 17,120 members in the Portal including 2,179 members from Brazil from schools, universities, learning-centres, educational and technology enterprises, local communities, and members from local councils. Figure 6 shows the geographical distribution of active participants, reasonably spread throughout Brazil, with a large concentration of groups in six states: Ceará and Bahia in the North East, São Paulo and Rio de Janeiro in the South East, and Paraná and Santa Catarina in the South.

    Figure 6 

    Teaching Staff participants of ENGAGE in 2017.

    Findings

    ENGAGE influence on academic and non-academic groups

    To answer the first research question, Table 1 provides an overview of the indicators for analysing the influence of ENGAGE to promote RRI among academic and non-academic groups in Brazil.

    Table 1

    Indicators for monitoring RRI.

    RRI DIMENSIONS ENGAGE
    Open Education strategies    		 ENGAGE 11
    countries    		 Average per
    country    		 Brazil
    Science Education RRI workshops provided 33 3 5
    RRI Resources (OER) created 330 30 42
    RRI MOOC (courses) delivered 22 2 1
    Public Engagement Academic members (universities) 12 1 5
    Non-academic (school & local communities) 660 60 319
    Teaching staff members 17,120 1,100 2,179
    Open Access Open presentations in conferences 32 3 5
    Open peer-reviewed articles 10 1 6
    Open schooling projects 27 2 2
    Equality Gender Women coordinators 15 2 4
    Women facilitators 9 2 3
    Total of partners (women + men) 25 2 6
    Ethics Ethical discussions 38 3 12
    Ethical issues raised 9 1 6
    Governance RRI Institutional debates (events) 12 1 9
    RRI communities 11 1 5
    Policies and Institutional changes 3 - 2

    The ENGAGE European consortium, which included 25 partners (15 women and 10 men) from 14 institutions (12 universities, 1 science centre and 1 technology company), developed the ENGAGE Portal with three RRI workshops, 330 OER, 30 MOOC and 42 open schooling projects materials in ten languages, to be implemented widely in and beyond Europe. Through the implementation of professional development within partners’ universities and projects with local groups, ENGAGE reached approximately 660 non-academic communities (public and private schools, centres, clubs, hospitals, museums and libraries). Various studies were published, such as ten peer-reviewed articles, 32 conference presentations and papers including at the international conference on Public Communication of Science and Technology in 2015 in Brazil. The ENGAGE portal and online courses were widely disseminated among the partners’ communities in their institutions and social networks, which included Brazilians. The ENGAGE portal presented different types of Open Education strategies in all six RRI dimensions (Table 1), generating various debates around the OER. In terms of ethical discussions there were 38 topics and nine issues.

    At the end of the project, each of the 11 countries organised a national conference to create opportunities for RRI debates and consolidate their ENGAGE RRI communities. There were two groups of Brazilian members who attended ENGAGE conferences in the UK and there were four events organised in Brazil that promoted ENGAGE open education widely in various states, particularly in Bahia, Ceará, Paraná and Santa Catarina.

    The impact in Brazil was significantly representative (2,179 members) compared to the average target expected per country of the consortium partners who were responsible for coordination and support actions in their country (average target per country was 1,100).

    Brazilian members produced a total of 42 OER (in various formats), they used the platform content to support five workshops and one course unit, which was part of the ENGAGE MOOC. They also adapted two existing ENGAGE OER (GM decision and Exterminate) to the Brazilian curriculum in Portuguese. They developed three studies about ENGAGE focusing on scientific skills for RRI in Brazil. Two large open schooling projects were created about ‘Genetic Modified Food’ and ‘Zika’ which are described in the next section (case studies). In addition, they contributed to the ENGAGE open publication with five conference papers and six peer-reviewed articles. The Brazilian community increased in 2017, especially in the LiteMap Platform (Okada et al., 2015a) with various dialogues about ethical issues (6), ethical discussions (12), global RRI debates (9), and RRI communities in states (5). Policies and institutional changes were promoted to embed OER for RRI in the Brazilian curriculum (Figures 7 and 8).

    ENGAGE Open Education strategies

    To answer the second research question, Table 2 presents the number of participants in Brazil grouped by the type of engagement for RRI teaching (adopters, adapters and transformers) distributed respectively by ENGAGE Open Education technologies and strategies (OER, MOOC and Communities).

    Table 2

    Promoting RRI through Open Education in Brazil.

    Open Education strategies Interactive Media environments Adopters: participated in ENGAGE Adapters: used OER and shared changes in practices Transformers: developed open schooling projects with students and articles
    OER WordPress 2179 100% 567 26% 75 3%
    SlideShare 1704 78% 890 41% 32 1%
    YouTube 78 4% 35 2% 12 1%
    MOOC Open edX/PDF 180 8% 26 1% 13 1%
    Communities WhatsApp 520 24% 320 15% 180 8%
    Facebook 1550 71% 768 35% 345 16%
    Twitter 264 12% 76 3% 15 1%
    Hangouts 70 3% 25 1% 25 1%
    LiteMap 1346 62% 125 6% 46 2%

    In terms of ENGAGE OER (WordPress), there were 2,179 participants who accessed OER topical lessons, but only (26% = 567) commented OER sequences and (3% = 75) reported outcomes with open schooling projects. Most of these participants (78% = 2,704) accessed OER slides in SlideShare, but a few accessed videos in YouTube (4% = 78).

    In terms of ENGAGE MOOC, delivered in an English platform (Open edX), there were very few registrations (8% = 180). The group who completed the course (1% = 13) translated and adapted the open content to support their teachers using their own platforms.

    In terms of ENGAGE communities (CoP), there were 1,346 participants in LiteMap, 1,550 participants in Facebook and 520 participants in WhatsApp at the beginning (Adopt stage). However, the number of participants in all CoP platforms were reduced at the end of the project during open schooling projects: 345 participants in Facebook, 180 participants in WhatsApp and 46 participants in LiteMap.

    LiteMap was one of the technologies suggested in the MOOC, which was translated to Portuguese by a member of ENGAGE Brazil. This platform was recommended by ENGAGE facilitators to help teachers and students discuss RRI and use evidence-based dialogue Maps for developing informed opinions and making evidence-based decisions.

    Figure 7 shows the registration of Brazilian participants during four years from March 2014 to November 2017 with a significant increase during December 2016 to November 2017.

    Figure 8 presents the number of open content produced by the community in terms of maps, issues, ideas, arguments, counter arguments notes and chats. This graph shows an increased number of content produced in the same period (December 2016 to November 2017) when there were various events organised by the Brazilian community and new OER materials, including RRI reports, published.

    Figure 7 

    Members registered from 2014 to 2017.
    Figure 8 

    OER production in LiteMap from 2014 to 2017.

    Impact on society and prosperity

    To answer the third question about the ways that ENGAGE research enabled or supported prosperity, two case studies about open schooling projects were selected based on the journal articles produced by Brazilian members (Tables 3 and 4).

    Table 3

    Journal articles about GM produced by Brazilian members during the ENGAGE project.

    Co-author Interviewed Journal Article Open Schooling Project (question) Open Schooling Project (findings) Impact reported by Interviewees (summary translated)
    Brazilian Postgraduate Coordinator Responsible Research and Innovation for the Media Facebook: Community Involvement in the Study on Agrobiodiversity. Creative Education. To what extent are OER used to explore the issues related to agricultural biodiversity, specifically to GM products in order to promote RRI? OER were successfully adopted by 54 teachers through a collaborative process of teaching and learning with social networks, which engaged 340 students who participated in the GM discussion from Brazil, including learners from other countries Portugal, Ecuador, Spain, Luxembourg, UK, USA. Participatory Research using OER and social media enabled the co-creation of new OER that were widely disseminated: images, games, video clips, articles presentations and interviews. The variety of OER enabled teachers to consider gender preferences. The combination of OER and social media promoted students and public engagement.
    Brazilian Postgraduate Coordinator Experience of Environmental Education using Responsible Research and Innovation of the Pontifical Catholic University of Paraná in the European Project Engage. Diálogo Educacional. In what ways are OER linked to formal education to foster RRI skills? RRI skills were fostered through a variety of OER that were embedded in the curriculum such as video clips, interviews, magazines, games and maps. These OER were used and co-produced by more than 583 students supported by 19 teachers from secondary school and 11 lecturers from higher education. OER were embedded in the curriculum by various teachers and lecturers from schools and university to foster skills for RRI through more learner-centred approaches. The positive outcomes from open schooling projects are useful to support policy and institutional changes.
    Brazilian PhD student Factors influencing teachers’ adoption of AR inquiry games to foster skills for Responsible Research and Innovation. Interactive Learning Environment. What are teachers’ views about a novel inquiry game, which is an OER created with an Augmented Reality (AR) open platform to foster inquiry skills for RRI? The AR inquiry game about GM developed by ENGAGE members from Brazil was considered easy-to-use and useful by 18 teachers from Brazil who used the game with their students (390). They found it meaningful to help students practise key skills: devise questions, estimate risks and communicate ideas. Open courses and guidelines for teachers’ professional development facilitated the adoption of AR inquiry game to foster skills for RRI. The game and pedagogical strategies discussed during the course were useful for teachers to work with skills that they were not used to; such as estimate risks, examine consequences and use ethics.

    Table 4

    Journal articles produced by Brazilian members during the ENGAGE project.

    Co-author Interviewed Journal Article Open Schooling Project (question) Open Schooling Project (findings) Impact reported by Interviewees (summary translated)
    Head of CPD (teachers’ professional development) Continuing teacher training using dilemmas with elements of ubiquity. Interfaces Científicas. To what extent teacher’s continuing pedagogical practices can be supported through mobile devices and elements of ubiquity, articulated with face-to-face (F2F) activities in the real classroom environment? Thirty-two teachers and four facilitators used WhatsApp and Google Hangouts. The materials and slides from the Open edX course about RRI were translated and made available through PDF for teachers to access on their mobile devices. Questions and answers were shared just-in-time during their lessons through Apps. Teachers’ professional development was enhanced by the use of mobile devices integrated to their workplace. They became more confident to use OER to foster inquiry skills for RRI. Students also used their mobile devices during their inquiry projects. They found that digital and scientific skills were vital to succeed in university, professional careers and for sustainable economic growth.
    Brazilian Postdoctoral Researcher Argumentation of basic education students about socio-scientific dilemmas in the engage project. Ibero-Americana de Estudos em Educação. To what extent students use evidence-based argumentation to justify opinions about socio-scientific dilemmas? All the elements of the argumentation were identified during the activities carried out by groups, however, it was observed that students had difficulties to elaborate a justification based on evidence and scientific thinking. The open schooling project enabled teachers to be aware of students’ strengths and difficulties in terms of scientific argumentation.
    Secondary school students who took part in this research had opportunity to practise argumentation about socio-scientific issues that are relevant for their community. The combination of relevant socio-scientific issue for the Irecê community and open schooling project to develop inquiry skills for RRI were very meaningful to reflect on social prosperity.
    Brazilian
    Secondary
    School
    Teacher    		 Rubric to assess evidence-based dialogue of socio-scientific issues with LiteMap. Technology Enhanced Assessment. In what ways LiteMap application tool can be used by teachers to annotate students’ socio-scientific discussion and assess their evidence-based dialogue? The discussion about Zika was mapped with LiteMap to support evidence-based decisions. Participants used icons to annotate questions, views, pros, cons, and evidence. Some graphs were used to visualise argumentation. This study focused on open schooling project developed by 24 teachers and 478 students from a public professional school in Irecê, including also 5 collaborators and 2 researchers. The open schooling project created opportunity to discuss science with and for society. This increased participants’ awareness of the importance of science projects to promote Sustainability, Social and Business prosperity. Both communities of academics and non-academics were engaged to develop their views about the various possibilities to reduce Aedes mosquitoes and Zika: through homemade repellents, Oxitec GM-mosquitoes and a natural solution mesocyclops that eat Aedes larvae.

    Three articles focused on GM decisions in Curitiba Paraná, and three articles focused on Zika in Irecê Bahia. The first case highlights prosperity in terms of professional development, participatory research and local policy change. Whereas the second study describes an increased awareness for relevant issues for social prosperity, business prosperity and sustainability. To examine the impact of open education to foster RRI in-depth, the authors of these studies were also interviewed.

    Case 1: GM DECISION – Would you buy GM cereal?

    The interviewees from Paraná described that various Brazilian communities in secondary school and higher education implemented the open schooling project on GM food and agrobiodiversity. GM decision was a very popular theme in this Brazilian state whose economy is based on agronomy and food engineering.

    The ENGAGE OER GM decision (8,600 views, 1,746 downloads) offered a set of editable resources: presentation (PPTX), guidelines (DOCX) and video clips (MP4) and a game to support teachers to discuss the risks of GM corn and GM corn treated with pesticides. To facilitate open schooling projects, ENGAGE courses promoted the CARE – KNOW – DO model (Okada et al., 2018) for Brazilian teachers to engage students in:

    • CARING: About health food, discussing the benefits and risks of GM and pesticides.
    • KNOWING: Justifying their views connecting to what they learned in Biology.
    • DOING: Communicating their scientific views to their family members, experts and friends and creating new OER to promote public engagement and participatory research to increase skills for RRI.

    During the open schooling project students developed their knowledge of inheritance using arguments for and against genetic modification, and weighing up the benefits and risks of an application of science to make a decision. They were also supported to develop seven inquiry skills for RRI:

    1. Devise questions: Create scientific questions about GM corn
    2. Interrogate media: Check fake news related to GM food
    3. Examine consequences: Evaluate GM solutions to population growth
    4. Estimate risks and benefits: Reflect about the use of pesticides versus GM
    5. Use ethics: Make informed decisions about GM food with ethical thinking
    6. Justify opinions: Present own opinion with evidence-based arguments about GM
    7. Communicate ideas: Use scientific discourse to share views about GM.

    Some of the key outcomes of open schooling projects in Paraná developed by students supported by teachers, experts and local community were:

    • The importance of regulations for the use and labelling of GM food, including derived products.
    • A sign for deaf people about GM created by students and a lecturer approved by the deaf association, which was integrated to Libras (Brazilian Sign Language).
    • The free access of reliable sources for citizens about GM food and food treated with pesticides including current research about implications for health, environment and economy.
    • Wide dissemination of open education (OER, open courses and open communities and open schooling projects) for helping citizens become scientifically literate, able to participate in debates and decision-making process.

    Case 2: Zika – Would you exterminate the Aedes Mosquitoes?

    The interviewees from Bahia described that the ‘Exterminate’ OER about Zika was the most used resource, which attracted a large community of educators and lecturers (2,230 downloads, 3,317 views). The whole set of ENGAGE activities about Zika – Exterminate was used by a public vocational school in Irecê, a municipality in Bahia largely affected by Aedes aegypti Mosquito and various epidemic diseases such as dengue, Zika and chikungunya. Thirty-two members of teaching staff developed open schooling projects with 478 students in all technical courses: agricultural administration, clinical analysis, commerce, nursing, environment, nutrition, advertising and occupational safety. Students were from 18 to 22 years old, 60.2% were female and 39.8% male.

    The open schooling project was developed in three phases:

    1. Set up the project (teacher-led): Teachers presented the socio-scientific dilemma: ‘Should we exterminate the Aedes aegypti mosquito?’ Using the ENGAGE game, students analysed data and drew conclusions about the food chain in the ecosystem. They created a table to record what they already knew about the subject, what they would like to investigate (devise questions), where to find data, how to interrogate sources and what they learned about the topic. They were guided to critique claims, reflect on ethical issues and justify opinions of exterminating mosquitoes.
    2. Analyse and solve (student-led): Students read and discussed the articles on genetically modified (GM) mosquitoes, released in Brazil by the British company Oxitec. Students used their mobile devices to search for information. After estimating the risks and examining consequences of using GM mosquitoes or other alternatives, they were challenged to communicate ideas based on arguments and evidence and interact with a scientist.
    3. Communicate (student-led with a teacher intro): Students created argumentative maps using the LiteMap mapping application tool to share their findings and engage their local communities. They included questions, ideas, arguments, counterarguments, data and facts to systematise their evidence-based opinions about how to reduce Zika virus.

    Some of the key outcomes of open schooling projects in Irecê developed by students, supported by teachers, experts and local community were:

    • Increased awareness of the ways that Zika virus was spreading, symptoms, effects including cases of infected mothers who had babies with microcephaly.
    • Increased awareness about the strong link between the lack of sanitation system and access to clean water and the current outbreak of Zika, dengue, yellow fever and chikungunya.
    • The importance of fostering scientific literacy through open schooling projects for students and citizens to be able to interact with scientists and professionals, and together weigh up the pros and cons of each solution for health, environment, economy (e.g. the expensive high-tech alternative, GM mosquitoes; the community owned-solution, mesocyclops that eats mosquitoes’ larvae; and the effective public service, sanitation and clean water).

    Discussion

    This study revealed that ENGAGE have influenced academic and non-academic groups by increasing awareness on scientific skills which are relevant in RRI. Yet, it is not possible to claim that the participants who were engaged with ENGAGE are more aware of the RRI concept (Sutcliffe, 2011; Von Schomberg, 2013) as this approach is very new in Brazil.

    A small percentage of teaching staff (teachers, lecturers, PhD students and course coordinators) attended the ENGAGE MOOC. This group became very committed to learning about RRI and developing open schooling projects as it is a novel area, very relevant for contemporary education. Participants who completed open schooling projects presented evidence of pedagogical changes (Torres et al., 2016; Pinto et al., 2018). The studies developed and published by Brazilian ‘transformer’ teachers, including academic and non-academic members, suggest that key findings are vital for evidence-based impact for society and prosperity.

    The partnership among academics, researchers, scientists, lecturers and society including schools, students and families are vital for opening up more opportunities for open education (Okada et al., 2015b). This can also enhance teachers’ professional development (Supovitz and Turner, 2000) to foster skills for RRI (Wickson and Carew, 2014) by bridging formal, informal and non-formal learning through open schooling projects (Ryan, 2015).

    This study also revealed the role of technologies for dissemination and reaching communities even with remote access. New digital platforms for collaborations between teachers, students, families and universities were used to examine potential solutions for difficulties that emerged during the process. For instance, LiteMap was applied by a group to identify, connect, systematise and evaluate the key components of argumentation to foster scientific thinking in oral and written discourses, with examples that were freely and widely disseminated by participants (Rocha et al., 2017). The AR inquiry game produced by Brazilians enabled teachers to practise and reflect on more learners-centred approaches (Okada, 2016b; Okada et al., 2015c, 2016, 2018).

    The combination of various ENGAGE strategies such as OER, MOOC, open schooling projects and open communities using interactive technologies (Ribeiro et al., 2017; Okada et al., 2015d) made it possible to reach large numbers in the ENGAGE project during three years in Brazil. New studies will be necessary to examine the impact of ENGAGE in the long term.

    This study also revealed the importance of engaging and empowering the community of teachers with lecturers and also students to reflect about their practices, achievements and lessons learned in a scientific way. That means by written scientific discourse. The papers published by the community are vital for evidence-based pedagogical changes supported by knowledge, skills and attitude for RRI that involves all members of society in research and innovation.

    Final remarks

    This study examined how the ENGAGE project has influenced communities in Brazil. Participants were involved through various activities at their own pace with the aim of preparing students to use their knowledge and skills for evidence-based dialogue for making informed decisions.

    This study presented some novel approaches, which combine:

    1. Materials (OER), course (MOOC), communities of practices (CoP) and open schooling projects to foster open education to promote inquiry skills for RRI.
    2. Formal, non-formal and informal learning through open schooling projects to help students to connect what they learn in school with real life scenarios.
    3. Adopt-adapt-transform model of CPD to support teaching inquiry pathways for pedagogical changes and innovation.
    4. A framework to assess the impact of open education initiatives for RRI with quantitative indicators to promote and monitor the impact/influence of RRI and qualitative indicators for societal prosperity.
    5. Participants’ best practices, lessons learned, and outcomes related to findings of their teaching inquiry CPD practice.

    The key limitation of this work is that to assess the long-term impact, more studies will be necessary to analyse data for a longer period, particularly to examine society and prosperity. This is our aim with our next project http://www.RRIdata.com.

    Further studies will be important to examine new issues, such as how can partnerships be promoted between schools, local communities, civil society organisations, universities and industry to foster a more scientifically interested and literate society? In what ways could technology be used more effectively by communities to develop inquiry skills for RRI? What is the correlation between students who increased their inquiry skills for RRI and their interest in pursuing a career in science? What are the effective teachers’ pedagogical approaches to equip the next generation for digital transformation based on RRI? How can open education empower disadvantaged students and low achievers to develop inquiry skills for RRI? In what ways can policy change support scientific literacy to improve social and business prosperity including sustainability?

    Appendix. Interview questions

    Introduction

    This interview focuses on the impact of Open Education promoted by the ENGAGE (open materials, open courses, open communities, open tools and open schooling projects) for preparing teachers and students to engage with RRI – Responsible Research and Innovation, especially science with and for society.

    1. Open education outcomes
      What were the outcomes of your open schooling project? In what ways, if any, did open education enhance teachers’ professional development, and students’ scientific skills?
    2. Influences on engagement
      In what ways, if any, did your open schooling promote participatory research? Was there any example of public engagement? Was there any gender or ethical issue?
    3. Effects on prosperity
      To what extent might your open schooling project have or had an effect on social prosperity or business prosperity?
    4. Policy or Institutional change
      Was there any policy or institutional change? If any, in what ways?

    Acknowledgements

    This work supported by the COLEARN research network has received funding from the European Commission FP7/2007-2013, grant agreement No. [612269].

    Competing Interests

    The authors have no competing interests to declare.

    References

    1. Aikenhead, GS. 2006. Science education for everyday life: Evidence-based practice. New York, Teachers College Press. 

    2. Alcaraz-Domínguez, S, Barajas Frutos, M, Malagrida, R and Pérez, F. 2015. ‘Els projectes Europeus Engaging Science, Xplore Health, RRI Tools i Scientix’. Ciències: revista del professorat de ciències de Primària i Secundària, 30: 47–54. 

    3. American Association for the Advancement of Science (AAAS). 1993. Benchmarks for Science Literacy. New York, Oxford University Press. 

    4. Bardone, E, Burget, M, Saage, K and Taaler, M. 2017. ‘Making Sense of Responsible Research and Innovation in Science Education through Inquiry-based Learning: Examples from the Field’. Science Education International, 28(4): 293–304. 

    5. Bayram-Jacobs, D. 2015. ‘Responsible Research and Innovation: What is it? How to Integrate in Science Education’. Presented at International Congress on Education for the Future: Issues and Challenges (ICEFIC 2015) Conference. Ankara University, Turkey, 13–15 May. 

    6. Bayram-Jacobs, D, Henze, I, Evagorou, M, Shwartz, Y, Aschim, E, Alcaraz-Domínguez, S, Dagan, E and Barajas, M. 2017. ‘Exploring the impact of educative materials on teachers’ pedagogical content knowledge’. ESERA Conference. Dublin City University, Dublin, 21–25 August [Online]. Available at: https://keynote.conference-services.net/resources/444/5233/pdf/ESERA2017_0408_paper.pdf (Accessed 9 November 2018). 

    7. Blonder, R, Rap, S, Zemler, E and Rosenfeld, S. 2017. ‘Assessing Attitudes about Responsible Research and Innovation (RRI): The Development and Use of a Questionnaire’. Sisyphus-Journal of Education, 5(3): 122–156. 

    8. Blonder, R, Zemler, E and Rosenfeld, S. 2016. ‘The story of lead: A context for learning about responsible research and innovation (RRI) in the chemistry classroom’. Chemistry Education Research and Practice, 17(4): 1145–1155. DOI: https://doi.org/10.1039/C6RP00177G 

    9. Bruner, JS. 1960. The Process of education. Cambridge, MA, Harvard University Press. 

    10. Burget, M, Bardone, E and Pedaste, M. 2017. ‘Definitions and conceptual dimensions of Responsible Research and Innovation: A literature review’. Science and Engineering Ethics, 23(1): 1–19. DOI: https://doi.org/10.1007/s11948-016-9782-1 

    11. Clarke, D and Hollingsworth, H. 2002. ‘Elaborating a model of teacher professional growth’. Teaching and Teacher Education, 18(8): 947–967. DOI: https://doi.org/10.1016/S0742-051X(02)00053-7 

    12. David, PA. 2007. The Historical Origins of ‘Open Science’. Discussion paper, Stanford, CA, Stanford Institute for Economic Policy Research [Online]. Available at: http://www-siepr.stanford.edu/papers/pdf/06-38.pdf. (Accessed 9 November 2018). 

    13. Davis, EA and Krajcik, JS. 2005. ‘Designing educative curriculum materials to promote teacher learning’. Educational Researcher, 34(3): 3–14. DOI: https://doi.org/10.3102/0013189X034003003 

    14. de Vocht, M, Laherto, A and Parchmann, I. 2017. ‘Exploring teachers’ concerns about bringing Responsible Research and Innovation to European science classrooms’. Journal of Science Teacher Education, 28(4): 326–346. DOI: https://doi.org/10.1080/1046560X.2017.1343602 

    15. DuFour, R and Eaker, R. 1998. Professional Learning Communities at Work. Bloomington, National Educational Service. 

    16. Dwyer, D, Ringstaff, C and Sandholtz, J. 1991. ‘Changes in teachers’ beliefs and practices in technology-rich classrooms’. Educational Leadership, 48(8): 45–52. 

    17. European Commission (EC). 2012. Responsible Research and Innovation: Europe’s ability to respond to societal challenges [Online]. Available at: http://ec.europa.eu/research/science-society/document_library/pdf_06/responsibleresearch-and-innovation-leaflet_en.pdf (Accessed 9 April 2018). 

    18. European Commission (EC). 2013. Options for Strengthening Responsible Research and Innovation – Report of the Expert Group on the State of Art in Europe on Responsible Research and Innovation [Online]. Available at: https://ec.europa.eu/research/science-society/document_library/pdf_06/options-for-strengthening_en.pdf (Accessed 24 June 2018). 

    19. European Commission (EC). 2015. Indicators for promoting and monitoring Responsible Research and Innovation [Online]. Available at: http://ec.europa.eu/research/swafs/pdf/pub_rri/rri_indicators_final_version.pdf (Accessed 24 June 2018). 

    20. Fullick, P and Ratcliffe, M. 1996. Teaching Ethical Aspects of Science. Southampton, The Bassett Press. 

    21. Gorghiu, G, Anghel, GA and Ion, RM. 2015. ‘Students’ Perception Related to a Responsible Research and Innovation Demarche’. Procedia-Social and Behavioral Sciences, 180: 600–605. DOI: https://doi.org/10.1016/j.sbspro.2015.02.166 

    22. Gorghiu, LM, Dumitrescu, C and Petrescu, AMA. 2016. ‘Introducing RRI in Science Teaching – An Actual Challenge for Science Teachers’. European Proceedings of Social and Behavioural Sciences, 18: 183–191. DOI: https://doi.org/10.15405/epsbs.2016.12.25 

    23. Gott, R and Duggan, S. 1996. ‘Practical work: Its role in the understanding of evidence in science’. International Journal of Science Education, 18(7): 791–806. DOI: https://doi.org/10.1080/0950069960180705 

    24. Guskey, TR and Huberman, M. 1995. Professional Development in Education: New Paradigms & Practices. New York, Teacher’s College Press. 

    25. Hoban, GF. 2002. Teacher Learning for Educational Change. Buckingham, Open University Press. 

    26. Hodson, D. 2011. Looking to the Future. Rotterdam, Sense Publishers. DOI: https://doi.org/10.1007/978-94-6091-472-0 

    27. Hodson, D. 2014. ‘Becoming part of the solution: Learning about activism, learning through activism, learning from activism’. In: Bencze, L and Alsop, S (Eds.), Activist Science and Technology Education, 67–98. Netherlands, Springer. DOI: https://doi.org/10.1007/978-94-007-4360-1_5 

    28. Jarman, R and McClune, B. 2007. Developing Scientific Literacy: Using News Media In The Classroom. Maidenhead, London: Open University Press. 

    29. Kiki-Papadakis, K and Chaimala, F. 2016. ‘The Embedment of Responsible Research and Innovation Aspects in European Science Curricula’. Romanian Journal for Multidimensional Education/Revista Romaneasca Pentru Educatie Multidimensionala, 8(2): 71–87. DOI: https://doi.org/10.18662/rrem/2016.0802.06 

    30. Kop, R, Fournier, H and Mak, JSF. 2011. ‘A pedagogy of abundance or a pedagogy to support human beings? Participant support on massive open online courses’. The International Review of Research in Open and Distributed Learning, 12(7): 74–93. DOI: https://doi.org/10.19173/irrodl.v12i7.1041 

    31. Lave, J and Wenger, E. 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge, Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511815355 

    32. Loucks-Horsley, S, Hewson, PW, Love, N and Stiles, KE. 1998. Designing Professional Development for Teachers of Science and Mathematics. Thousand Oaks, CA, Corwin Press. 

    33. Mikroyannidis, A, Okada, A, Correa, A and Scott, P. 2016. ‘Inquiry-Based Learning on the Cloud’. In: Chao, L (Eds.), Handbook of Research on Cloud-Based STEM Education for Improved Learning Outcomes, 291–310. Hershey PA, USA, Information Science Reference. DOI: https://doi.org/10.4018/978-1-4666-9924-3.ch019 

    34. NGSS Lead States. 2013. Next Generation Science Standards: For States, by States. Washington, DC, The National Academies Press. 

    35. Okada, A. 2014. ‘Scaffolding school students’ scientific argumentation in inquiry-based learning with evidence maps’. In: Okada, A, Buckingham, S, Simon, J and Sherborne, T (Eds.), Knowledge Cartography, 135–172. London, Springer. 

    36. Okada, A. 2016a. Engaging Science: Innovative Teaching for Responsible Citizenship. Milton Keynes, The Open University – UK. 

    37. Okada, A. 2016b. Responsible research and innovation in science education report. Milton Keynes, The Open University – UK. 

    38. Okada, A and Bayram-Jacobs, D. 2016. ‘Opportunities and challenges for equipping the next generation for responsible citizenship through the ENGAGE HUB’. Research papers presented at the 2016 LSME International Conference on Responsible Research in Education and Management and its Impact, 42–57. London 13th–15th January 2016. 

    39. Okada, A, Costa, A, Kowalski, R, Torres, P, Nakayama, M and Souza, K. 2016. ‘Open Educational Resources for Responsible Research and Innovation: a case study with Brazilian universities and schools’. International Conference on Responsible Research in Education and Management and its Impact, 42–57. London 13th–15th January 2016. 

    40. Okada, A, Kowalski, RPG, Kirner, C and Torres, PL. 2018. ‘Factors influencing teachers’ adoption of AR inquiry games to foster skills for Responsible Research and Innovation’. Interactive Learning Environments, 26(5). Early access. DOI: https://doi.org/10.1080/10494820.2018.1473257 

    41. Okada, A, Rossi, LC and Costa, A. 2015a. ‘Online argumentative maps for facilitating international debates with experts at large scale’. EC-TEL The 10th European Conference on Technology Enhanced Learning. Toledo, 15–18 September 2015. 

    42. Okada, A, Wolff, A, Mikroyannidis, A and Ashton, S. 2015b. ‘Promoting partnerships among Universities, Schools and Research Centres to foster Responsible Research and Innovation for smart citizenship’. Proceedings of the 15th International Conference on Technology, Policy and Innovation, Milton Keynes, 17–19 June 2015. 

    43. Okada, A, Young, G and Sanders, J. 2015d. ‘Fostering Communities of Practices for teachers’ professional development integrating OER and MOOC’. EC-TEL The 10th European Conference on Technology Enhanced Learning, Toledo, 15–18 September 2015. 

    44. Okada, A, Young, G and Sherborne, T. 2015c. ‘Innovative teaching of responsible research and innovation in science education’. E-Leaning Papers, Open Education Europa Journal, 44(1): 180–195. 

    45. Open University (OU-UK). 2018. Outstanding Impact of Research on Society and Prosperity – Criteria. The Open University Research Excellence Awards 2018. 

    46. Osborne, J. 2010. ‘Arguing to learn in science: The role of collaborative, critical discourse’. Science, 328(5977): 463–466. DOI: https://doi.org/10.1126/science.1183944 

    47. Owen, R, Macnaghten, P and Stilgoe, J. 2012. ‘Responsible research and innovation: From science in society to science for society, with society’. Science and Public Policy, 39(6): 751–760. DOI: https://doi.org/10.1093/scipol/scs093 

    48. Owen, R, Stilgoe, J, Macnaghten, P, Gorman, M, Fisher, E and Guston, D. 2013. ‘A framework for responsible innovation’. In: Owen, R, Bessant, J and Heintz, M (Eds.), Responsible Innovation: Managing the Responsible Emergence of Science and Innovation in Society, 27–50. John Wiley & Sons. DOI: https://doi.org/10.1002/9781118551424.ch2 

    49. Peciuliauskiene, P. 2017. ‘The engagement of secondary school learners for learning science by responsible research and innovation’. Proceedings of the International Scientific Conference, II: 151–162. 26–27 May. DOI: https://doi.org/10.17770/sie2017vol2.2249 

    50. Petrescu, AM, Gorghiu, G and Lupu, RA. 2015. ‘Non-formal Education-Frame for Responsible Research and Innovation Demarches’. Procedia-Social and Behavioral Sciences, 180: 682–687. DOI: https://doi.org/10.1016/j.sbspro.2015.02.178 

    51. Pinto, SM, Ribeiro, SF, Rocha, AKLT and Okada, A. 2018. ‘Argumentação de estudantes da educação básica sobre dilemas sócio-científicos no Projeto ENGAGE’. Revista Ibero-Americana de Estudos em Educação, 13(1): 207–228. DOI: https://doi.org/10.21723/riaee.v13.n1.2018.10242 

    52. Rocha, AK, Rocha, AB and Okada, A. 2017. ‘Rubric to assess evidence-based dialogue of socio-scientific issues with LiteMap’. The 2017 International Technology Enhanced Assessment Conference, 5–6 October 2017. Barcelona. 

    53. Rogers, EM. 2003. Diffusion of innovations, 5th edn. New York, NY, Free Press. 

    54. Ryan, C. 2015. Science Education for Responsible Citizenship [Online]. Available at: http://ec.europa.eu/research/swafs/pdf/pub_science_education/KI-NA-26-893-EN-N.pdf (Accessed 9 November 2018). 

    55. Sherborne, T and Okada, A. 2013. ‘Equipping the next generation for Research and Innovation’. Scientix Conference 2013, 14–16 March 2013. Lisbon, Portugal. 

    56. Supovitz, JA and Turner, HM. 2000. ‘The effects of professional development on science teaching practice in classroom culture’. Journal of Research in Science Teaching, 37(9): 963–980. DOI: https://doi.org/10.1002/1098-2736(200011)37:9%3C963::AID-TEA6%3E3.0.CO;2-0 

    57. Sutcliffe, H. 2011. A report on Responsible Research & Innovation. MATTER and the European Commission. 

    58. Torres, PL, Fialho, NN, Kowalski, RPG and Okada, A. 2016. ‘Responsible Research and Innovation for the Media Facebook: Community Involvement in the Study on Agrobiodiversity’. Creative Education, 7(15): 2141–2150. DOI: https://doi.org/10.4236/ce.2016.715212 

    59. Von Schomberg, R. 2013. ‘A vision of responsible research and innovation’. In: Owen, R, Bessant, J and Heintz, M (Eds.), Responsible Innovation: Managing the Responsible Emergence of Science and Innovation in Society, 51–74. London, Wiley. DOI: https://doi.org/10.1002/9781118551424.ch3 

    60. Wickson, F and Carew, AL. 2014. ‘Quality criteria and indicators for responsible research and innovation: Learning from transdisciplinarity’. Journal of Responsible Innovation, 1(3): 254–273. DOI: https://doi.org/10.1080/23299460.2014.963004 

    61. Wideman, H. 2010. Online Teacher Learning Communities: A Literature Review. Institute for Research on Learning Technologies, Technical Report 2. 

    Jump to Discussions Related content
    comments powered by Disqus