Innovations from science and technology are vital to Europe’s future (Hodson, 2011, 2014). To ensure that the process and products of research are acceptable to society, these endeavours now fall under the framework of Responsible Research and Innovation (RRI). RRI sets out six keys for how all societal actors should work together: Science Education, Public Engagement, Open Access, Ethics, Gender and Governance (EC, 2012, 2013). For citizens to participate in the processes of RRI, they will need to be sufficiently literate about how science works and understand, among other things, the benefits and risks of technology, and ethical thinking, in order to participate in debates and make informed choices (Owen et al., 2012; Ryan, 2015).
The international initiatives in science education which are emerging this decade pay particular attention to fostering new skills and knowledge for teachers, centred on the use of Inquiry-Based Science Education (IBSE) for RRI (Okada et al., 2015; Alcaraz-Domínguez et al., 2015; Bayram-Jacobs, 2015; Mikroyannidis et al., 2016). This approach to teaching science has major potentials to raise students’ interest in science, to increase knowledge of how science works, and to develop competencies such as critical and creative thinking, vital for understanding socio-scientific issues. Science teachers typically focus on delivering the canon of scientific knowledge, transferring knowledge. RRI-based teaching, by contrast, focuses on how we know what we know, i.e. the nature of science, and the effects of that knowledge, that is its social impact. Such approach requires the adoption of different pedagogies such as inquiry (Okada, 2014; Blonder et al., 2016; Bardone et al., 2017; Gorghiu et al., 2016). Inquiry-based teaching aims at developing the skills of scientific thinking, so that learners will be able to interpret evidence, weigh up technologies, make informed judgements, and argue their views (AAAS, 1993). The literature demonstrates a poor record of success for attempts to build an RRI teaching force, and transformations generally occur with only small numbers (Gorghiu et al., 2015; Kiki-Papadakis and Chaimala, 2016; Okada et al., 2016; Bayram-Jacobs et al., 2017; Peciuliauskiene, 2017; Blonder et al., 2017).
To explore this gap, this study focuses on the project ENGAGE – equipping the next generation of teachers and students for RRI through a novel model that combined Open Educational Resources (OER), Massive Open Online Courses (MOOC), Communities of Practice (CoP) and Open Schooling to bridge informal, non-formal with formal learning for RRI. This approach was implemented during three years from 2014 to 2017 and linked with the national curriculum of 11 countries mostly in Europe: UK, France, Germany, Spain, Switzerland, Norway, Greece, Cyprus, Romania, Lithuania and Israel. The purpose of ENGAGE was to support the EU’s ambition by shifting the practice of science teachers on a massive scale by engaging 12,000 teachers (approximately 1,090 teachers per country) to foster scientific knowledge and inquiry skills for RRI.
The key to engaging the next generation to be aware of and participate in socio-scientific issues is to change how science is taught. Traditionally, students gain an image of science as a body of content. Teaching about RRI (Burget et al., 2017; de Vocht et al., 2017) however, focuses more on areas of emerging science and technology whose applications and implications are uncertain, and where values and debate are as important as established facts (Owen et al., 2012; Von Schomberg, 2013). The shift is hugely challenging. High stakes education systems marginalise teaching about the nature of science. The greater challenge is to help teachers develop the beliefs, knowledge and classroom practice for RRI teaching in formal education (Kiki-Papadakis and Chaimala, 2016) and non-formal learning (Petrescu et al., 2015). This requires adopting a more inquiry-based methodology, which gives students opportunities for self-expression and responsibility for coming to informed decisions.
While primarily a teacher-based intervention, ENGAGE targets all three components of student scientific literacy – motivation (care), knowledge (know), and skills and attitude (do) – for preparing students to engage with issues around emerging technologies (Okada, 2016a). ENGAGE shows how learning science concepts can be set within the context of its implications to society.
Our curriculum materials invited students on a journey to the future. Showing possible future scenarios means making the pros and cons of technology more concrete; and help all students to think through the current and emerging issues in more depth. Making science more relevant to students’ concerns – which are known to be future orientated – will increase the likelihood that they can apply what they have learned outside school and respond to societal challenges (Sherborne and Okada, 2013).
ENGAGE has specified its ‘RRI curriculum’ (Figure 1), synthesised content from the European Commission about RRI (Owen et al., 2013) and US Next Generation Science Standards curriculum framework (NGSS Lead States, 2013) based on key components:
This set of skills was designed to maximise understanding of the nature of science and decision making, particularly through collaborative inquiry-based learning projects. Yet the shift towards RRI-based or ‘humanistic’ science teaching (Aikenhead, 2006), and inquiry-based pedagogy (Bardone et al., 2017; Okada and Bayram-Jacobs, 2016) is challenging.
Aikenhead (2006) underscores the lack of success of continuing professional development (CPD) programmes to transform teachers with a list of reasons for failure. The notion that any event or short-term set of workshops produces lasting change has been discredited. Real transformation such as the one required to move to RRI-based teaching is a long term and rather complex process. The ENGAGE programme synthesised contemporary models of CPD with the curriculum development expertise into a framework for supporting teachers through this process of transformation, which combined two models (Sherborne and Okada, 2013).
The first model is a continuing professional development based on three stages, adopt-adapt-transform (Figure 3), to represent an RRI learning and teaching pathway integrated to teacher’s CPD (Dwyer et al., 1991; Rogers, 2003).
The implications of the model influenced the design of ENGAGE:
We use a second model (Figure 4), which represents these conditions and relationships as an ‘inquiry cycle of learning’ (Guskey and Huberman, 1995). It involves experimentation, feedback, reflection and input of new ideas. The teacher inquiry cycle is shown in the diagram below.
There are major advantages of structuring ENGAGE’s CPD as a teacher inquiry:
ENGAGE targeted each part of the cycle, with these key strategies (project outputs):
This study focuses on the impact of the ENGAGE open education for RRI beyond Europe. The meaning of impact refers to the influence and effect of an Open Education project to promote RRI on academic and non-academic groups. This work examines the integrated model that combines OER, MOOC, CoP and Open Schooling to foster inquiry skills for RRI in Brazil. With this aim in mind, we investigate the following research questions:
Two key references were used to design a framework (Figure 5) on impact assessment of OER for RRI. The first document provided some quantitative ‘indicators for promoting and monitoring RRI’ (EC, 2015) grouped in six key components: science education, public engagement, open access, gender equality, ethics and governance.
The second document provided some qualitative ‘indicators for assessing impact on society and prosperity’ (OU-UK, 2018) which were grouped also in six key components: professional development, participatory research, policy change, social prosperity, business prosperity and sustainability.
Quantitative and qualitative data were produced by the communities from the ENGAGE portal (WordPress), OER (SlideShare), MOOC (Open edX) questionnaires, interviews and webinars (Hangouts), video library (YouTube), collective dialogue maps (LiteMap), and social media (WhatsApp, Facebook and Twitter) during three years of the project. Participants signed the consent form and became aware of ethical procedures including data protection and privacy, as well as transparency and openness of research procedures, key issues for open science (David, 2007) and RRI (EC, 2012). Anonymised data were available for Brazilian communities who developed their own studies, supported by the European ENGAGE team. Six Brazilian groups who authored the peer-reviewed articles were also interviewed (Table 3).
At the end of three years of the European project ENGAGE, there were 17,120 members in the Portal including 2,179 members from Brazil from schools, universities, learning-centres, educational and technology enterprises, local communities, and members from local councils. Figure 6 shows the geographical distribution of active participants, reasonably spread throughout Brazil, with a large concentration of groups in six states: Ceará and Bahia in the North East, São Paulo and Rio de Janeiro in the South East, and Paraná and Santa Catarina in the South.
To answer the first research question, Table 1 provides an overview of the indicators for analysing the influence of ENGAGE to promote RRI among academic and non-academic groups in Brazil.
Open Education strategies
|Science Education||RRI workshops provided||33||3||5|
|RRI Resources (OER) created||330||30||42|
|RRI MOOC (courses) delivered||22||2||1|
|Public Engagement||Academic members (universities)||12||1||5|
|Non-academic (school & local communities)||660||60||319|
|Teaching staff members||17,120||1,100||2,179|
|Open Access||Open presentations in conferences||32||3||5|
|Open peer-reviewed articles||10||1||6|
|Open schooling projects||27||2||2|
|Equality Gender||Women coordinators||15||2||4|
|Total of partners (women + men)||25||2||6|
|Ethical issues raised||9||1||6|
|Governance||RRI Institutional debates (events)||12||1||9|
|Policies and Institutional changes||3||-||2|
The ENGAGE European consortium, which included 25 partners (15 women and 10 men) from 14 institutions (12 universities, 1 science centre and 1 technology company), developed the ENGAGE Portal with three RRI workshops, 330 OER, 30 MOOC and 42 open schooling projects materials in ten languages, to be implemented widely in and beyond Europe. Through the implementation of professional development within partners’ universities and projects with local groups, ENGAGE reached approximately 660 non-academic communities (public and private schools, centres, clubs, hospitals, museums and libraries). Various studies were published, such as ten peer-reviewed articles, 32 conference presentations and papers including at the international conference on Public Communication of Science and Technology in 2015 in Brazil. The ENGAGE portal and online courses were widely disseminated among the partners’ communities in their institutions and social networks, which included Brazilians. The ENGAGE portal presented different types of Open Education strategies in all six RRI dimensions (Table 1), generating various debates around the OER. In terms of ethical discussions there were 38 topics and nine issues.
At the end of the project, each of the 11 countries organised a national conference to create opportunities for RRI debates and consolidate their ENGAGE RRI communities. There were two groups of Brazilian members who attended ENGAGE conferences in the UK and there were four events organised in Brazil that promoted ENGAGE open education widely in various states, particularly in Bahia, Ceará, Paraná and Santa Catarina.
The impact in Brazil was significantly representative (2,179 members) compared to the average target expected per country of the consortium partners who were responsible for coordination and support actions in their country (average target per country was 1,100).
Brazilian members produced a total of 42 OER (in various formats), they used the platform content to support five workshops and one course unit, which was part of the ENGAGE MOOC. They also adapted two existing ENGAGE OER (GM decision and Exterminate) to the Brazilian curriculum in Portuguese. They developed three studies about ENGAGE focusing on scientific skills for RRI in Brazil. Two large open schooling projects were created about ‘Genetic Modified Food’ and ‘Zika’ which are described in the next section (case studies). In addition, they contributed to the ENGAGE open publication with five conference papers and six peer-reviewed articles. The Brazilian community increased in 2017, especially in the LiteMap Platform (Okada et al., 2015a) with various dialogues about ethical issues (6), ethical discussions (12), global RRI debates (9), and RRI communities in states (5). Policies and institutional changes were promoted to embed OER for RRI in the Brazilian curriculum (Figures 7 and 8).
To answer the second research question, Table 2 presents the number of participants in Brazil grouped by the type of engagement for RRI teaching (adopters, adapters and transformers) distributed respectively by ENGAGE Open Education technologies and strategies (OER, MOOC and Communities).
|Open Education strategies||Interactive Media environments||Adopters: participated in ENGAGE||Adapters: used OER and shared changes in practices||Transformers: developed open schooling projects with students and articles|
In terms of ENGAGE OER (WordPress), there were 2,179 participants who accessed OER topical lessons, but only (26% = 567) commented OER sequences and (3% = 75) reported outcomes with open schooling projects. Most of these participants (78% = 2,704) accessed OER slides in SlideShare, but a few accessed videos in YouTube (4% = 78).
In terms of ENGAGE MOOC, delivered in an English platform (Open edX), there were very few registrations (8% = 180). The group who completed the course (1% = 13) translated and adapted the open content to support their teachers using their own platforms.
In terms of ENGAGE communities (CoP), there were 1,346 participants in LiteMap, 1,550 participants in Facebook and 520 participants in WhatsApp at the beginning (Adopt stage). However, the number of participants in all CoP platforms were reduced at the end of the project during open schooling projects: 345 participants in Facebook, 180 participants in WhatsApp and 46 participants in LiteMap.
LiteMap was one of the technologies suggested in the MOOC, which was translated to Portuguese by a member of ENGAGE Brazil. This platform was recommended by ENGAGE facilitators to help teachers and students discuss RRI and use evidence-based dialogue Maps for developing informed opinions and making evidence-based decisions.
Figure 7 shows the registration of Brazilian participants during four years from March 2014 to November 2017 with a significant increase during December 2016 to November 2017.
Figure 8 presents the number of open content produced by the community in terms of maps, issues, ideas, arguments, counter arguments notes and chats. This graph shows an increased number of content produced in the same period (December 2016 to November 2017) when there were various events organised by the Brazilian community and new OER materials, including RRI reports, published.
To answer the third question about the ways that ENGAGE research enabled or supported prosperity, two case studies about open schooling projects were selected based on the journal articles produced by Brazilian members (Tables 3 and 4).
|Co-author Interviewed||Journal Article||Open Schooling Project (question)||Open Schooling Project (findings)||Impact reported by Interviewees (summary translated)|
|Brazilian Postgraduate Coordinator||Responsible Research and Innovation for the Media Facebook: Community Involvement in the Study on Agrobiodiversity. Creative Education.||To what extent are OER used to explore the issues related to agricultural biodiversity, specifically to GM products in order to promote RRI?||OER were successfully adopted by 54 teachers through a collaborative process of teaching and learning with social networks, which engaged 340 students who participated in the GM discussion from Brazil, including learners from other countries Portugal, Ecuador, Spain, Luxembourg, UK, USA.||Participatory Research using OER and social media enabled the co-creation of new OER that were widely disseminated: images, games, video clips, articles presentations and interviews. The variety of OER enabled teachers to consider gender preferences. The combination of OER and social media promoted students and public engagement.|
|Brazilian Postgraduate Coordinator||Experience of Environmental Education using Responsible Research and Innovation of the Pontifical Catholic University of Paraná in the European Project Engage. Diálogo Educacional.||In what ways are OER linked to formal education to foster RRI skills?||RRI skills were fostered through a variety of OER that were embedded in the curriculum such as video clips, interviews, magazines, games and maps. These OER were used and co-produced by more than 583 students supported by 19 teachers from secondary school and 11 lecturers from higher education.||OER were embedded in the curriculum by various teachers and lecturers from schools and university to foster skills for RRI through more learner-centred approaches. The positive outcomes from open schooling projects are useful to support policy and institutional changes.|
|Brazilian PhD student||Factors influencing teachers’ adoption of AR inquiry games to foster skills for Responsible Research and Innovation. Interactive Learning Environment.||What are teachers’ views about a novel inquiry game, which is an OER created with an Augmented Reality (AR) open platform to foster inquiry skills for RRI?||The AR inquiry game about GM developed by ENGAGE members from Brazil was considered easy-to-use and useful by 18 teachers from Brazil who used the game with their students (390). They found it meaningful to help students practise key skills: devise questions, estimate risks and communicate ideas.||Open courses and guidelines for teachers’ professional development facilitated the adoption of AR inquiry game to foster skills for RRI. The game and pedagogical strategies discussed during the course were useful for teachers to work with skills that they were not used to; such as estimate risks, examine consequences and use ethics.|
|Co-author Interviewed||Journal Article||Open Schooling Project (question)||Open Schooling Project (findings)||Impact reported by Interviewees (summary translated)|
|Head of CPD (teachers’ professional development)||Continuing teacher training using dilemmas with elements of ubiquity. Interfaces Científicas.||To what extent teacher’s continuing pedagogical practices can be supported through mobile devices and elements of ubiquity, articulated with face-to-face (F2F) activities in the real classroom environment?||Thirty-two teachers and four facilitators used WhatsApp and Google Hangouts. The materials and slides from the Open edX course about RRI were translated and made available through PDF for teachers to access on their mobile devices. Questions and answers were shared just-in-time during their lessons through Apps.||Teachers’ professional development was enhanced by the use of mobile devices integrated to their workplace. They became more confident to use OER to foster inquiry skills for RRI. Students also used their mobile devices during their inquiry projects. They found that digital and scientific skills were vital to succeed in university, professional careers and for sustainable economic growth.|
|Brazilian Postdoctoral Researcher||Argumentation of basic education students about socio-scientific dilemmas in the engage project. Ibero-Americana de Estudos em Educação.||To what extent students use evidence-based argumentation to justify opinions about socio-scientific dilemmas?||All the elements of the argumentation were identified during the activities carried out by groups, however, it was observed that students had difficulties to elaborate a justification based on evidence and scientific thinking.||The open schooling project enabled teachers to be aware of students’ strengths and difficulties in terms of scientific argumentation.
Secondary school students who took part in this research had opportunity to practise argumentation about socio-scientific issues that are relevant for their community. The combination of relevant socio-scientific issue for the Irecê community and open schooling project to develop inquiry skills for RRI were very meaningful to reflect on social prosperity.
|Rubric to assess evidence-based dialogue of socio-scientific issues with LiteMap. Technology Enhanced Assessment.||In what ways LiteMap application tool can be used by teachers to annotate students’ socio-scientific discussion and assess their evidence-based dialogue?||The discussion about Zika was mapped with LiteMap to support evidence-based decisions. Participants used icons to annotate questions, views, pros, cons, and evidence. Some graphs were used to visualise argumentation. This study focused on open schooling project developed by 24 teachers and 478 students from a public professional school in Irecê, including also 5 collaborators and 2 researchers.||The open schooling project created opportunity to discuss science with and for society. This increased participants’ awareness of the importance of science projects to promote Sustainability, Social and Business prosperity. Both communities of academics and non-academics were engaged to develop their views about the various possibilities to reduce Aedes mosquitoes and Zika: through homemade repellents, Oxitec GM-mosquitoes and a natural solution mesocyclops that eat Aedes larvae.|
Three articles focused on GM decisions in Curitiba Paraná, and three articles focused on Zika in Irecê Bahia. The first case highlights prosperity in terms of professional development, participatory research and local policy change. Whereas the second study describes an increased awareness for relevant issues for social prosperity, business prosperity and sustainability. To examine the impact of open education to foster RRI in-depth, the authors of these studies were also interviewed.
The interviewees from Paraná described that various Brazilian communities in secondary school and higher education implemented the open schooling project on GM food and agrobiodiversity. GM decision was a very popular theme in this Brazilian state whose economy is based on agronomy and food engineering.
The ENGAGE OER GM decision (8,600 views, 1,746 downloads) offered a set of editable resources: presentation (PPTX), guidelines (DOCX) and video clips (MP4) and a game to support teachers to discuss the risks of GM corn and GM corn treated with pesticides. To facilitate open schooling projects, ENGAGE courses promoted the CARE – KNOW – DO model (Okada et al., 2018) for Brazilian teachers to engage students in:
During the open schooling project students developed their knowledge of inheritance using arguments for and against genetic modification, and weighing up the benefits and risks of an application of science to make a decision. They were also supported to develop seven inquiry skills for RRI:
Some of the key outcomes of open schooling projects in Paraná developed by students supported by teachers, experts and local community were:
The interviewees from Bahia described that the ‘Exterminate’ OER about Zika was the most used resource, which attracted a large community of educators and lecturers (2,230 downloads, 3,317 views). The whole set of ENGAGE activities about Zika – Exterminate was used by a public vocational school in Irecê, a municipality in Bahia largely affected by Aedes aegypti Mosquito and various epidemic diseases such as dengue, Zika and chikungunya. Thirty-two members of teaching staff developed open schooling projects with 478 students in all technical courses: agricultural administration, clinical analysis, commerce, nursing, environment, nutrition, advertising and occupational safety. Students were from 18 to 22 years old, 60.2% were female and 39.8% male.
The open schooling project was developed in three phases:
Some of the key outcomes of open schooling projects in Irecê developed by students, supported by teachers, experts and local community were:
This study revealed that ENGAGE have influenced academic and non-academic groups by increasing awareness on scientific skills which are relevant in RRI. Yet, it is not possible to claim that the participants who were engaged with ENGAGE are more aware of the RRI concept (Sutcliffe, 2011; Von Schomberg, 2013) as this approach is very new in Brazil.
A small percentage of teaching staff (teachers, lecturers, PhD students and course coordinators) attended the ENGAGE MOOC. This group became very committed to learning about RRI and developing open schooling projects as it is a novel area, very relevant for contemporary education. Participants who completed open schooling projects presented evidence of pedagogical changes (Torres et al., 2016; Pinto et al., 2018). The studies developed and published by Brazilian ‘transformer’ teachers, including academic and non-academic members, suggest that key findings are vital for evidence-based impact for society and prosperity.
The partnership among academics, researchers, scientists, lecturers and society including schools, students and families are vital for opening up more opportunities for open education (Okada et al., 2015b). This can also enhance teachers’ professional development (Supovitz and Turner, 2000) to foster skills for RRI (Wickson and Carew, 2014) by bridging formal, informal and non-formal learning through open schooling projects (Ryan, 2015).
This study also revealed the role of technologies for dissemination and reaching communities even with remote access. New digital platforms for collaborations between teachers, students, families and universities were used to examine potential solutions for difficulties that emerged during the process. For instance, LiteMap was applied by a group to identify, connect, systematise and evaluate the key components of argumentation to foster scientific thinking in oral and written discourses, with examples that were freely and widely disseminated by participants (Rocha et al., 2017). The AR inquiry game produced by Brazilians enabled teachers to practise and reflect on more learners-centred approaches (Okada, 2016b; Okada et al., 2015c, 2016, 2018).
The combination of various ENGAGE strategies such as OER, MOOC, open schooling projects and open communities using interactive technologies (Ribeiro et al., 2017; Okada et al., 2015d) made it possible to reach large numbers in the ENGAGE project during three years in Brazil. New studies will be necessary to examine the impact of ENGAGE in the long term.
This study also revealed the importance of engaging and empowering the community of teachers with lecturers and also students to reflect about their practices, achievements and lessons learned in a scientific way. That means by written scientific discourse. The papers published by the community are vital for evidence-based pedagogical changes supported by knowledge, skills and attitude for RRI that involves all members of society in research and innovation.
This study examined how the ENGAGE project has influenced communities in Brazil. Participants were involved through various activities at their own pace with the aim of preparing students to use their knowledge and skills for evidence-based dialogue for making informed decisions.
This study presented some novel approaches, which combine:
The key limitation of this work is that to assess the long-term impact, more studies will be necessary to analyse data for a longer period, particularly to examine society and prosperity. This is our aim with our next project http://www.RRIdata.com.
Further studies will be important to examine new issues, such as how can partnerships be promoted between schools, local communities, civil society organisations, universities and industry to foster a more scientifically interested and literate society? In what ways could technology be used more effectively by communities to develop inquiry skills for RRI? What is the correlation between students who increased their inquiry skills for RRI and their interest in pursuing a career in science? What are the effective teachers’ pedagogical approaches to equip the next generation for digital transformation based on RRI? How can open education empower disadvantaged students and low achievers to develop inquiry skills for RRI? In what ways can policy change support scientific literacy to improve social and business prosperity including sustainability?
This interview focuses on the impact of Open Education promoted by the ENGAGE (open materials, open courses, open communities, open tools and open schooling projects) for preparing teachers and students to engage with RRI – Responsible Research and Innovation, especially science with and for society.
This work supported by the COLEARN research network has received funding from the European Commission FP7/2007-2013, grant agreement No. .
The authors have no competing interests to declare.
Alcaraz-Domínguez, S, Barajas Frutos, M, Malagrida, R and Pérez, F. 2015. ‘Els projectes Europeus Engaging Science, Xplore Health, RRI Tools i Scientix’. Ciències: revista del professorat de ciències de Primària i Secundària, 30: 47–54.
Bardone, E, Burget, M, Saage, K and Taaler, M. 2017. ‘Making Sense of Responsible Research and Innovation in Science Education through Inquiry-based Learning: Examples from the Field’. Science Education International, 28(4): 293–304.
Bayram-Jacobs, D. 2015. ‘Responsible Research and Innovation: What is it? How to Integrate in Science Education’. Presented at International Congress on Education for the Future: Issues and Challenges (ICEFIC 2015) Conference. Ankara University, Turkey, 13–15 May.
Bayram-Jacobs, D, Henze, I, Evagorou, M, Shwartz, Y, Aschim, E, Alcaraz-Domínguez, S, Dagan, E and Barajas, M. 2017. ‘Exploring the impact of educative materials on teachers’ pedagogical content knowledge’. ESERA Conference. Dublin City University, Dublin, 21–25 August [Online]. Available at: https://keynote.conference-services.net/resources/444/5233/pdf/ESERA2017_0408_paper.pdf (Accessed 9 November 2018).
Blonder, R, Rap, S, Zemler, E and Rosenfeld, S. 2017. ‘Assessing Attitudes about Responsible Research and Innovation (RRI): The Development and Use of a Questionnaire’. Sisyphus-Journal of Education, 5(3): 122–156.
Blonder, R, Zemler, E and Rosenfeld, S. 2016. ‘The story of lead: A context for learning about responsible research and innovation (RRI) in the chemistry classroom’. Chemistry Education Research and Practice, 17(4): 1145–1155. DOI: https://doi.org/10.1039/C6RP00177G
Burget, M, Bardone, E and Pedaste, M. 2017. ‘Definitions and conceptual dimensions of Responsible Research and Innovation: A literature review’. Science and Engineering Ethics, 23(1): 1–19. DOI: https://doi.org/10.1007/s11948-016-9782-1
Clarke, D and Hollingsworth, H. 2002. ‘Elaborating a model of teacher professional growth’. Teaching and Teacher Education, 18(8): 947–967. DOI: https://doi.org/10.1016/S0742-051X(02)00053-7
David, PA. 2007. The Historical Origins of ‘Open Science’. Discussion paper, Stanford, CA, Stanford Institute for Economic Policy Research [Online]. Available at: http://www-siepr.stanford.edu/papers/pdf/06-38.pdf. (Accessed 9 November 2018).
Davis, EA and Krajcik, JS. 2005. ‘Designing educative curriculum materials to promote teacher learning’. Educational Researcher, 34(3): 3–14. DOI: https://doi.org/10.3102/0013189X034003003
de Vocht, M, Laherto, A and Parchmann, I. 2017. ‘Exploring teachers’ concerns about bringing Responsible Research and Innovation to European science classrooms’. Journal of Science Teacher Education, 28(4): 326–346. DOI: https://doi.org/10.1080/1046560X.2017.1343602
European Commission (EC). 2012. Responsible Research and Innovation: Europe’s ability to respond to societal challenges [Online]. Available at: http://ec.europa.eu/research/science-society/document_library/pdf_06/responsibleresearch-and-innovation-leaflet_en.pdf (Accessed 9 April 2018).
European Commission (EC). 2013. Options for Strengthening Responsible Research and Innovation – Report of the Expert Group on the State of Art in Europe on Responsible Research and Innovation [Online]. Available at: https://ec.europa.eu/research/science-society/document_library/pdf_06/options-for-strengthening_en.pdf (Accessed 24 June 2018).
European Commission (EC). 2015. Indicators for promoting and monitoring Responsible Research and Innovation [Online]. Available at: http://ec.europa.eu/research/swafs/pdf/pub_rri/rri_indicators_final_version.pdf (Accessed 24 June 2018).
Gorghiu, G, Anghel, GA and Ion, RM. 2015. ‘Students’ Perception Related to a Responsible Research and Innovation Demarche’. Procedia-Social and Behavioral Sciences, 180: 600–605. DOI: https://doi.org/10.1016/j.sbspro.2015.02.166
Gorghiu, LM, Dumitrescu, C and Petrescu, AMA. 2016. ‘Introducing RRI in Science Teaching – An Actual Challenge for Science Teachers’. European Proceedings of Social and Behavioural Sciences, 18: 183–191. DOI: https://doi.org/10.15405/epsbs.2016.12.25
Gott, R and Duggan, S. 1996. ‘Practical work: Its role in the understanding of evidence in science’. International Journal of Science Education, 18(7): 791–806. DOI: https://doi.org/10.1080/0950069960180705
Hodson, D. 2011. Looking to the Future. Rotterdam, Sense Publishers. DOI: https://doi.org/10.1007/978-94-6091-472-0
Hodson, D. 2014. ‘Becoming part of the solution: Learning about activism, learning through activism, learning from activism’. In: Bencze, L and Alsop, S (Eds.), Activist Science and Technology Education, 67–98. Netherlands, Springer. DOI: https://doi.org/10.1007/978-94-007-4360-1_5
Kiki-Papadakis, K and Chaimala, F. 2016. ‘The Embedment of Responsible Research and Innovation Aspects in European Science Curricula’. Romanian Journal for Multidimensional Education/Revista Romaneasca Pentru Educatie Multidimensionala, 8(2): 71–87. DOI: https://doi.org/10.18662/rrem/2016.0802.06
Kop, R, Fournier, H and Mak, JSF. 2011. ‘A pedagogy of abundance or a pedagogy to support human beings? Participant support on massive open online courses’. The International Review of Research in Open and Distributed Learning, 12(7): 74–93. DOI: https://doi.org/10.19173/irrodl.v12i7.1041
Lave, J and Wenger, E. 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge, Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511815355
Mikroyannidis, A, Okada, A, Correa, A and Scott, P. 2016. ‘Inquiry-Based Learning on the Cloud’. In: Chao, L (Eds.), Handbook of Research on Cloud-Based STEM Education for Improved Learning Outcomes, 291–310. Hershey PA, USA, Information Science Reference. DOI: https://doi.org/10.4018/978-1-4666-9924-3.ch019
Okada, A. 2014. ‘Scaffolding school students’ scientific argumentation in inquiry-based learning with evidence maps’. In: Okada, A, Buckingham, S, Simon, J and Sherborne, T (Eds.), Knowledge Cartography, 135–172. London, Springer.
Okada, A and Bayram-Jacobs, D. 2016. ‘Opportunities and challenges for equipping the next generation for responsible citizenship through the ENGAGE HUB’. Research papers presented at the 2016 LSME International Conference on Responsible Research in Education and Management and its Impact, 42–57. London 13th–15th January 2016.
Okada, A, Costa, A, Kowalski, R, Torres, P, Nakayama, M and Souza, K. 2016. ‘Open Educational Resources for Responsible Research and Innovation: a case study with Brazilian universities and schools’. International Conference on Responsible Research in Education and Management and its Impact, 42–57. London 13th–15th January 2016.
Okada, A, Kowalski, RPG, Kirner, C and Torres, PL. 2018. ‘Factors influencing teachers’ adoption of AR inquiry games to foster skills for Responsible Research and Innovation’. Interactive Learning Environments, 26(5). Early access. DOI: https://doi.org/10.1080/10494820.2018.1473257
Okada, A, Rossi, LC and Costa, A. 2015a. ‘Online argumentative maps for facilitating international debates with experts at large scale’. EC-TEL The 10th European Conference on Technology Enhanced Learning. Toledo, 15–18 September 2015.
Okada, A, Wolff, A, Mikroyannidis, A and Ashton, S. 2015b. ‘Promoting partnerships among Universities, Schools and Research Centres to foster Responsible Research and Innovation for smart citizenship’. Proceedings of the 15th International Conference on Technology, Policy and Innovation, Milton Keynes, 17–19 June 2015.
Okada, A, Young, G and Sanders, J. 2015d. ‘Fostering Communities of Practices for teachers’ professional development integrating OER and MOOC’. EC-TEL The 10th European Conference on Technology Enhanced Learning, Toledo, 15–18 September 2015.
Osborne, J. 2010. ‘Arguing to learn in science: The role of collaborative, critical discourse’. Science, 328(5977): 463–466. DOI: https://doi.org/10.1126/science.1183944
Owen, R, Macnaghten, P and Stilgoe, J. 2012. ‘Responsible research and innovation: From science in society to science for society, with society’. Science and Public Policy, 39(6): 751–760. DOI: https://doi.org/10.1093/scipol/scs093
Owen, R, Stilgoe, J, Macnaghten, P, Gorman, M, Fisher, E and Guston, D. 2013. ‘A framework for responsible innovation’. In: Owen, R, Bessant, J and Heintz, M (Eds.), Responsible Innovation: Managing the Responsible Emergence of Science and Innovation in Society, 27–50. John Wiley & Sons. DOI: https://doi.org/10.1002/9781118551424.ch2
Peciuliauskiene, P. 2017. ‘The engagement of secondary school learners for learning science by responsible research and innovation’. Proceedings of the International Scientific Conference, II: 151–162. 26–27 May. DOI: https://doi.org/10.17770/sie2017vol2.2249
Petrescu, AM, Gorghiu, G and Lupu, RA. 2015. ‘Non-formal Education-Frame for Responsible Research and Innovation Demarches’. Procedia-Social and Behavioral Sciences, 180: 682–687. DOI: https://doi.org/10.1016/j.sbspro.2015.02.178
Pinto, SM, Ribeiro, SF, Rocha, AKLT and Okada, A. 2018. ‘Argumentação de estudantes da educação básica sobre dilemas sócio-científicos no Projeto ENGAGE’. Revista Ibero-Americana de Estudos em Educação, 13(1): 207–228. DOI: https://doi.org/10.21723/riaee.v13.n1.2018.10242
Rocha, AK, Rocha, AB and Okada, A. 2017. ‘Rubric to assess evidence-based dialogue of socio-scientific issues with LiteMap’. The 2017 International Technology Enhanced Assessment Conference, 5–6 October 2017. Barcelona.
Ryan, C. 2015. Science Education for Responsible Citizenship [Online]. Available at: http://ec.europa.eu/research/swafs/pdf/pub_science_education/KI-NA-26-893-EN-N.pdf (Accessed 9 November 2018).
Supovitz, JA and Turner, HM. 2000. ‘The effects of professional development on science teaching practice in classroom culture’. Journal of Research in Science Teaching, 37(9): 963–980. DOI: https://doi.org/10.1002/1098-2736(200011)37:9%3C963::AID-TEA6%3E3.0.CO;2-0
Torres, PL, Fialho, NN, Kowalski, RPG and Okada, A. 2016. ‘Responsible Research and Innovation for the Media Facebook: Community Involvement in the Study on Agrobiodiversity’. Creative Education, 7(15): 2141–2150. DOI: https://doi.org/10.4236/ce.2016.715212
Von Schomberg, R. 2013. ‘A vision of responsible research and innovation’. In: Owen, R, Bessant, J and Heintz, M (Eds.), Responsible Innovation: Managing the Responsible Emergence of Science and Innovation in Society, 51–74. London, Wiley. DOI: https://doi.org/10.1002/9781118551424.ch3
Wickson, F and Carew, AL. 2014. ‘Quality criteria and indicators for responsible research and innovation: Learning from transdisciplinarity’. Journal of Responsible Innovation, 1(3): 254–273. DOI: https://doi.org/10.1080/23299460.2014.963004