Start Submission Become a Reviewer

Reading: Professional Development for Sustaining the ‘Pivot’: The impact of the Learning Design and C...

Download

A- A+
Alt. Display

Articles

Professional Development for Sustaining the ‘Pivot’: The impact of the Learning Design and Course Creation Workshop on Six Belarusian HEIs

Authors:

Tom Olney ,

The Open University, GB
X close

Siarhei Piashkun

Polotsk State University, BY
X close

Abstract

Throughout 2020, and into 2021, national governments were forced at different times to impose ‘lockdown’ on traditional approaches to education to cope with the impact of COVID-19. Higher education institutions (HEI) with face-to-face models scrambled to ‘pivot’ to distance and online learning.  Whilst originally conceived as a temporary measure, the longevity of the virus has raised significant questions about the sustainability of these approaches and revealed a need for teacher professional development (TPD) activity to support teachers in designing learning based on robust distance and online education frameworks. One example of such teacher professional development is the Learning Design and Course Creation (LDCC) Workshop from The Open University UK. Prior to the pandemic, in September 2018, this workshop was attended by staff from six Belarusian HEIs involved in a project titled Enhancement of Lifelong Learning in Belarus (BELL). The 18-month longitudinal study presented here used the Academic Professional Development Effectiveness Framework as a structure to evaluate the impact of this activity on the participants’ practice, and the five courses they were tasked with creating. The findings suggest that on the whole the team-based, reflective and experiential TPD pedagogy employed by the LDCC Workshop was effective in preparing the BELL Project participants for designing and creating their online and blended learning courses and this approach should be utilised effectively to support other HEIs that plan to enhance their distance capacity, either due to COVID-19, or for other reasons.

How to Cite: Olney, T. and Piashkun, S., 2021. Professional Development for Sustaining the ‘Pivot’: The impact of the Learning Design and Course Creation Workshop on Six Belarusian HEIs. Journal of Interactive Media in Education, 2021(1), p.10. DOI: http://doi.org/10.5334/jime.639
51
Views
12
Downloads
3
Twitter
  Published on 08 Sep 2021
 Accepted on 25 May 2021            Submitted on 29 Jan 2021

Introduction

In order to slow the transmission rates of COVID-19, previously co-located students and educators have been forced apart and have had to quickly adapt, or ‘pivot’, to distance learning approaches (Ferdig et al. 2020; Weller 2020). In some (usually Western and developed) settings this ‘pivot’ has been facilitated via the internet giving rise to online solutions, whilst in others (global South, less-developed nations) more traditional and frugal technologies such as TV, radio and print are also being successfully utilised as delivery methods (Dreesen et al. 2020). This rapid shift has contributed to heightened levels of anxiety in educators who are not experienced in distance education (JISC 2020).

In May 2020 the International Task Force on Teachers for Education (ITFTE), the United Nations Educational, Scientific and Cultural Organization (UNESCO) and the International Labour Organisation (ILO) published guidance for policy makers on measures to support teachers and education staff with the impact of COVID-19. The document calls for targeted TPD and a review of ‘policies and development frameworks to update provisions for distance education and education in emergencies to strengthen the resilience of education systems’ (UNESCO & ILO 2020: 4).

Whilst much of the focus has been about ‘pivoting online’ in order to cope with the immediate emergency, this only captures part of the challenge. In the UK at least, surveys show an increasing percentage of lecturers and executive leaders in HEI believe online will be a fundamental component of the future (JISC 2020; McVitty 2021) and preparation is urgent (Heap, Thompson & Fein 2020; Glantz & Gamrat 2020). Developing systems and resources – via TPD, for example – ‘…that can be leveraged in times of shock when core delivery models are disrupted’ is vital for future sustainability (Portillo & Lopez de la Serna 2020: 3).

Therefore, highlighted here is a TPD activity designed to develop teachers in an approach specifically for distance learning. The BELL participants were tasked with adapting what they learnt to develop online and blended learning (OBL) courses which can be defined as, ‘the thoughtful integration of classroom face-to-face learning experiences with online learning experiences’ (Garrison & Kanuka 2004: 96). The LDCC Workshop has been in iterative development by experts from The Open University, UK (UKOU), Europe’s largest HEI with over 50 years of experience in the field of distance learning. Since 2014, the workshop has been presented 27 times to a total of nearly 650 educators from at least ten external partner organisations.

This impact evaluation study uses an adapted version of the Academic Professional Development Effectiveness Framework (APDEF) (Chalmers & Gardiner 2015) to frame the evaluation of one specific instance of the LDCC Workshop in the context of the Enhancement of Lifelong Learning in Belarus (BELL) Project. Being active members of the BELL Project for over 18 months meant the researchers could achieve the desirable aims of selecting ‘relevant and varied data sources’ and gathering ‘evidence over time’ in order to prevent having only, ‘at best, a snapshot of the delivery of their programmes rather than evidence of their impact’ (Chalmers & Gardiner 2015: 82).

The APDEF matrix was developed in the Australian higher education context to assist academic developers to ‘evidence the effectiveness of their teacher development programmes’ at both a programme and institutional level (Chalmers & Gardiner 2015: 82). Since the focus of this study is on the impact of the TPD on the teacher it has been adapted here to include programme level data only. The APDEF is presented as a matrix that contains four impact indicator groupings (Input, Process, Output and Outcome), four areas of focus, and twenty specific ‘impact indicator statements’ that sit within the framework and guide the evaluation (Table 1).

Table 1

The APDEF indicators.


FOCUS AREA/IMPACT INDICATOR INPUT INDICATORS PROCESS INDICATORS OUTPUT INDICATORS OUTCOME INDICATORS

Teacher knowledge, skills, and practice 1. TPD delivered by staff with appropriate qualifications and experience.
2. TPD is aligned with University guidelines on good teaching and staff needs.
3. TPD provides a pedagogical framework for understanding teaching and learning in HE.
4. Delivery of TPD models teaching and learning strategies, resources and assessment practices which enhance the quality of teaching and learning.
5. Number of staff who complete the TPD. 6. TPD evaluations.
7. Teacher perceptions of changes in their approach to teaching and learning following completion of TPD.
8. Quality of teaching following completion of TPD.
9. Evidence of student focused approach in course or teaching materials.

Teacher reflective practice and scholarship of teaching 10. TPD aligns with institutional commitment to self-reflective practice and research informed teaching practice. 11. TPD encourages critical reflection of participants’ beliefs and practices regarding teaching, learning and assessment.
12. TPD incorporates research which informs teaching and learning in higher education.
13. TPD participants report the use of student feedback when reviewing courses and teaching.

Student engagement and enhancement of learning 14. TPD aligns with espoused priorities related to student learning experiences and engagement. 15. TPD draws on a framework of evidence-based teaching and learning practices.
16. Student perceptions of teaching are incorporated into TPD.
17. Unit evaluations.

Student approaches to learning 18. TPD incorporates University graduate attributes. 19. TPD highlights importance of relevant, authentic and inclusive assessment tasks. 20. TPD participant perceptions of quality of student assessment tasks.

Nine teams who trialled the APDEF in the Australian context utilised it in different ways. Five teams reported using the APDEF in developing a narrative about the TPD, and that approach has been adopted here. This narrative will use the four indicator groupings as main titles, whilst the impact indicator statements provide guidance as to the content to be included. Further, seven teams reported utilising the APDEF as a basis for reflective practice which is also consistent with the aims of this study. The APDEF provides a lens which enables a wider and deeper reflection to take place than might previously have been possible (Chalmers & Gardiner 2015).

To provide further focus, two research questions were defined based on the experience of the BELL participants at and after the LDCC Workshop:

RQ1: which components or elements of TPD activity were reported to be most effective in designing OBL courses?

RQ2: to what extent could the LDCC Workshop be utilised as a TPD activity for other HEIs wishing to embed sustainable distance learning approaches?

Literature Review

A recent comprehensive literature review of articles written between 2005–2014 on TPD programs that target OBL highlighted the scarcity of relevant research in this area. The review found that whilst research for TPD that is delivered online is well represented in the literature, TPD for OBL is not. Further, only one study, which focused on designing OBL, rather than teaching, was identified that could be said to be of direct comparison with the TPD presented here.

One specific finding highlighted by this review (Philipsen et al. 2019) is the importance of addressing the emotional and professional identity of teachers in TPD for OBL. This assertion is supported by Lockee (2020) who points out that both experienced and novice educators who are forced to change their teaching practice in compressed timescales (for example to accommodate OBL approaches) face serious challenges to their teaching identities, emotions and motivations. A study of eleven academics over six years from Finland revealed that TPD that sought to build reflective practice and peer interaction assisted most in the development of a teacher identity (Nevgi & Lofstrom 2015).

Although developed pre-pandemic, responses to the six component TPD framework suggested by Philipsen et al. (2019), agree that whilst it provides a useful mechanism to structure a post-COVID TPD discussion (Heap, Thompson & Fein 2020; Lockee 2020; Portillo & Lopez de la Serna 2020) the fact that it was conducted in a West-European context ‘is an overall limitation from an international perspective’ (Portillo & Lopez de la Serna 2020: 2).

Feedback gathered from 220 Chinese OU staff who participated in seven LDCC workshops between November 2017 and June 2019 found that a majority considered implementing the learning design approach to be ‘difficult’, whilst designing instructional interventions, keeping up to date with design theories and communicating in order to manage stakeholders, teams and projects were the most needed skills and competencies required to be developed further (Olney, Li & Luo 2021). The conclusion from a UKOU-led learning design workshop held in Nairobi, Kenya with 34 educational leaders from five African nations, was that learning design approaches needed to be heavily contextualised to meet the requirements of diverse institutions and cultures (Mittelmeier et al. 2018). A different study from Tanzania emphasised how the teacher teams that participated in TPD whilst being tasked with designing and delivering distance education found a collaborative team-based approach supportive and beneficial in building confidence and a favourable environment (Nihuka & Voogt 2012).

A review of literature that looked at TPD activities in the related field of the introduction of technology into the practice of student teachers, found that six of the nine studies analysed also addressed the importance of the relationship between beliefs about teaching and learning and changes to pedagogical practice (Uerz, Volman & Kral 2018). These studies typically described how the integration of educational technologies went along with opportunities for teacher educators to move from teacher-focused situations to student-focused ones. However, it also found that the current literature was ‘ambiguous’ about the nature of the relationship between the beliefs of teacher educators and the use of technology in teaching. In order to establish how one influences the other the authors concluded, ‘…further research is needed’ (Uerz, Volman & Kral 2018: 21).

In a quantitative meta-analysis Ilie et al. (2020) found that Instructional Development Programs (IDP) generally had little effect on the practice of those who participate. Their analysis suggested that non-pedagogical elements such as duration of IDP and mandatory enrolment were the most influential factors. However, they also recognised that the ‘internal connection between various elements’ may be the key to effectiveness, and future research should explore this (Ilie et al. 2020: 15).

The use of templates and concrete procedures to help with decision making is also a commonly incorporated feature of TPD programs in this field, such as that used by Nihuka & Voogt (2012), the Arena Blended Connected (ABC) Learning Design approach (Young & Perovic 2016), the Co-Designs Workshop (Co-Designs 2020), and the Integrated Learning Design Environment (ILDE) (Asensio-Perez et al. 2017).

Input Indicators

The input indicators suggested by the APDEF are:

1. TPD delivered by staff with appropriate qualifications and experience.

2. TPD is aligned with university guidelines on good teaching and staff needs.

10. TPD aligns with institutional commitment to self-reflective practice and research informed teaching practice.

14. TPD aligns with espoused priorities related to student learning experiences and engagement.

18. TPD incorporates university graduate attributes.

Enhancement of Lifelong Learning in Belarus (BELL) Project

Governmental guidance and strategic direction in the distance and OBL sector of Belarus is limited to one brief document which recognizes a desire to establish and organize ‘an educational process using modern information and communication technologies’ (Ministry of Education of Belarus 2010: 1). However, to date, no further decisions on questions such as modes of delivery, quality assurance, the basis for legal recognition or fundamental educational principles have been taken and this inaction has resulted in the sector being in urgent need of development.

The BELL Project formed part of the ERASMUS+ funded programme to build capacity in education through co-operation between European Union (EU) members and partner countries, such as Belarus. The central objective of the BELL project was the ‘development and enhancement of lifelong learning in Belarus with effective use of distance learning and Bologna instruments for the development of the regional labour market’ (BELL Project 2020).

Staff from six regional Belarusian HEIs (Brest, Horki, Gomel, Grodno, Polotsk and Vitebsk) worked in cross-institutional teams according to subject expertise towards the development of five OBL distance courses in the subject areas of: (1) English Language, (2) Internet Security, (3) Law Literacy, (4) Energy Saving, and (5) Entrepreneurship and Financial Literacy. The subject areas of the courses were selected as being relevant to the required skills and competencies needed by Belarusian target groups of learners which were, (1) the unemployed, (2) people between jobs, (3) retired people, (4) housewives, (5) students, (6) professionals wanting to update their practical skills.

The Belarusian staff were supported in this activity by experienced staff with expertise in OBL from four EU partner institutions (Rezekne Academy of Technologies Latvia, Linnaeus University Växjö Sweden, Cadiz University Spain, and the Open University UK). Each EU partner university facilitated a focused TPD activity designed to help the visitors form the skills necessary to design and deliver the OBL courses, primarily on the Moodle platform. The UKOU TPD in focus here, was designed and facilitated by three members of staff from the STEM Faculty with between them sixty-four years of experience.

Learning Design at UKOU

In Western HEIs, LD has sought to build on instructional design (ID) to the point where, ‘the two domains now overlap’ (MacLean & Scott 2011: 558). However, the two disciplines also differ in emphasis. For example, LD concentrates more closely than ID on the student experience, rather than that of the teacher/instructor. It also encourages taking a wider pedagogical approach as well as supporting the development of individual activities. Moreover, LD emphasises developing visualisations as a way of representing the student experience. These can then be shared and interrogated. LD is therefore inextricably linked with the field of learning analytics (LA) (Conole 2013; Conole & Wills 2013; Dalziel et al. 2016; Lockyer, Heathcote & Dawson 2013; Olney, Rienties & Toetenel 2019).

The particular interpretation of LD that is currently in practice at the UKOU has its foundation in the findings from the OU Learning Design Initiative (OULDI) which ran from 2007 to 2012. The UKOU and 13 other higher education institutions participated in the Institutional Approaches to Curriculum Design and Delivery programme which was co-funded by the not-for-profit Joint Information Systems Committee (JISC) and the European Union (EU) (Conole & Wills 2013). Wide ranging interviews with staff at these institutions revealed a multitude of design practices. As a consequence of the OULDI, since 2012 LD practitioners at the UKOU have sought to embed design approaches that are student-focused and based around the three principles of:

  1. encouraging design conversations and collaboration in design;
  2. using tools, instruments, and activities to describe and share designs;
  3. developing LA approaches to support and guide decision-making (Galley 2015).

Practitioners from the UKOU have facilitated many TPD sessions of differing duration and intensity based on the OULDI approach to over forty international HEIs including in Belarus, China, Israel, Kenya, Myanmar, the Netherlands, Pakistan, South Africa, and Sri Lanka.

In the daily life of the UKOU, LD workshops provide a mechanism for bringing together learning designers, academics, project managers, employability specialists, and technical support staff in teams to design new curricula. Outputs from these workshops, such as student profiles, learning outcomes, activity planners and assessment strategies, are then recognised as key components in quality assurance (Galley 2015).

Process Indicators

The process indicators suggested by the APDEF are:

3. TPD provides a pedagogical framework for understanding teaching and learning in HE.

4. Delivery of TPD models teaching and learning strategies, resources and assessment practices which enhance the quality of teaching and learning.

11. TPD encourages critical reflection of participants’ beliefs and practices regarding teaching, learning and assessment.

12. TPD incorporates research which informs teaching and learning in higher education.

15. TPD draws on a framework of evidence-based teaching and learning practices.

16. Student perceptions of teaching are incorporated into TPD.

19. TPD highlights importance of relevant, authentic, and inclusive assessment tasks.

The Learning Design and Course Creation Workshop

The LDCC Workshop allows staff from the UKOU to engage with the professional development of educators from other HEIs. It provides participants with an experience which mirrors as closely as possible the experience of a multi-disciplinary team tasked with creating and producing a new distance education course at the UKOU, although within a narrower timespan. It has been designed to promote student-focused design and to model the learning design principles outlined previously.

In the BELL Project LDCC Workshop the participants were organised into teams of five and worked together through a programme of activities. Each team decided on their course subject area, duration and level, allocated roles and responsibilities to one another (in line with self-declared discipline expertise, skills, interests and competencies that they perceived as important) and developed a vision statement for their course using the free, online ‘learning design wordwheel’ (Learning Design Wordwheel 2016; Olney, Rienties & Toetenel 2019). The participants then considered the particular needs, characteristics and learning preferences of their hypothetical students by creating one or more student profiles. Once completed, they were supported in the process of generating learning outcomes, learning activities and assessment tasks in accordance with constructive alignment principles (Biggs 1996). The iterative design cycle process was then visualised and structured using the Activity Types Classification Framework incorporating the allocation of anticipated student workload (Conole 2013; Olney, Rienties & Toetenel 2019).

Participants were given access to their own bespoke Moodle-based UKOU Virtual Learning Environment (VLE) website with an initial layout of three to five weeks’ worth of study. They worked together to transfer their design online and to see it come to life. The participants were encouraged to reflect on both the hypothetical student experience – and their own – through a number of inbuilt activities which included a final presentation to their peers.

Output Indicators

The output indicators suggested by the APDEF are:

5. Number of staff who complete the TPD.

The participants who took part in this study were Belarusian staff engaged in some way in the development of the five OBL courses and who, in October 2018, visited the main UKOU campus in Milton Keynes and attended the LDCC Workshop. Typically, the participants were academics but there were also academic-related and technical support staff in attendance. None of the participants had any prior knowledge of the LDCC approach or of developing modules in an online and distance setting.

There were 22 participants at the LDCC workshop.

Outcome Indicators

The outcome indicators suggested by the APDEF are:

6. TPD evaluations.

7. Teacher perceptions of changes in their approach to teaching and learning following completion of TPD.

8. Quality of teaching following completion of TPD.

9. Evidence of student focused approach in course or teaching materials.

13. TPD participants report the use of student feedback when reviewing courses and teaching.

17. Unit evaluations.

20. TPD participant perceptions of quality of student assessment tasks.

Design, collection, and analysis of outcome indicator data

Outcome indicator data was gathered from the LDCC Workshop participants using 10 different instruments (labelled A-J) over 18 months by the UKOU researchers and the BELL Project Quality Officer (Table 2).

Table 2

Sources of evidence used for the outcome indicators.


EVIDENCE TYPE NAME NO. OF RESPONDENTS WHEN

A Written survey
Likert scale & free text
UKOU learning and teaching survey (immediate) 18 Sep 2018

B Online survey
Likert scale & free text
UKOU LDCC workshop experience survey 15 Sep 2018

C 2 x face-to-face focus groups BELL Project LDCC workshop focus groups 18 Sep 2018

D Records Teaching awards and promotions 32 Oct 2020

E Online survey BELL Project satisfaction survey 18 Jan 2020

F Online survey
Likert & multiple choice
UKOU learning and teaching survey (reflective) 19 Mar 2020

G Online semi- structured interviews UKOU learning and teaching interviews (reflective) 9 April 2020

H Face to face interviews BELL Project learning and teaching interviews (reflective) 18 May 2020

I Evaluation report ERASMUS+ evaluations of the courses 5 courses Oct 2020

J Evaluation report HEI Management evaluations of the courses 5 courses Oct 2020

Mapping the sources of evidence to the APDEF Outcome Indicators (Table 3) demonstrates the comprehensive cover provided by the evaluation approach.

Table 3

Sources of evidence mapped to outcome indicators.


FOCUS AREA APDEF OUTCOME INDICATORS EVIDENCE SOURCES

Teacher knowledge, skills and practice 6. TPD evaluations. B, C & E

7. Teacher perceptions of changes in their approach to teaching and learning following completion of TPD. A, B, F & G

8. Quality of teaching following completion of TPD. D & H

9. Evidence of student focused approach in course or teaching materials. F & G

Teacher reflective practice and scholarship of teaching 13. TPD participants report the use of student feedback when reviewing courses and teaching. G & I

Student engagement and enhancement of learning 17. Unit evaluations. I & J

Student approaches to learning 20. TPD participant perceptions of quality of student assessment tasks. G, H & I

The sources of evidence were gathered using the following methodologies:

A: Participants were asked to respond in writing to two questions which asked how easy or difficult they thought implementing the approach they had been shown would be using a four-point Likert scale. Participants were also asked to comment on what they felt would need to change in order to make that implementation easier. The open text data underwent an inductive thematic coding analysis to establish the most commonly identified responses.

B: Participants were asked to rate their experience and satisfaction of the workshop across nine areas (clarity of objectives, participation, content, materials, timing, facilities, realisation of objectives, relevance, and the facilitators’ suitability) using a 5-point Likert scale. Participants were also asked to comment on the element of the workshop they liked most and any improvements.

C: Participants were asked to contribute to two focus groups, facilitated in Belarusian by Author2 and structured to elicit reflective responses in regard to (1) their experience at the LDCC Workshop and (2) how they intended to use the approaches in their courses. The interviews were recorded, transcribed, and translated into English by Author2.

D: Participants were asked to advise the BELL Project of any teaching awards or promotions they believed could be attributed to their work in this project.

E: Participants were asked to respond to an online survey designed to gather their opinions on the extent to which the project experience had met their expectations and whether the project aims had been achieved. Data was exported from MS Forms, formatted, and manipulated using Excel software and analysed by the BELL Project Quality Team.

F: Participants were asked to respond in an online survey to four questions designed to gather understanding on perceptions of ‘helpfulness’, ‘ease of use’, ‘how used’ and ‘intention to use in future’ for the seven learning design activities presented in the LDCC workshop using a five-point Likert scale which included a ‘not used’ option. Participants were also re-asked the second question from A, but this time as a multiple-choice question, using options generated from the original. They were also asked if they were willing to take part in a qualitative interview. Data was exported from Microsoft Forms, formatted, and manipulated using Excel software and uploaded to NVivo.

G: Participants took part in semi-structured online synchronous interviews with Author1 and the other co-facilitators designed to encourage reflections on how they had used the LDCC activities in the development of their BELL project module, in their other teaching and learning work and how they planned to use these activities in the future. Interviews were conducted and recorded using Skype, and then uploaded as MP4 files to a secure OneDrive folder where they were transcribed, anonymised, and transferred into NVivo. Interviews were coded by Author1 using the structure of the APDEF.

H: Participants took part in face-to-face interviews designed and facilitated by the BELL Project Quality Team.

I: The five courses were externally evaluated by an expert hired to assess the quality of the courses against metrics decided by the BELL Project Quality Team. Their report was submitted to the BELL Project Management Team and shared with the participants at the BELL Project Final Conference.

J: The five courses were evaluated by the managers and lead teachers at the respective Institutes for Continuing Education at the six Belarusian HEIs involved in the BELL Project, against their own internally established metrics for quality, student satisfaction and innovation.

Findings

For increased clarity in this section, the findings are presented here under each of the specific ‘outcome indicator statements’ headings.

6. TPD evaluations

In responding to the online survey immediately after the TPD, respondents indicated very high satisfaction with the LDCC Workshop. All of the areas were overwhelmingly rated as ‘excellent’. When asked to comment on aspects of the TPD that they liked best; 6 respondents focused on the practical nature of the workshop, 6 mentioned the opportunity to work in teams, and 5 referenced the attitude and competence of the facilitators. 3 respondents thought more time would be an improvement (evidence source B).

In the qualitative focus group interviews the participants also expressed a high level of satisfaction with the TPD. Fifteen participants highlighted the novelty of the approaches presented, and the effective implementation of practical examples and peer learning to stimulate their attention. Fourteen participants referenced how they perceived the experience of individualising learning and the profiling of potential students as a benefit to their practice. Whilst all the participants expressed their willingness to adapt the learnt practices to their work in Belarus, four were sceptical about successfully being able to do so given the educational culture at their institution (evidence source C). Overall, the participants reported that the LDCC Workshop was the most relevant and helpful of the four EU Partner visits for the development of the skills they needed to deliver the aims of the BELL Project (evidence source E).

7. Teacher perceptions of changes in their approach to teaching and learning following completion of TPD

When asked at interview 18 months after the TPD the general reactions offered by the participants (anonymised and referred to by the numbers in square brackets) to the LDCC Workshop were still very positive. For example, the interviewed participants described their experience as, ‘very fruitful’ [06], ‘very useful’ [07], and, making ‘changes in our way of thinking in our minds’ [08]. One participant went so far as to describe their reaction to the TPD as like the, ‘discovery of a new continent by Columbus’ [02] due to the way it introduced approaches that could lead to design being ‘done in a completely different way’ [02] (evidence source G). Figure 1 shows that 88% of the participants who responded to the survey considered the activities to be either ‘very helpful’ or ‘helpful’ in designing their module (evidence source F).

Figure 1 

Helpfulness of LDCC activities (evidence source F).

Two-thirds (12) of the 18 participants initially considered implementing the OU approach to be either ‘difficult’ or ‘very difficult’ (evidence source A). However, on reflection, after the modules were designed and delivered, many considered implementing the LDCC approaches to be much easier than originally perceived. For example, despite one participant noting that ‘in our educational culture teamwork is not developed at all as we saw [in the EU partner HEIs]’ [13] the interviews revealed that the teams adapted and employed a variety of different communication and decision-making strategies in order to successfully collaborate. Several participants also pointed out that once they started with the work and faced the practical necessities of the challenge any big concerns about difficulties were dealt with successfully [01] [06] [10] [14]. On completion, some participants commented that they actually found the approaches ‘easy to adapt’ [19] or unproblematic because they were similar to what they may have ‘usually used by initiative’ [02] (evidence source G). In fact, the responses presented in Figure 2 suggest that after using them themselves, 77% of the participants who responded now considered the LDCC activities either ‘easy’ or ‘very easy’ to use (evidence source F).

Figure 2 

Ease of use of LDCC approaches (evidence source F).

Figure 3 shows that 79% of participants who responded said they intended to use the activities ‘exactly as shown’ or as an ‘adapted version’ in their normal practice (evidence source F). When asked to elaborate on which activities they intended to use one respondent said, ‘I think all of them’ [06] whilst another highlighted the importance of modifications ‘due to the specifics of the Belarusian system of education’ [10]. Respondent [08] reflected on how what they had learnt at the TPD provided them with a mechanism to ask their students about ‘the effectiveness of the study process’ and eventually ‘rearrange my teaching process’ (evidence source G).

Figure 3 

Intention to use LDCC approaches in future (evidence source F).

8. Quality of teaching following completion of TPD

Three members of the five project teams reported promotion to positions of lead teachers due to their experience of implementing changes to the existing curricula and their department’s policies, as well as providing mentoring and peer assessment for other teachers (evidence source D). Three of the six Belarussian HEIs organised their own TPDs for 490 of their wider staff in 2020 which incorporated the topic of learning design and course creation (evidence source H).

9. Evidence of student focused approach in course or teaching materials

The student profile activity appeared to particularly resonate with the participants as a way to focus in on matching design decisions with prospective students’ needs and motivations [01] [13] [14] in the design of their BELL courses. In one case it even led to one module team specifically sourcing data on their students that was previously unavailable [13] whilst another reported that they felt they needed more ongoing support from experts to make the approach work [06]. Several others referred to using the Activity Types Classification Framework to structure their learning designs in a student-focused way [01] [07] [10] and this TPD activity was, for one respondent at least, ‘my best experience of the OU’ [14] (evidence source G). Figure 4 shows a relatively equal distribution of engagement from the respondents with the range of student focused approaches they were exposed to (evidence source F).

Figure 4 

Comparison of the use of the student focused approaches from the LDCC Workshop (evidence source F).

13. TPD participants report the use of student feedback when reviewing courses and teaching

Many of the participants reported gathering some basic learning analytics from Moodle or direct feedback from students via questionnaires to inform changes to their module design with improved retention or quality in mind [01] [02] [06] [07] [08] [10] [13] [14] when previously this ‘was not common practice for us’ [02]. One respondent expressed difficulty in arranging the data they did receive [19] (evidence source G) and the external evaluation suggested more attention could be paid to motivating students during the course to prevent student dropout. (evidence source I).

17. Unit evaluations

To date, the five courses have been presented twice. The results of the evaluation from the external expert concluded that the learning designs adopted were closely aligned with their stated aims in terms of pedagogy, student outcomes and practical skills and that the employment of several activities from the LDCC Workshop had contributed to this. After adjustments to the learning designs the percentage of students who successfully completed the six courses rose from 37.6% to 49.8% (evidence source I). The Belarusian HEIs’ internal evaluations of the five courses also found: (1) the stated goals, learning design, methods, and learning outcomes made the courses well-rounded and comprehensive; (2) the use of different online education tools and methods was exemplary; (3) the formative assessments enhanced students’ soft skills; and (4) students and their learning was the focal point (evidence source J).

20. TPD participant perceptions of quality of student assessment tasks

Nine participants directly linked their choices about adopting formative or summative assessment approaches as being informed by their experience of the LDCC Workshop (evidence source H). There was also evidence in the interviews of some participants attempting to move away from assessment of learning tasks, such as end of module tests, and towards more assessment for learning models, including the use of ‘practical tasks’ [01] [02] [14]. However, ‘knowledge based’ traditional assessment was still preferred by some respondents [06] [13] [19] (evidence source G) and the external evaluation of the courses suggested this could be an area requiring further attention (evidence source I).

Discussion

The establishment of resilient and robust ‘teacher identities’ is highly desirable for an HEI because it develops teachers who are better equipped to cope with change and can support the introduction of and sustainability of new practices (Philipsen et al. 2019; Uerz, Volman & Kral 2018; Nevgi & Lofstrom 2015) and the sudden challenge of the ‘pivot’ to OBL due to COVID-19 makes this of particular relevance right now. A TPD that can help to establish such identities in a relevant OBL field, such as learning design and the use of educational technologies, is, therefore, of significant value.

Interpreting the findings of the feedback from the BELL participants and the other sources of evidence through the lens of the 20 APDEF impact indicator statements suggests that the LDCC Workshop was very effective in supporting the development of an ‘OBL teacher identity’, which was primarily achieved through the employment of a team-based, reflective and experiential TPD pedagogy. The findings suggest that the tools, activities and practices contained within the LDCC Workshop coupled with a practical, peer-supported setting enabled many of the participants to ‘pivot’ their thinking from a teacher-focused view of design and course creation to a more student-focused one. The BELL participants made this particularly clear in their responses to the outcome indicators:

6. TPD evaluations; 7. Teacher perceptions of changes in their approach to teaching and learning following completion of TPD; and, 9. Evidence of student focused approach in course or teaching materials.

Establishing multi-disciplinary collaborative design practice can be challenging and ownership of the courses that are created, contributors’ expertise and the degree of course development input required have been identified as problematic areas (Chao, Saj & Hamilton 2010; Halupa 2019). However, utilising a collaborative team-based approach is core to the LDCC Workshop pedagogy and has been considered vital to good practice in learning design and course creation (Schmidt, Tschida & Hodge 2016; Chao, Saj & Hamilton 2010; Galley 2015; Halupa 2019). The findings of this evaluation support the view that despite some initial concerns amongst the participants that adopting a collaborative approach would be difficult, time consuming and outside their normal practice, the activities from the LDCC Workshop provided the participants with confidence and strategies in getting started and adapting this approach to their purposes.

Nihuka & Voogt (2012) evaluated the experiences of six teacher design teams who attended a collaborative TPD and were then tasked with designing and creating a set of distance courses. The evaluation reported that all the teams regarded the collaborative element of the TPD pedagogy as a specific strength and had assisted them in the realisation of their courses. The teacher design teams in this study also particularly highlighted the value of using the TPD to share and experiment with design activities and procedures (similar to those explored by the BELL participants in the LDCC Workshop) and the positive impact this had on their professional learning. Also, built into the design of the LDCC Workshop, and the wider BELL project, were many opportunities for the participants to reflect, both individually and in groups on their progress. The importance of such reflective activities, as well as developing knowledge of theoretical pedagogical constructs, has been recognised as crucial to the development of teacher identity (Nevgi & Lofstrom 2015).

The gathering, visualising, and interpreting of learning design analytics is an important component of learning design approaches, particularly in OBL (Olney, Rienties & Toetenel 2019; Dalziel et al. 2016). Whilst the BELL participants were able to explain how they had gathered some direct student feedback this was an area that could be developed further. Similarly, the concept of assessment for learning was also underdeveloped by some teams.

Limitations and Future Work

This study focused on the impact of one instance of TPD on a group of participants tasked with designing and creating OBL. However, these participants also attended other TPD during this time, and whilst the evidence sources specifically asked the participants to focus on their experience of the LDCC Workshop, the overall cumulative effect of practice and TPD is acknowledged as being difficult to control for.

In terms of providing a structure or organising principle the APDEF proved useful in grouping and arranging previously gathered evaluation data in a logical and relevant way. The editors of the special edition in which the APDEF is described, applied the APDEF to the evaluation of several TPDs contained in the journal volume, concluding that the focus areas of student engagement and enhancement of teaching and student approaches to learning were not as well represented in the evaluations as the other focus areas (Saroyan & Trigwell 2015).

This observation is consistent with the evaluation provided here. On reflection, this study has been able to provide much good quality evidence about the effectiveness of the workshop in the focus areas of teacher knowledge, skills and practice and teacher reflective practice and scholarship of teaching but the other two focus areas are underrepresented.

Also, despite the relative weight placed on the ‘outcomes’, considering ‘input’ and ‘process’ as impact indicators provided the authors with a previously unthought of lens to consider. Both as a narrative and reflective tool the experience of this evaluation suggests that the APDEF could be utilised as a useful framework for other TPD.

The LDCC Workshop was not specifically designed for use with Belarusian HEI, rather it was developed primarily for Chinese academics and, as has been previously mentioned, this group constitutes a far larger cohort. Olney, Li & Luo (2021) used data gathered from Chinese LDCC Workshop participants to identify learning design skills and competencies that could be developed further to enhance the quality of open and distance learning in China. A further series of in-depth qualitative interviews will be taking place to better understand the impact of the LDCC Workshop in this context.

Conclusions

Now the BELL Project has come to an official close, opportunities and challenges remain. What seems certain is that the professional development undertaken by the staff at the six Belarusian HEIs leaves them well placed to leverage their newly acquired skills and competencies to build a sustainable capacity in OBL that can cope with the ongoing presence of COVID-19. In fact, some of the Belarusian HEIs involved have already developed plans to extend their OBL provision to different subject areas and levels (Laganovska & Pazdniakova 2020).

The findings presented here suggest that on the whole the team-based, reflective and experiential TPD pedagogy employed by the LDCC Workshop was effective in preparing the BELL Project participants for designing and creating their OBL courses but that in some other areas, such as learning analytics and assessment, the participants might benefit from further TPD.

The evidence also points strongly to the conclusion that team-based, reflective, and experiential TPD pedagogies, such as those in the LDCC Workshop, should be utilised to support other HEIs wishing to embed sustainable approaches to distance education as a consequence of COVID-19.

Acknowledgements

The authors would also like to acknowledge the contribution of Mark Endean and Duncan Banks. They are co-developers and co-facilitators of the LDCC Workshop and interviewed some of the participants.

Competing interests

The authors have no competing interests to declare.

Author Contributions

Author1 is Tom Olney, Senior Manager for Teaching and Learning in STEM at The Open University, UK. He is a co-developer and co-facilitator of the LDCC workshop. He led on the writing of this paper, the design, collection, and analysis of some of the evidence types, as well as the interviews with the participants.

Author2 is Siarhei Piashkun, Head of Strategic Planning and Internationalisation at Polotsk State University, Belarus. He was a member of the BELL Project Quality Team. He provided the rest of the evidence types and contributed to the writing of this paper.

References

  1. Asensio-Perez, J, Dimitriadis, Y, Pozzi, F, Hernandez-Leo, D, Prieto, L, Persico, D and Villagra-Sobrino, S. 2017. Towards teaching as design: Exploring the interplay between full-lifecycle learning design tooling and teacher professional development. Computers & Education, 114: 92–116. DOI: https://doi.org/10.1016/j.compedu.2017.06.011 

  2. BELL Project. 2020. About Project. Available at https://bell-iln.by/en/about-project. (Last accessed 28 January 2021). 

  3. Biggs, J. 1996. Enhancing teaching through constructive alignment. Higher Education, 32(3): 347–364. ISSN 0018-1560. DOI: https://doi.org/10.1007/BF00138871 

  4. Chalmers, D and Gardiner, D. 2015. An evaluation framework for identifying the effectiveness and impact of academic teacher development programmes. Studies in Educational Evaluation, 46: 81–91. DOI: https://doi.org/10.1016/j.stueduc.2015.02.002 

  5. Chao, I, Saj, T and Hamilton, D. 2010. Using collaborative course developments to achieve online course quality standards. International Review of Research in Open and Distance Learning, 11(3): 106–126. ISSN: 1492-3831. DOI: https://doi.org/10.19173/irrodl.v11i3.912 

  6. Co-Designs. 2020. Learning Design Framework. Available at https://codesignsframework.com/ (Last accessed 22 January 2021). 

  7. Conole, G. 2013. Designing for learning in an open world. Explorations in the Learning Sciences, Instructional Systems and Performance Technologies, 4. New York: Springer. DOI: https://doi.org/10.1007/978-1-4419-8517-0 

  8. Conole, G and Wills, S. 2013. Representing learning designs – making design explicit and shareable. Education Media International, 50(1): 24–38. DOI: https://doi.org/10.1080/09523987.2013.777184 

  9. Dalziel, J, Conole, G, Wills, S, Walker, S, Bennett, S, Dobozy, E, Cameron, L, Badilescu-Buga, E and Bower, M. 2016. The Larnaca declaration on learning design. Journal of Interactive Media in Education, 2016(1): 7. DOI: https://doi.org/10.5334/jime.407 

  10. Dreesen, T, Akseer, S, Brossard, M, Dewan, P, Giraldo, JP, Kameii, A, Mizunoya, S and Santiago, J. 2020. Promising practices for equitable remote learning. Emerging lessons from COVID-19 education responses in 127 countries. UNICEF Office of Research – Innocenti Research Brief 2020–10. Available at https://www.unicef-irc.org/publications/1090-promising-practices-for-equitable-remote-learning-emerging-lessons-from-covid.html (Last accessed 22 January 2021). 

  11. Ferdig, R, Baumgartner, E, Hartshorne, R, Kaplan-Rakowski, R and Mouza, C. (eds.) 2020. Teaching, technology, and teacher education during the COVID-19 pandemic: Stories from the field. Association for the Advancement of Computing in Education (AACE). Available at https://www.learntechlib.org/p/216903/ (Last accessed 29 January 2021). 

  12. Galley, R. 2015. Learning design at The Open University: Introducing methods for enhancing curriculum innovation and quality. Quality Enhancement Report Series (1), The Open University, Milton Keynes, UK: Institute of Educational Technology (IET). 

  13. Garrison, D and Kanuka, H. 2004. Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education, 7: 95–105. DOI: https://doi.org/10.1016/j.iheduc.2004.02.001 

  14. Glantz, E and Gamrat, C. 2020. The new post-pandemic normal of college traditions. In: Proceedings of the 21st Annual Conference on Information Technology Education, US Virtual Conference, 7–9 October 2020, 279–284. DOI: https://doi.org/10.1145/3368308.3415375 

  15. Halupa, C. 2019. Differentiation of roles: Instructional designers and faculty in the creation of online courses. International Journal of Higher Education, 8(1): 55–68. DOI: https://doi.org/10.5430/ijhe.v8n1p55 

  16. Heap, T, Thompson, R and Fein, A. 2020. Designing teacher professional development programs to support a rapid shift to digital. Educational Technology and Research Development. DOI: https://doi.org/10.1007/s11423-020-09863-5 

  17. Ilie, M, Maricutoiu, L, Iancu, D, Smarandache, I, Mladenovici, V, Stoia, D and Toth, S. 2020. Reviewing the research on instructional development programs for academics. Trying to tell a different story: A meta-analysis. Educational Research Review, 30: 1–18. DOI: https://doi.org/10.1016/j.edurev.2020.100331 

  18. JISC. 2020. Learning and teaching reimagined: Synthesis of audience surveys. Available at https://repository.jisc.ac.uk/8153/1/learning-and-teaching-reimagined-synthesis-of-audience-surveys.pdf. (Last accessed 28 January 2021). 

  19. Laganovska, K and Pazdniakova, I. 2020. BELL ERASMUS+ capacity-building project: Implementation of innovative practices in lifelong learning in Belarus. In: Proceedings of the Final Conference of the BELL Project, 13–14 October 2020. Available at https://bell-iln.by/en/final-conference (Last accessed 29 January 2021). 

  20. Learning Design Wordwheel. 2016. Available at https://www.open.edu/openlearn/wordwheel (Last accessed 22 January 2021). 

  21. Lockee, B. 2020. Shifting digital, shifting context: (re)considering teacher professional development for online and blended learning in the COVID-19 era. Educational Technology and Research Development. DOI: https://doi.org/10.1007/s11423-020-09836-8 

  22. Lockyer, L, Heathcote, E and Dawson, S. 2013. Informing pedagogical action: Aligning learning analytics with learning design. American Behavioural Scientist, 57(10). DOI: https://doi.org/10.1177/0002764213479367 

  23. MacLean, P and Scott, B. 2011. Competencies for learning design: A review of the literature and a proposed framework. British Journal of Educational Technology, 42(4): 557–572. ISSN 0007-1013. DOI: https://doi.org/10.1111/j.1467-8535.2010.01090.x 

  24. McVitty, D. 2021. Seven things the HE sector learned in 2020 and what universities should prepare for in 2021. Available at https://wonkhe.com/blogs/seven-things-the-he-sector-learned-in-2020-and-what-universities-should-prepare-for-in-2021/ (Last accessed 28 January 2021). 

  25. Ministry of Education of Belarus. 2010. On some features of obtaining higher education of the first stage in a distance form in higher educational institutions. Minsk, Belarus: Ministry of Education in the Republic of Belarus, No. 108. 19 October 2010. Available at https://edu.gov.by/sistema-obrazovaniya/vysshee-obrazovanie/norm-akty/org-protsessa/zaochnoe/ (Last accessed 21 May 2021). 

  26. Mittelmeier, J, Long, D, Melis-Cin, F, Reedy, K, Gunter, A, Raghuram, P and Rienties, B. 2018. Learning design in diverse institutional and cultural contexts: Suggestions from a participatory workshop with higher education professionals in Africa. Open Learning: The Journal of Open, Distance and e-Learning. 33(3): 250–266. DOI: https://doi.org/10.1080/02680513.2018.1486185 

  27. Nevgi, A and Lofstrom, E. 2015. The development of academics’ teacher identity: Enhancing reflection and task perception through a university teacher development programme. Studies in Educational Evaluation, 46: 53–60. DOI: https://doi.org/10.1016/j.stueduc.2015.01.003 

  28. Nihuka, K and Voogt, J. 2012. Collaborative e-learning course design: Impacts on instructors in the Open University of Tanzania. Australasian Journal of Educational Technology, 28(2): 232–248. DOI: https://doi.org/10.14742/ajet.871 

  29. Olney, T, Li, C and Luo, J. 2021. Enhancing the quality of open and distance learning in China through the identification and development of learning design skills and competencies. Asian Association of Open Universities Journal, ahead of print. DOI: https://doi.org/10.1108/AAOUJ-11-2020-0097 

  30. Olney, T, Rienties, B and Toetenel, L. 2019. Chapter 6: Gathering, visualising and interpreting learning design analytics to inform classroom practice and curriculum design. In: Lodge, J, Horvath, J and Corrin, L (eds.), Learning analytics in the classroom: Translating learning analytics for teachers. Routledge Press. ISSN 1351113011. DOI: https://doi.org/10.4324/9781351113038-6 

  31. Philipsen, B, Tondeur, J, Pareja Roblin, N, Vanslambrouck, S and Zhu, C. 2019. Improving teacher professional development for online and blended learning: A systematic meta-aggregative review. Educational Technology and Research Development, 67: 1145–1174. DOI: https://doi.org/10.1007/s11423-019-09645-8 

  32. Portillo, J and Lopez de la Serna, A. 2020. An international perspective for ‘improving teacher professional development for online and blended learning: A systematic meta-aggregative review’. Educational Technology and Research Development. DOI: https://doi.org/10.1007/s11423-020-09851-9 

  33. Saroyan, A and Trigwell, K. 2015. Higher education teachers’ professional learning: Process and outcome. Studies in Educational Evaluation, 46: 92–101. DOI: https://doi.org/10.1016/j.stueduc.2015.03.008 

  34. Schmidt, S, Tschida, C and Hodge, E. 2016. How faculty learn to teach online: What administrators need to know. Online Journal of Distance Learning Administration, 19(1): 1–10. 

  35. Uerz, D, Volman, M and Kral, M. 2018. Teacher educators’ competences in fostering student teachers’ proficiency in teaching and learning with technology: An overview of relevant research literature. Teaching and Teacher Education, 70: 12–23. DOI: https://doi.org/10.1016/j.tate.2017.11.005 

  36. UNESCO and ILO. 2020. Supporting teachers in back-to-school efforts: Guidance for policymakers. International Task Force on Teachers for Education 2030. Available at https://unesdoc.unesco.org/ark:/48223/pf0000373479 (Last accessed 22 January 2021). 

  37. Weller, M. 9 March 2020. The COVID-19 online pivot. The Ed-Techie [online]. Available at https://blog.edtechie.net/higher-ed/the-covid-19-online-pivot/ (Last accessed 18 May 2021). 

  38. Young, C and Perovic, N. 2016. Rapid and creative course design: As easy as ABC. In: Proceedings of the 2nd International Conference on Higher Education Advances, Valencia, Spain, June 2016. DOI: https://doi.org/10.1016/j.sbspro.2016.07.058 

comments powered by Disqus