A Learning Design Toolkit to Create Pedagogically Effective Learning Activities

The Open University's repository of research publications and other research outputs A learning design toolkit to create pedagogically effective learning activities Journal Article How to cite: Conole, Grainne and Fill, Karen (2005). A learning design toolkit to create pedagogically effective learning activities. Journal of Interactive Media in Education(1) For guidance on citations see FAQs. c [not recorded] Version: [not recorded] Link(s) to article on publisher's website: Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyright owners. For more information on Open Research Online's data policy on reuse of materials please consult the policies page. A learning design toolkit to create pedagogically effective learning activities Abstract: Despite the plethora of Information and Communication Technologies (ICT) tools and resources available, practitioners are still not making effective use of e-learning to enrich the student experience. This article describes a learning design toolkit which guides practitioners through the process of creating pedagogically informed learning activities which make effective use of appropriate tools and resources. This work is part of a digital libraries project in which teaching staff at two universities in the UK and two in the USA are collaborating to share e-learning resources in the subject domains of Physical, Environmental and Human Geography. Finding, or creating, suitable e-learning resources and embedding them in well designed learning activities can be both challenging and time consuming. Sharing and adapting effective designs and solutions is both a stimulant and a time saver. This article describes the background to the specification of a learning activities design toolkit to support teachers as they create or adapt e-learning activities. This uses a model of pedagogical approaches as a basis for developing effective learning design plans and illustrates its use. The authors share their definition of a learning activity and taxonomies for the constituent elements. Real examples are discussed to illustrate their approach. All JIME articles are published with links to a commentaries area, which includes part of the article's original review debate. Readers are invited to make use of this resource, and to add their own commentaries. The authors, reviewers, and anyone else who has 'subscribed' to this article via the website will receive e-mail copies of your postings.


Introduction
The use of Information and Communication Technologies (ICT) to support all forms of learning has expanding exponentially in the last decade.The perceived benefits of elearning include the opportunity to learn anytime, anywhere and to communicate and collaborate virtually across countries.For teachers e-learning is seen as having the potential to reach new student markets, offering more flexible learning opportunities for students, facilitating the tracking of student progress and activities, as well as providing an opportunity for creating new and innovative learning environments.E-learning is now no longer a peripheral activity, the province of the isolated enthusiast, but is pervading Higher Education (HE), not just as an effective infrastructure for distance courses but blended with more traditional approaches on campus.Indeed, Carr-Chellman & Duchastel (2000) went as far as to term it a "new online paradigm [which] calls not so much for providing instruction at a distance, as for making available learning resources and instructional activities to students."In effect, the context is a potentially rich learning environment, where "students are typically engaged in multiple activities in pursuit of multiple learning goals, with the teacher serving the role of coach and facilitator."(Wilson, 1996) There are now a plethora of online learning environments and associated tools to support teaching and learning (Conole, 2004).These include communication (email, discussion boards, synchronous chat), authoring and assessment tools, as well as integrated learning environments such as Blackboard and WebCT.Similarly, there is a range of tools to facilitate searching.Despite this, uptake in the use of these technologies within higher education has been fragmented and slow (Conole, 2004;Littlejohn and McGill, 2004).A recent report concludes 'amongst the factors that are slowing the uptake…is the lack of a coherent framework within which to evaluate both the pedagogical benefits and the organisational changes required to effectively implement it' (Britain and Liber, 2004).Research to date shows that it is difficult to encourage authentic virtual learning or collaboration; discussion board use, for example, often shows a pattern of peak use directly related to teacher intervention or responses to particular 'hot' topics.Collaborative group work needs to be carefully set up and orchestrated to achieve desired results and despite this may still end up as a rather stilted collaboration exercise not comparable with direct face-toface equivalent group work (Jones, 1999).Integrated learning environments are still predominantly used as shells for displaying web pages and rarely get beyond basic information dissemination and administration (Thomas and Wyatt, 1999).With respect to information seeking and handling, the sheer volume available to learner/researchers is increasing exponentially, unmatched by the sophistication of the searching and handling tools (Lawrence and Lee Giles, 1999).Information overload, coupled with confusion of where to look, is increasingly problematic and, despite a growth in the range of searching tools and portals, it is not evident that the right information is being dispatched to the right users in a timely and quality assured fashion (Conole, 2002).
The paper begins by considering the reasons for the lack of uptake of ICT to support learning and teaching.The paper describes a learning design toolkit which aims to support practitioners in the creation of pedagogically informed learning activities.

Skills for e-learning
Despite the variety of ICT tools and resources available and the recognized potential benefits of using these to support teaching and learning, many practitioners lack the necessary e-learning skills to take full advantage of the potential affordances (Conole and Dyke, 2004) that these technologies offer and complain that support and training in this area is inadequate (Oliver, McBean, Conole and Harvey, 2002).Furthermore, despite the fact that many described instances of e-learning claim to draw upon theoretical positions, such as constructivism, many do not explain how they embody the principles and values of that approach (Oliver et al., 2002).As a result many designs reflect 'commonsense' rather than theoretically informed design (Conole, Dyke, Oliver, & Seale, 2004).A more theoretically consistent approach to learning design is needed which inter-relates theory with the desired features of learning, and then maps relevant tools and resources (both human and technical) against these.This approach makes the relationship between practice and underpinning theory more explicit and, we argue, should enable practitioners to make more theoretically informed choices of the tools and resources used to support learning.
Few academic staff have had the opportunity to develop the pre-requisite expertise to design and implement an effective strategy for acquisition, use and evaluation of either new materials or methods of delivery.In order to provide these practitioners with support and encouragement, easy-to-use guidelines and resources are required.Not only must these be based on sound, tested pedagogical theories but, first and foremost, they must be practical for academics using (or evaluating the use of) learning technologies.One approach to addressing this issue is the use of toolkits which help guide the practitioner through a series of pedagogically informed decisions in the design process.
There has been an increasing interest in recent years in addressing these problems and in particular for developing mechanisms for facilitating the uptake and effective use of ICT and the repurposing of existing resources.Simple models and frameworks have proved popular such as the e-moderating framework for setting up and managing discussion groups (Salmon, 2000) and study guides such as those produced by the LTSN generic centre (LTSN, 2002) and the Association of Learning Technology (Seale and Ruis Rui, 2002).However, as a recent review of this area concludes (Mayes and De Freitas, 2004), there is little in the way of concrete e-learning models and, arguably, there is little in the way of innovative practice (Oliver, 2004).Furthermore, some would argue that there is a surface approach to the use of these guidelines and models and, despite the hype surrounding the potential for re-use and repurpose, the reality is that it is fraught with difficulties (Littlejohn, 2003).

Situating learning design
Some authors contend that design for computer based or online learning is rooted in behaviourist theories of knowledge acquisition (Gagne, Briggs, & Wager, 1974), whilst others argue that the needs of different learning styles are better served by a constructivist approach (Honebein, 1996), most particularly in "multimedia based lessons" (Clark & Wentworth, 1997).The range of approaches based on behaviourist and/or cognitive theories is usefully summarised by Meyer (1998).Doolittle (1999) graded the ability of online education to meet eight primary requirements of a constructivist pedagogy and concluded that "Overall, online education provides the resources necessary for students to engage in rich and effective construction of knowledge.The key to online education and constructivism is not whether or not the potential exists, but rather, whether or not the potential will be actualised."Pragmatically, it is clear that "designers and instructors need to choose for themselves the best mixture of behaviourist and constructivist learning experiences for their online courses" (Carr-Chellman & Duchastel, 2000).There is also a mix of the online paradigm with traditional (offline) experiences, a combination now termed blended learning.In creating online learning experiences, designers and instructors are increasingly able to draw on digital resources available from web-based repositories, or stored in offline media.As facilitators of these experiences they can use asynchronous or synchronous communications technologies.As evaluators of the outcomes of the learning they may combine on and offline assessments.
As we move beyond the innovation and early adopter phase of this paradigm, there is increasingly a need and a willingness to share both resources and effective e-learning design with colleagues.Hicks, Reid, & Rigmor (2001) suggested that "quality depends on the way technology is used to provide access to relevant learning opportunities at the optimum time."(p.144).They provided a useful table of the characteristics, learning opportunities and learning demands of online environments.Hedberg (2003) made a number of important points about design for quality in e-learning.These include fostering learner engagement, enabling multiple roles for both learners and tutors, and appropriate focusing of assessed tasks.He claimed that well designed e-learning offers learners the opportunity to "gain a greater understanding of their own experiences than those remaining in the classroom expecting that the 'knowledge' will be given to them."(p.179)With respect to learner engagement, Laurillard (2002) distinguished between 'sit forward' interactive media and 'sit back' narrative media (p.110) and suggested that they do not combine well (p.136).She concluded that "improvements in university teaching are more likely to be achieved through 'multiple media', appropriately balanced for their pedagogic value, than through reliance on any one learning technology."(p.174) The increasing availability and use of online, digital resources to support teaching and learning is stimulating a convergence between the fields of learning design and learning object technologies.Indeed, in some quarters, the reusability debate has moved on from how to label digital objects so that other people, or systems, can find and use them, to how to describe "a whole learning experience" so that it can be "tweaked" for use elsewhere (Kraan, 2003).IMS learning design specification is concerned with articulating learning design processes.This was developed in part to shift attention from a focus on content to process.Koper and Olivier (2004) argue that e-learning specifications (such as the IEEE LOM) consider learning in terms of a resticted pedagogical perspective namely that 'in order to learn, a single learner has to work through a sequence of learning objects' therefore suggesting that learning is a process of consuming of content.They acknowledge that current educational practice is more complex and advanced than this and the learning design specification was developed to reflect this.Learning design is defined as an application of a pedagogical model for a specific learning objective, target group and a specifc context or knowledge domain.The learning design specifies the teaching and learning process, along with the conditions under which it occurs and the activites performed by the teachers and learners in order to achieve the required learning objectives.LD is based on the metaphor of learning as a play instatiated through a series of acts with associated roles and resources.The core concept of LD is that a person is assigned a role in the teaching-learning process and works towards certain outcomes by performing learning activities within a given environment.The environment consists of appropriate learning objects and services used during the performance of the activities.See Koper and Tattersall (2005) for a recent overview of learning design.
With the emergence of the Learning Design specification (IMSLD, 2003), a number of applications are now being offered to guide users through the learning design process and help them create effective learning activities with pedagogically informed use of tools and resources, such as the method and tool described by Paquette et al. (2005).A further example is the Learning Activity Management System (LAMS) which guides practitioners through the process of learning design (Dalziel, 2003).Using a 'drag and drop interface', users can pick and mix different types of learning activities.Littlejohn and McGill (2004) have identified lesson plans as an important category of resource which are used commonly in schools and FE (and to a lesser extend in HE).They state that resources 'are particularly useful within the context of staff development, since they give insight into different educational approaches'.Therefore tools like LAMS are useful in terms of guiding practitioners through the production of these lesson plans.
Another mechanism for supporting practitioners through the different approaches and theories associated with promoting effective learning design is through the use of a 'toolkit' such as described in this paper.The next section defines the concept and rationale for toolkits, followed by a more detailed description of a specific toolkit to support the development of learning activities.

Toolkits as a mechanism for supporting practitioners
Most commercial software now comes with some form of in-built help system and reference manual.In addition many also provide templates or how-to wizards to guide the user through a particular set of activities.Word's paper clip and PowerPoint's template presentations are good examples.As a consequence, practice has shifted from a culture of reading the manual of instructions to a just in time culture based on immediate need.This is echoed in research into children's use of gaming software which shows that they learn to use these applications through trial and error and by exploring the package rather than referring to the manual (Sutherland, Keri, Furlong, & Furlong, 1999).A wizard is a software tool that makes decisions on behalf of the user, based on solicited information and drawing on pre-defined templates.In most cases, the way in which the outputs from a wizard are generated is hidden from the user.As a result, they are easy to use, but restrictive in terms of the type and variety of potential outputs from user interactions with the tool.Another category of support tool is toolkits, which are decision-making systems based on expert models, positioned between wizards and generic conceptual frameworks which can provide a theoretical overview of an area and hence be used as a point of reference for decision making (Conole and Oliver, 2002;Oliver et al., 2002).
Frameworks, toolkits and wizards lie at different points along a continuum, with open but unsupportive theoretical maps at one end, and restrictive but easy to use software 'black boxes' at the other.No value judgement is made about which of these points is 'best' for users; clearly, each is suited to supporting users with different needs and varying levels of expertise.By definition, all toolkits include an expert model of a process derived from recognised theory and best practice.This provides a manageable process, supporting the implementation of performance monitoring systems.Furthermore, by providing a common conceptual framework (particularly one in which multiple interpretations of terms can be negotiated and agreed), it becomes possible to define and establish standards.
A toolkit provides a structured resource that can be used to plan, scope and cost an activity (examples include the development of an evaluation plan, choosing and integrating different types of media into teaching, or managing information).By providing increasingly detailed layers of information, the user can follow up relevant issues when and if such detail is required.In addition, by providing a simple, logically organised structure toolkits help to reduce the time required to plan work of this type.Toolkits are designed to be easy-to-use for practitioners; provide guidance, but not be prescriptive; be adaptable and easy to customise to the local context; provide a comprehensive resource of relevant material; and provide demonstrable benefit to users.

Methodology
The methodology adopted in developing the learning design toolkit described here follows the approach adopted in the development of previous toolkits (Conole and Oliver, 2002;Oliver et al., 2002).There are five main strands to this approach: 1. Work closely with practitioners to analyse their methods, when creating or repurposing resources, and be guided by their requirements.
2. Enshrine good practice within the toolkit, such that it will guide and support teachers as they create, modify, and share teaching and learning resources.
3. Research, understand and apply what is going on in the learning design field, particularly evolving standards in the areas of sharing digital resources, interoperability, searching, re-purposing, and permissions.
4. Embrace new technologies, such as adaptive hypermedia and semantically structured metadata, to provide a tailored development environment, accessing heterogeneous data repositories across a grid service infrastructure.
5. Develop, test and evaluate a prototype toolkit with practitioners and then revise in light of feedback.
A series of interactive interviews with practitioners helps articulate the key components involved in their learning design processes and in particular helped identify areas where further support was required.Some of the activities enhanced existing courses, whilst others involved the creation of completely new courses.These discussions and detailed analysis informed the initial requirements analysis for the toolkit.
The learning design toolkit described can be used for three main purposes: 1.As step-by-step guidance to help practitioners make theoretically informed decisions about the development of learning activities and choice of appropriate tools and resources to undertake them.
2. As a database of existing learning activities and examples of good practice which can then be adapted and reused for different purposes.
3. As a mechanism for abstracting good practice and metamodels for e-learning

A definition for learning activities
At the heart of the toolkit is the notion of a learning activity (LA), which we define as consisting of three elements: 1.The context within which the activity occurs, this includes the subject, level of difficulty, the intended learning outcomes and the environment within which the activity takes place.
2. The learning and teaching approaches adopted, including the theories and models 3. The tasks undertaken, which specifies the type of task, the techniques used, associated tools and resources, the interaction and roles of those involved and the assessments associated with the learning activity.
This is represented diagrammatically in Figure 1 and each element is briefly described and exemplified below.Each of the elements with a (+) in the diagram has an underlying taxonomy or set of instances.

Figure 1: Learning Activity -Top Level
The essence of a learning activity is that it must have one or more 'learning outcomes' associated with it.Learning outcomes are what the learners should know, or be able to do, after completing the LA; e.g.understand, demonstrate, design, produce, appraise.In order to achieve the intended learning outcomes there is a 'sequence of tasks' which must be completed.Examples of tasks are reading paper(s), discussing ideas, accessing database(s), extracting or manipulating data, answering questions, making decisions.The task 'type' taxonomy is shown in Figure 2, with one of the elements expanded to show the full tree, and other elements of 'task' are described below.

Figure 2: Task Type
Task techniques include brainstorming, exercise, field work, role play, reflection and syndicates.We have identified almost thirty techniques to be stored in the toolkit such that advice can be offered to practitioners.Interactions required are likely to be individual, one to many, student to student, student to tutor, group or class based.When undertaking tasks participants in the learning activity (both teachers and students) are assigned appropriate 'roles', such as individual learner, group participant, or presenter.Some tasks are assessed; assessment types and techniques are shown in Figure 3.

Figure 3: Assessment
'Resources' such as web pages, databases, video streams or interactive maps, may be included.'Tools', essentially facilitating applications, may also be needed; for example search engines, discussion boards, spreadsheets, or media players.Within the toolkit, taxonomies for resources and tools have been based on Laurillard's five principal media forms (Narrative, Communicative, Adaptive, Productive, and Interactive) (Laurillard, 2002, p.90).Narrative media tell or show the learner something (e.g.text, image).Interactive media respond in a limited way to what the learner does (e.g.search engines, multiple choice tests, simple models).Communicative media facilitate exchanges between people (e.g.email, discussion forum).Adaptive media are changed by what the learner does (e.g.some simulations, virtual worlds).Productive media allow the learner to produce something (e.g.word processor, spreadsheet).Based on the classification of media in the five categories, and the desired learning outcomes and strategies, the toolkit suggests appropriate media types and combinations as a teacher designs a learning activity.
The tasks and associated roles undertaken to achieve the prescribed learning outcomes occur within a particular context with characteristics which include a description of the subject domain (e.g.Physical Geography), the level (e.g.introductory), the perceived skills Conole, Fill (2005) which will be used or acquired (e.g.numeracy, critical analysis), the time anticipated for completion of the activity (e.g. 2 hours), and any associated prerequisites (e.g.first year course completion, database skills).
A central premise of this approach is that e-learning is ideally "centred on the set of student tasks … that constitute the learning experiences that the students will engage in, either independently or collaboratively, in order for them to master the objectives" (Carr-Chellman & Duchastel, 2000), p. 234).In designing a learning activity a teacher usually has a linear sequence of tasks in mind but, especially in an online learning environment, learners will not necessarily follow that sequence.Indeed an early project experience flagged up the need to enable learners to move easily around the resources and tasks.
Similarly, one of the drawbacks of the content manifest and organisation approach adopted in the useful, and increasingly popular, Reload tool (Reload, 2004) is that it presupposes and enforces a linear approach.
In addition to context and tasks, the toolkit includes taxonomies and models for learning and teaching approaches based on a review by Mayes et al. (2004) which groups learning theories according to whether they are associative (learning as activity), cognitive (learning through understanding) or situative (learning as social practice) (Figure 4).We have started to populate the toolkit with theorists and models, and anticipate two main benefits.The first is to offer guidance to practitioners, as they create or modify learning activities, by suggesting possible approaches and offering them links to further information and examples of how these can be used.The second is to store and subsequently investigate the approaches actually selected.

Figure 4: Learning and teaching Approaches
In the course of deriving and refining the definitions and typologies, life examples were modelled as learning activities and discussed with teaching colleagues.The representation proved both robust and informative.One activity was for individual learners and entailed reading two online documents, answering questions in a multiple choice quiz and receiving automatic feedback.The other involved learners viewing a wide range of online resources, data, text and images, accessing a public database, manipulating extracted data using a spreadsheet, deriving and submitting numeric answers, selecting and ranking choices from tutor supplied drop-down lists, composing and submitting written critiques.Tutors were involved in responding to learner comments and questions via an asynchronous message board and in marking written submissions.Numeric answers and some multiple choice selections were automatically marked.
Once these learning activities had been modelled, it was clear that the resulting information would enable other teachers to review them quite swiftly and decide whether to adopt or adapt them.Furthermore, the modelling of resources and tools facilitates re-purposing.For example, the first activity was based on two documents internal to one of the US partners.
It would be easy for the other partners to substitute their own documents.In the second activity, all the resources pertained to river habitats in the UK.US colleagues could substitute indigenous resources without changing the sequence of tasks and outcomes the learners are required to undertake and produce.It would be possible to replace some of the individual analytical tasks with group work, and so on.
These modifications are conceptually possible but remain complex in terms of technical infrastructure.A challenge for the toolkit developers is to resolve interoperability issues.They are aware of, and involved with, research and development in this field.The current project offers an excellent proving ground for potential solutions, as all four partners have different virtual leaning environments.Copyright and permissions processes may also prove testing.From a pedagogic point of view, there are also limitations and issues yet to be resolved.These include measuring tacit knowledge, abstracting out models, capturing the dynamics and evaluating the quality of a learning activity.
An online version of the toolkit is available at http://www.nettle.soton.ac.uk/toolkit/Default.aspx.Finally, for the toolkit to be useful, to a wider community than the DialogPlus project, as a searchable database of learning activities/resource, it needs a certain critical mass.The developers are aware of current efforts in the fields of interoperability, metadata, digital libraries and permissions and are resolved to design and test potential solutions within the context of their current project.

Conclusions
An online version of the toolkit is now available which is populated with a range of learning activities across different theoretical perspectives.A detailed comparison of the DialogPlus taxonomy with IMS Learning Design and other pedagogical taxonomies (such Conole, Fill (2005) as the LTSN's vocabulary http://www.ltsn.ac.uk/genericcentre/index.asp?id=19232; the Source and SESDL project taxonomies http://www.source.ac.uk and http://www.source.ac.uk -see Beetham, 2004 for more details on pedagogical reviews) has also been carried out.
We have defined the concept of a learning activity as the underpinning for the development of a learning activity design toolkit which aims to provide easy to use guidelines for practitioners to make pedagogically informed decisions on designing learning activities and making choices about the use of appropriate tools and resources to support this.A learning activity occurs within a context with a set of associated attributes and addresses a set of learning outcomes.Learning outcomes are achieved through a sequence of tasks and associated roles adopted by the participants which might call upon a set of tools and resources.Some of the tasks are assessed.
Higher Education in the 21 st century can involve teachers and learners in a collaborative exploration of global resources.Learning activities that effectively utilise these resources are being created and can be shared, adopted or adapted.Teachers need support and guidance with respect to quality of resources and e-learning design, as well as methods for understanding, unpacking and repurposing existing offerings.The learning activity toolkit described in this paper has been specified, and is being developed, to meet these needs.

Acknowledgements:
JISC/NSF funding of the Digital Libraries in Support of Innovative Approaches to Teaching and Learning in Geography involving the universities of Penn State, Leeds, UCSB and Southampton (DialogPLUS) project.Dr Christopher Bailey, Learning Technologies Research Group, University of Southampton, for his thoughtful contributions to the design and excellent, ongoing work on the development of the toolkit.