Adaptation in e-learning can be defined as “a method to create a learning experience for the student, but also for the tutor, based on the configuration of a set of elements in a specific period aiming to increase the performance of pre-defined criteria” (Burgos, Tattersall, & Koper, 2007, p. 162). The source of adaptation can vary in nature from being invoked by the machine (‘adaptivity’) to being enacted by users (‘adaptibility’) (Burgos, et al., 2007). There are several benefits of adaptation in e-learning that primarily centre around the ability to personalize learning to meet the requirements of individuals, especially in light of increasing student diversity, at times massive student numbers, and the large array of modalities afforded by contemporary learning technologies (Paramythis & Loidl-Reisinger, 2004).
Adaptation can take many forms, including interface based, learning flow based, content based, interactive problem solving support, adaptive information filtering, adaptive user grouping, adaptive evaluation, and changes on-the-fly (Burgos, et al., 2007). A considerable amount of work has been undertaken to address the way e-learning standards can support adaptivity so that adaptive learning systems are interoperable and adaptive learning resources are reusable (for example, see Paramythis & Loidl-Reisinger, 2004).
However there has been less work examining how learning designers (humans) might adapt the environment and tasks during lessons in order to support emergent learning needs. In fact, historically there has been a degree of vagueness about whether learning design is something that only occurs before a class occurs or whether it can take place during lessons. This paper argues that learning design can and should happen during live lessons, in order to optimise the learning experience of students. Always relying on pre-emptive design fails to acknowledge the true nature of learning and teaching, in which teachers need to modulate the environment based on the conceptions (and misconceptions) that their students present (Laurillard 2012). A failure to adapt lessons during runtime increases the possibility that the learning design does not meet student learning requirements.
For the purposes of this paper, adaptive learning design is defined as follows:
Adaptive learning design is the process whereby educators strategically modify a learning design during lessons in order to meet the emerging requirements of learners.
The definition above implies that providing direct answers to student questions in class would not constitute adaptive learning design, because it is part of the learning sequence and is not strategic in nature. On the other hand, responding to a student question by re-configuring the learning environment or switching the type of task that students complete would be considered adaptive learning design under the definition above.
The live nature of adaptive learning design requires that teachers rapidly and appropriately adjust the environment, which increases the importance of having context specific design patterns and heuristics from which they can draw. Teachers are often able to identify that students are experiencing difficulty with certain concepts or processes or tasks, yet are unable to dynamically and strategically adapt the environment because they do not possess pedagogical schema that relates design patterns to student ability levels or knowledge types. Adaptive design can be challenging in technology mediated learning environments because educators not only need to know how to adjust the nature of the task and associated explanations, but also how to redesign and deploy the technological resources at their disposal.
That adaptive design occurs in real-time is a consequence of adaptations needing to be made within lessons that are being enacted, and thus it relates more to an ‘activity’ level of granularity. This implies that ‘atomic patterns’ which contain more specific information about the enactment of an activity or part of activity (Prieto, Dimitriadis, & Villagrá-Sobrino, 2011) may often be of more use than many well established learning design patterns that typically span across lessons or modules of work. For instance, as part of an adaptive redesign of the environment it may be appropriate to change the layout of an interface or a task that students complete, rather than embarking on an entirely new Think-Pair-Share or Predict-Observe-Explain learning design. Thus adaptive learning design does not imply teachers will replace an entire lesson with a new, but redesign parts of a lesson based on emergent needs.
Some learning technologies have foreseen the educational importance of adaptive learning design, and have incorporated it into their functionality. For instance, LAMS (LAMS International, 2014) not only allows teachers to design collaborative learning experiences for their classes and monitor their students’ contributions, but also enables teachers to dynamically adjust the learning design during the lesson via the “Live Edit” feature in light of unfolding student responses.
Web-conferencing systems are another class of learning technologies that allow ‘on-the-fly’ adaptation, and are the focus of this study. Adobe Connect (Adobe Systems Inc, 2014), Blackboard Collaborate (Blackboard Inc, 2014) and WebEx (Cisco Inc, 2014) allow a range of rich-media tools to be integrated on demand, offering previously unavailable possibilities for instantiating synchronous online learning experiences. Voice over IP, text-chat, whiteboards, screen-sharing, communal note areas and so on provide a powerful suite of tools with which to present information, model processes, and share concepts. Tools can be inserted, resized and removed during a live learning episode. However the flexibility of web-conferencing systems raises a raft of design questions. If teachers can adjust the size and placement of the tools in a web-conferencing environment, on what basis should they do so? What factors should influence the tools that are selected? How should the tools be arranged?
Approaches to applying a transmissive or ‘teacher-centred’ style of teaching in web-conferencing environments are self-evident. A low level of interpretation is required to understand how teachers can use such systems to present information to students since this is a typical use for which these systems are designed. However solely applying transmissive teaching approaches is insufficient because to learn effectively people need to be engaged in learning activities (Britain, 2007). Contemporary technologies afford the potential to improve the quality of learning by engaging students in an interactive dialogue (Laurillard, 2012) and facilitate collaborative problem solving (Jonassen, Lee, Yang, & Laffey, 2005).
This paper presents findings from a three-semester design-based research study that investigated how to adaptively design an online multimodal (web-conferencing) environment to meet the evolving cognitive and collaborative requirements of learning activities. Capturing and sharing successful patterns of collaboration and meaning making in web-conferencing environments is particularly useful because compared to single modality technologies (such as discussion boards or straight text-chat) design possibilities are more complex, and thus leave greater possibility for sub-optimal design by novice designers. While the study related to teaching graduate students computing through a web-conferencing platform, the outcomes of the study provide a framework to make sensible adaptive learning design decisions for a variety of learners, subject areas, and multimodal learning systems.
Designing online environments to facilitate context sensitive learning requires consideration of the type of information being presented, the type of pedagogy being applied, and interface design principles to mediate these concept representations and interactions. The literature reviewed below provides a conceptual frame and nomenclature for analysis and results to follow.
Knowledge representation in web-conferencing environments
Web-conferencing systems offer a wide array of modalities for facilitating collaboration and meaning making. Chat tools and notes areas can be used to communicate written text, Voice over IP enables audio-based discussion, and screen-sharing and whiteboard tools allow visual information to be exchanged. Kress et al. (2001, p. 2) point out the importance of a deliberate approach to deciding upon the modality of representation because of its impact on the effectiveness with which meaning is shared:
Making a representation now goes well beyond simple encoding. It has become a matter of active, deliberate design, and meaning making becomes a matter of the individual’s active shaping and reshaping of the resources that he or she has available, in the wish to make representations match intentions as closely as possible.
There are several types of knowledge that may be represented in online environments:
—(Anderson & Krathwohl, 2001, pp. 27–29).
- Factual (declarative) knowledge – discrete pieces of elementary information, required if people are to be acquainted with a discipline and solve problems within it
- Procedural knowledge – the skills to perform processes, to execute algorithms and to know the criteria for their appropriate application
- Conceptual knowledge – interrelated representations of more complex knowledge forms, including schemas, categorization hierarchies, and explanations
- Metacognitive knowledge – knowledge and awareness of one’s own cognition as well as that of other people.
If there are several available modes by which to represent knowledge and they each have different characteristics, the decision needs to be made about which modes to use for different types of knowledge. Note that metacognitive knowledge may be represented in either factual, procedural or conceptual forms, and as such will be subsumed into the other levels for the purposes of this paper.
Just as there is a variety of knowledge that may be represented, there is also a range of interactive patterns that may be instantiated. Ruth Clark (2005) identifies three types of collaborative designs for e-learning:
- Teacher-centred (receptive) – transmission based information delivery approaches, where the teacher communicates a stream of information to students. For instance the expert modelling approaches encompassed by Collins, Brown & Holums’ cognitive apprenticeship model (1991) applies a teacher-centred pedagogy.
- Teacher-led (directive) – small chunks of content, examples or demonstrations presented by the teacher, followed by periods of student activity including practise, feedback and interaction with the teacher and peers. This is more in line with Laurillard’s (2002) Conversational Framework, a variant of which has been successfully applied in the field of computing (Waite, Jackson, & Diwan, 2003).
- Student-centred (guided discovery) – more inductive learning environments in which learners complete a series of goal-related tasks. Collaborative learning pedagogies are often based on this approach, whereby student-to-student flow of concept forming discourse is central and the teacher adopts a more facilitatory role (Jonassen, et al., 2005).
Clark (2005, p. 594) suggests that different approaches to instruction may be appropriate for different levels of knowledge and understanding:
the effectiveness of any one e-learning lesson is shaped by the context in which it is deployed. Thus a given lesson that is effective for novice learners who need to build specific procedural skills will be less effective for more experienced learners who need to build mental models that they can apply to diverse situations.
A critical aspect of designing the environment is ensuring the interface can facilitate the type of collaboration intended. If the mediating technology cannot support the frequency or type of communication, then a form of ‘distributed process loss’ may result, which can render collaboration ineffective (Neale, Carroll, & Rosson, 2004).
Because in education the interface is responsible for supporting the ‘forming’ of concepts rather than merely ‘informing’ existing ones, some researchers have posited that educational user interface design requires its own approaches and specialized theoretical HCI frameworks (Rappin, Guzdial, Reallf, & Ludovice, 1997; Sedig, Klawe, & Westrom, 2001). A range of research in the field of education informs the effective design of interfaces for learning.
Educational interfaces that engage students at the level of concepts rather than just operations on objects can result in more effective learning (Sedig, et al., 2001). This supports the development of more generic problem solving skills, allowing students to apply their learning beyond the specific context in which they are being used (Johnson & Hyde, 2003; Salomon, 1992). Providing students with spatial flexibilities to resize an re-arrange icons, objects and emergent structure allows students to align the interface to meet the changing requirements of their thinking processes and intersubjective activities (Hollan, Hutchins, & Kirsh, 2000). The interface should also match the level of user ability, for instance by supporting more deconstructed, sequential operation for novice users as opposed to top-down composite operations for more experienced users (Vu, Hanley, Strybel, & Proctor, 2000).
Multimedia learning principles provide a scientifically grounded point of reference for designing interfaces in multimodal learning environments. Principles particularly relevant to dynamic design of the web-conferencing environment include:
- The ‘multimedia principle’ – people learn more effectively from words and pictures than from words alone (Fletcher & Tobias, 2005)
- The ‘modality principle’ – presenting some of the instructional content in visual mode and other parts of the material in auditory mode can lead to more effective learning than using text to supplement the visual information (Low & Sweller, 2005)
- The ‘split-attention effect’ – people learn less effectively when words and pictures are physically and temporally separated (Ayres & Sweller, 2005; Mayer, 2005b)
- The ‘redundancy effect’ – information should not be unnecessarily repeated in different forms, for instance by including textual explanations with a visual presentation that already incorporates an auditory explanation of the same content (Mayer, 2005b, p. 184; Sweller, 2005)
- Symbol System Theory – matching the modality to the nature of the information being communicated can reduce the level of elaboration and recoding required for learner comprehension (Salomon, 1994).
As well as efficiently representing knowledge, interfaces are responsible for supporting interaction and collaboration. Collaborative learning approaches are espoused to increase effort in selecting, organizing and integrating information in order to promote greater transfer of problem solving capacities (Mayer, 2005c). Interface designs can influence collaborations by determining the extent to which loose or tightly coupled interactions are possible (Neale, et al., 2004) and the type of information that is exchanged (Kress, et al., 2001; Salomon, 1994). Key principles for educational user interface design derived from socio-constructivist theory include:
- The interface should be a mediator for social construction of artifacts and meanings ('Situated Cognition Theory', Wilson & Myers, 2000). In this way the interface should allow for integration of levels of scale – viewing individual cognition within the larger physical and socially constructed context of interactions, tools and meanings (Wilson & Myers, 2000).
- Interfaces should be designed to support development at the level of the learner’s abilities (their 'zone of proximal development', Vygotsky, 1978). Thus scaffolding such as cues, heuristics, hints, examples, etc. should be provided through the interface when required and without interrupting the flow of learning.
- In identifying understanding as being shared between people and artifacts rather than residing entirely within the minds of individuals, interfaces should allow for cognitive processes to be distributed across group members and facilitate interactions between internal and external knowledge representations (in alignment with 'Distributed Cognition theory', Hollan, et al., 2000).
- Interfaces should allow participants to gauge the orientation and frame of reference of others as a way to coordinate more effective interaction with peers and the environment (Luff, et al., 2003).
Elaboration of these educational user interface design issues with relation to the design of web-conferencing environments are outlined in an earlier paper (Bower & Hedberg, 2009). While they operate at a more micro level than some of the ‘higher order’ issues confronting the Learning Design field (for instance, the way in which learning design systems can be best structured to assist educators), the results and effects described above provide an evidential basis for the creation of specific web-conferencing interface patterns.
This design-based research study traces how the level of knowledge being represented, the type of pedagogy being applied and a variety of interface design principles were used to form an integrated framework for adaptive design in a web-conferencing environment.
A design-based research approach
Design-based research is a methodology advocated by many researchers to attend to the complex and messy nature of real-world learning design problems (Barab & Squire, 2004; Cobb, Confrey, diSessa, Lehrer, & Schauble, 2004; Lesh, 2003). While design-based research is not strictly considered to be a Learning Design concept or practice, its typical focus on improving learning and teaching in technology enabled environments means that it is often used in learning design contexts. Through iterative cycles of theory-based reflection, analysis and refinement, design research aims to produce guidelines and principles to address authentic e-learning problems (Herrington, Reeves, & Oliver, 2010). A design-based research methodology was adopted in this study for its ability to:
- incorporate the influence of authentic settings and the potentially multifaceted nature of outcomes (Barab & Squire, 2004; Lesh, 2003)
- provide insights into the complexity of learning processes not just learning products (The Design-Based Research Collective, 2003)
- integrate the teacher as participant researcher, removing the artificial line between researcher and subjects (Lesh, 2003) and promoting greater methodological alignment (Hoadley, 2004)
- align with the cyclic and iterative nature of authentic design of learning environments (Lesh, 2003)
- encourage theory generation about teaching and learning (The Design-Based Research Collective, 2003)
- value all types of data to arrive at an effective design, for instance reflections of teachers as designers and lessons learnt from ineffectual designs (Gorard, Roberts, & Taylor, 2004; Wilson, 2004).
Enactment of the iterations
Data for this study was based upon teaching an introductory computer programming subject to graduate students through a web-conferencing environment over three semesters from Semester 2 of 2005 to Semester 2 of 2006. At the risk of oversimplification the strategic redesigns across semesters can be summarized as follows:
- Iteration 1: Predominantly instructive approaches primarily based upon standard graphical user interface designs. These included presentation of tutorial solutions and demonstration of programming processes. The teacher used audio and student contributions were limited to text chat responses. Changes to the layout were not premeditated or designed on a principled basis. The main way in which the interface was adjusted to meet the needs of the learning episode was by switching between the three default layouts. This iteration offered a baseline for the design-based research analysis.
- Iteration 2: Use of more student-centred learning approaches requiring students to contribute and collaborate. For instance several activities required students to perform collaborative programming processes using the web-conferencing tools. In these cases the environment was redesigned to meet the collaborative requirements of the learning episodes. Dynamic design attempted to draw upon principles identified in the literature as well as emerging needs evidenced by activity in the learning environment. As well, student feedback (either solicited or unsolicited) regarding the efficacy of interfaces was often utilized to apply these dynamic redesigns.
- Iteration 3: Frequent spontaneous integration of whiteboards to support conceptual understanding. Audio was used pervasively in an attempt to promote more cognitively efficient interaction through the use of multimedia learning principles (Mayer, 2005a). Iteration 3 was also an attempt to refine adaptive design approaches adopted in Iteration 2 and to repeat designs in order to gauge the consistency of their effects.
Twelve two-hour classes (sessions) were conducted for each of the three semesters using the same curriculum materials. The same text-book, recorded lectures, and tutorial and practical sheets were used across the three semesters in order to provide a consistent curriculum across the three iterations and hence reduce the possibility of intervening environmental effects. In order to avoid persistent effects of interactions between particular individuals, an effort was made to mix the membership of any groups as far as possible throughout each semester.
Observations both between lessons and iterations informed the refinement of redesigns. By providing students with the access control they could also dynamically redesign the interfaces based on their perceived needs. In such cases the teacher could then question students as to the rationale for their changes, providing a more complete body of evidence for how interface designs in the web-conferencing environment affected learning. Note that no redesigns occurred in the first week of Iteration 2 and Iteration 3 to provide a means of calibration between iterations and allowing the consistency of collaborations for the same learning designs across different cohorts of students to be confirmed.
Approach to analysis
Throughout the study a project database was maintained that included recordings of all sessions, artefacts arising from the learning activities, student feedback (both anecdotal ad from structured surveys), and a reflective journal. All observations relating to cause and effect relationships occurring within the learning episodes were noted, with particular emphasis upon how different designs engaged different levels of interaction and enabled concepts to be more effectively represented. Both successes and failures were documented, with the failures seen as making an important contribution to understanding teaching and learning in the web-conferencing environment.
The analysis contained in the reflective journal notes along with lesson artefacts and student feedback were then used as a basis for redesign. Strategic redesigns occurred across iterations as described in the ‘Enactment of Iterations’ sub-section above, while tactical redesigns occurred within iterations by focusing on incrementally improving the way in which the affordances of the web-conferencing environment were applied. The influence of both strategic and tactical redesigns on collaboration and learning were added to the project database through their inclusion in the reflective journal. Particular attention was dedicated to the way in which theories of multimedia and collaborative learning could inform the redesign of the environment. Examples and explanations of this reflective analysis and redesign process are provided in the Results section.
Reliability in this study was promoted through empirically based explanation building involving prolonged engagement and persistent observation. The three-semester design-based research process as well as the multiple instances of implementing designs within each iteration allowed explanations for effects to be constructed and validated via repeated observation. In order to promote validity multiple sources of data were used to form conclusions (lesson recordings, teacher observations, student feedback) and the analysis was critically reviewed by a number of national and international experts as part of a doctoral program.
Characteristics of the Learning Environment (Context)
The learning domain
This study is conducted in the context of teaching and learning introductory computer programming as part of an online Graduate Diploma of Information Technology (GDIT) at Macquarie University. As part of the semester long subject students completed a two-hour weekly online class where they learnt the fundamentals of writing computer programs (in Java). As a four credit point graduate course the pace and amount of content covered was greater than an undergraduate introduction to programming course, covering basic programming syntax and semantics, objects and classes, polymorphism and inheritance, Applets and GUIs, arrays and ArrayLists, as well as error handling and file operations.
Students who undertook the online GDIT were graduate students (or students with commensurate professional experience that qualified them to enter the course) from a discipline other than computing that wished to extend their IT knowledge and skills. There were 26 students who enrolled across the three semesters that were analyzed in this study, of which 20 completed the unit. Of the 20 students who completed the subject, ten were enrolled in 2005 Semester 2, seven in 2006 Semester 1, and three in 2006 Semester 2. This decline in enrolments across semesters was commensurate with other courses at the University and other universities generally (Cassel, McGettrick, Guzdial, & Roberts, 2007). Of the 26 students, 9 were female and 17 were male.
The web-conferencing environment
The Adobe Connect Meeting platform (Adobe Systems Inc, 2014) was the web-conferencing system used in this study. Adobe Connect includes a range of tools through any Flash enabled browser including those to present documents, to broadcast webcam and voice, to screen share, to exchange text chat and files, to vote, as well as those providing a shared notes space and whiteboard. Each of these tools (or ‘pods’) can be instantly resized, drag-and-dropped, created or deleted. This provides the session host with ultimate choice over the tools that are provided within an interface and the way that they are arranged, allowing the interface to be dynamically redesigned to meet the evolving requirements of the learning episode. A default layout is shown in Figure 1.
Each room comes with three predefined layouts, which the host can switch between by toggling tabs at the bottom of the browser window. As well, new layouts can be created so that a room can have several pre-designed layouts or layouts can added on the fly. The meeting ‘host’ (or super-user) can spontaneously adjust the access control of ‘presenters’ and ‘participants’ to each of the tools. Copies of room designs can be created with relative ease and several rooms can run at once, providing separate working spaces for collaborative group-work. Finally, all sessions have the capacity to be recorded, on which basis the data for this study has been harvested, analysis performed, and results derived.
Results have been derived by organizing and describing key observations from the three semesters in order to demonstrate the evolving understanding of how adaptive design could be used to cater to the emerging cognitive and collaborative requirements of the learning episodes. Based on these observations, principles for design of multimodal synchronous online learning environments and the framework for adaptive learning design are described in the Discussion section.
Even though predominantly transmissive approaches were adopted in Iteration 1, there were still times where reasonable levels of student contribution arose, for instance in the form of responses to questions from the teacher. In cases where the number of contributions was high it was sometimes necessary to adapt the interface to better accommodate the amount of text-chat arising, even if individuals were only making short factual contributions. For example, in Figure 1 the teacher is presenting the solutions to tutorial questions using direct instruction approaches. Then at one point a degree of student participation was encouraged by asking them to contribute their answers to the factual style questions by typing responses in the text-chat pod.
It was noted that the small size of the text-chat pod reduced the capacity students’ text-chat contributions to be reviewed and monitored due to the small number of comments visible at any one time. As a result the teacher chose to enlarge the chat pod and place it along the bottom section of the browser window (see Figure 2).
In the lesson students indicated that enabling more text-chat contributions to be viewed at once represented an improvement to the interface. One student also made the suggestion to elongate the attendee pod in the interface shown in Figure 2 to utilise the empty space and to allow students to immediately see who else was in the room. This suggestion was subsequently enacted, again improving the ability for participants to interact. This episode not only illustrated how the size of pods could influence collaboration in the learning environment, but also how the students (as the end-user of interface) could be critical sources of in-situ interface design ideas. Student observations and design suggestions were derived from their actual attempts and experiences using the interface rather than from estimating the student experience in advance (as is the traditional context of the educational designer).
To facilitate the practical activities, the interface was often changed to desktop broadcasting mode so that the teacher could model programming processes (as exemplified in Figure 3). This allowed students to observe programming processes such as editing, compiling and debugging program code, offering them a ‘cognitive apprenticeship’ (Brown, Collins, & Duguid, 1989). The screen-sharing provided a modality that was able to dynamically represent the process information being shared, again representing information in a ‘cognitively efficient’ form (in accordance with Symbol System Theory, Salomon, 1994).
By switching to the screen-share layout at times when students had questions about programming processes the teacher was able to show them a range of practical skills relating to writing and running computer programs. This allowed students to efficiently develop their procedural knowledge in cases where they had little or no previous understanding of how to program. The teacher could use audio to provide insight into underlying thought processes in a more cognitively efficient manner than if text-chat was being used. However, the transmissive nature of the approach meant that students had little opportunity to practise their programming skills, and as such the teacher could not accurately diagnose student problems or provide remedial instruction.
Iteration 2 was characterised by redesigning the environment to facilitate more student-centred pedagogies. For instance, in the second online tutorial students were divided into two rooms and asked to construct a group answer identifying the classes, objects, instance fields, methods and local variables in a program. The interface was redesigned to provide the program in the middle column of the window, a communal solution space in the top right note-pod and a text-chat pod at the bottom right of the interface. A shared solution space was provided in order to allow activity to centre around students rather than the teacher (see Figure 4).
Students in both groups were able to complete this exercise in their teams, negotiating collaborative solutions to the factual knowledge task. The teacher was able to review the group-work room note pods and text-chat transcripts to identify any misconception and then attend to these once students had returned to the main room. As opposed to the teacher delivery approach that had been adopted in Iteration 1, the redesign of the environment had enabled far greater levels of student engagement and collaboration about the foundational subject matter knowledge.
In other instances students were provided with authentic programming process tasks that required them to build solutions in teams. In order to accomplish this the environment was redesigned to provide each group with their own room and increased permissions so that they could add and share their computer code. For instance, in Figure 5 below students were divided into groupwork rooms and asked to complete the following task:
Write a class TinCan that creates cylindrical TinCan objects and has a method to return the volume. Write a class TinCanTest to test your class.
Instructions were provided as part of the interface design to avoid split attention. The two most capable students in the class were selected by the teacher as the ‘facilitators’ to be guided by the other team members. However those two students had not been given prior instruction on how to facilitate such an activity, and as a consequence there were some skills that they lacked relating to effectively collaborate while using screen sharing. As well, the teacher had not considered that without audio they would not be able to respond to their peers using text-chat without switching focus from the IDE that they were broadcasting. If the teacher would have anticipated the collaborative requirements of the learning episode in advance then students could have been trained in the use of audio and directed to use it. Students in both groups were highly engaged by the student centred task. However their collaborative efficiency was compromised by the interface design and their incomplete technological skills, which were in turn rectified in future lessons by enabling student audio and providing them with prior practice in using it.
In order to allow students to more effectively work between multiple programming files the interface was often redesigned to incorporate several program files at once. For instance, in the activity shown in Figure 6 below, students were required to combine a resize circle program and re-centre circle program into one applet so that a circle could be both re-centred and resized. For this interface the resize and re-centre programs were displayed in note-pods, and a third note-pod column was provided for students to write their combined program. The layout allowed all the relevant information to be accessed from the one interface, resulting in less split attention than when the IDE is used. Once again the text-chat discourse allowed the teacher to easily review collaborations that had transpired in each room. The interface supported high levels of student contribution by providing spaces for people to collaboratively problem solve and discuss concepts. In this exercise Group 1 contributed 52 comments and Group 2 contributed 78 comments, which was several times more than in the more transmissive approach adopted in Iteration 1.
Once the relevant code from the re-centre and resize program had been incorporated into the solution space, the pods containing the code became redundant. This led to Group 2 also spontaneously deciding to maximize the pod containing the integrated program to cover the other obsolete note-pods (see Figure 7). This is an example of interface flexibility affording the capacity to dynamically adjust the interface to suit the changing collaborative and cognitive requirements of the activity (Hollan, et al., 2000), and by virtue of repeatedly using the web-conferencing tool throughout the semester students intuitively appreciated the changes that should be made.
While the approaches above enabled more efficient collaboration for programming processes, it did not directly respond to conceptual difficulties that students were experiencing.
In Iteration 3 whiteboards were used, often spontaneously, to support concept development. Whiteboards allow persistent visual presentation and interrelation of several items of information. For instance for the ‘Circle Combine’ activity previously described in Iteration 2 the starting programs were once again shown in note-pods, however students were slow to make progress on this task. The poor progress was partially due to their difficulty understanding mathematical concepts relating to centring the circle. In order to provide a clearer explanation of the coordinate geometry underlying the task, the teacher chose to spontaneously use a whiteboard to represent the situation (see Figure 8).
The whiteboard allowed audio explanations to be supported by visual means, leveraging the cognitive gains afforded by the multimedia principle (Fletcher & Tobias, 2005). The audio modality was easier to use in conjunction with the visual solution space and diagram than the text-chat modality (modality principle, Low & Sweller, 2005) and as such the text-chat pod was not used. Having the whiteboard next to the note pod enabled the programming code to be directly compared to the concept being addressed, allowing students to interrelate the syntax of the programming language with its semantic meaning. As a result of the adaptive redesign students were able to better understand the programming concepts with which they were working and henceforth completed the programming process expeditiously.
Whiteboards were often introduced to represent dynamic concepts. They allowed the teacher and students to step through programs and emulate their operations. For instance, at one point in the subject students were required to write a program that produced random permutations of the numbers from 1 to 10. Some students struggled with the activity and a student’s erroneous program was run using screen-broadcasting to demonstrate how it was (incorrectly) repeating certain numbers. Students still indicated uncertainty about the underlying logic of the program, so the teacher chose to use a whiteboard to allow the group to emulate the operation of the code (see Figure 9).
The whiteboard allowed the process of element extraction from a random position in anArray (originally containing numbers zero to nine in ascending order) and placed in a second array (while the last element in anArray is shifted to the gap created by the extraction). Students could perform the next step in the program in order to demonstrate their level of understanding. The numerous pieces of information could be represented and interrelated in a way that would have most likely caused cognitive overload (van Merriënboer & Ayres, 2005) if students were required to follow a purely auditory explanation. The public solution space allowed cognition to be offloaded to the environment (Hollan, et al., 2000). The approach allowed the program and general process of using arrays to perform selections to be comprehended by students whereas in previous iterations explanations had been poorly understood.
At other times in Iteration 3 learning commenced with conceptual discussion and progressed to more practical application of processes that applied those concepts. In such cases whiteboards could be used as a starting point to represent the conceptual information before transitioning to screen sharing to perform programming processes. For instance in order to develop students’ understanding of the concept of polymorphism the teacher broadcasts a whiteboard containing the four files of the Polymorphism program (see Figure 10). Having the four files in the program (interface, main method, and two classes implementing the interface) on the whiteboard enables students to relate the features of the code that reference between files, reducing split attention (Ayres & Sweller, 2005). Initially a transmissive approach was adopted, but midway through the explanation the teacher asked if students had any questions. This resulted in a number of questions, for instance about whether variable names needed to correspond between programs, whether constructors could be different for the two classes implementing the interface, and general questioning relating to strategic interface design.
In order to demonstrate their understanding the students were required to add a Colour interface to the original polymorphism example. This was conducted as a teacher-led programming activity, which differed from Iteration 1 and Iteration 2 where it was an independent activity and group-work activity respectively. A standard sharing interface was used with the teacher broadcasting the IDE (see Figure 11), and students were required to instruct the teacher what to do next in order to solve the problem. The teacher-led approach enabled a considerable amount of student questioning and contribution to be elicited, thus allowing the teacher to gauge that students ultimately understood the concept.
In Iteration 3 students were also provided with the opportunity to collaboratively negotiate concepts as a way of taking ownership over them. For instance Figure 12 shows how students were handed control of the whiteboard and required to visually represent the programming concept of a ‘deep copy’ after being provided with a diagram of a ‘shallow copy’. The approach encouraged students to collaborate with one another and the productive nature of the task allowed their mental models to be revealed. From the diagram they provided the teacher could immediately assess that students had negotiated a correct visual representation of the ‘deep copy’ construct. The fact that a diagram of a ‘shallow copy’ had already been provided meant that students did not need to spend time discussing how to depict objects and references (as conjectured by Or-Bach & Lavy, 2004). Students indicated that the approach clarified their understanding of the difference between shallow and deep copying.
In Iteration 3 students were given more guidance on how to share their screen and were encouraged to use audio throughout. This enabled students to demonstrate their programming process capabilities and collaboratively solve programming problems. For instance in an activity requiring the development of a program that allows users to select the input file at runtime using a JFileChooser, initially students used a notes-pod approach much like that in Figure 6 and Figure 7. This allowed students to rapidly combine their ideas with equal access to the solution space. However the approach did not enable them to collaboratively test their program and resolve errors in their code.
Subsequently a student volunteered to lead the debugging process, changing the interface so as to broadcast their screen (see Figure 13).
This allowed students to represent their procedural understanding through the activity performed as part of the negotiated desktop sharing process, as well as their conceptual understanding through their collaborative programming discussions. The teacher was able to perceive the extent of student understanding, and that together they had mastered the ability to apply the concepts to solve a real-life problem. The task was completed with the student leading the exercise, the group responding, and the teacher contributing occasionally.
While the descriptions and examples above cannot fully explicate the design-based research process, observations and results, it does provide an indication of the nature of the data collected and the type of analysis that was conducted. A more complete portrayal is available online for readers who may be interested (Bower, 2008).
The ability to select the combinations of modalities that will be used to facilitate interaction and knowledge representation in many synchronous learning environments means educational designers have a strong influence over the collaboration and learning that transpires. The ability to adaptively design some of these multimodal learning environments affords the potential to tailor the interface to meet the evolving collaborative and cognitive requirements of the learning episode.
On the basis of the observations and analysis conducted in this study, principles for design of multimodal synchronous learning environments are outlined below. This is followed by a framework for adaptive learning design, based on the level of interaction anticipated and the type of knowledge being represented.
Principles for design of multimodal synchronous environments
1. Select modalities that best suit the desired form of information representation
Different representational possibilities are afforded by different modalities. Evidence from this study suggested that the modality of representation should be selected to match the cognitive and collaborative requirements of the learning episode as follows:
- Text-chat – effective for simultaneous sharing of short pieces of factual information among a large group of contributors (for instance, when several participants are making suggestions about what to do next when writing a computer program)
- Audio – affords more rapid contribution of extensive descriptions by one person (for instance a team leader) or a small group of users (who are collaboratively designing) where turn-taking is occurring
- Note-pods – useful for organizing textual information between multiple users where sequencing, editing, copying and deletion is required (for instance, collaborative authoring of a solution such as a computer program
- Screen-sharing – useful for sharing process-based information with relation to computer usage, for instance when sharing how to perform a programming process or collaborative authoring on the fly.
- Whiteboard – effective for supporting shared representation and development of conceptual knowledge (for instance, drawing diagrams to represent the schematic design of a program)
2. Designing multimodal clusters according to multimedia learning principles to improve cognitive efficiency
Different modalities not only offer different individual possibilities for representing, but also different possibilities in combination as ‘multimodal clusters’ (Baldry & Thibault, 2006). The design of effective multimodal clusters was observed to rely upon application of multimedia learning principles (Mayer, 2005a). Examples include:
- Use of audio (rather than text-chat) in combination with visual modalities (such as note-pods, whiteboards or screen-sharing) during groupwork processes to avoid having to monitor two visual modalities at once (Low & Sweller, 2005).
- Using diagrams to embellish audio explanations to support clearer sharing and formation of mental models (Fletcher & Tobias, 2005).
- Placing related information in the same interface to avoid fracturing students’ attention (Ayres & Sweller, 2005).
- Removing irrelevant pods to allow greater attention to be placed on the material being learnt (Mayer, 2005b; Sweller, 2005)
- Using combinations of modalities that most closely resemble the different types of information being represented (Salomon, 1994).
Using multimedia learning principles to design the shared cognitive space allows representations to be shared in way that lowers the load on peoples’ working memory.
3. Design tasks that require students to be productive in order to increase their involvement and more fully reveal their mental models
Because students were discussing curriculum matter more extensively and producing solutions in student-centred tasks during Iteration 2 and Iteration 3 their mental models were more fully revealed. This provided the teacher with greater insight into the accuracy of student schema and the form of remediation that may be required. Students were able to practice subject specific processes and co-construct understandings with one another. Successful learning may depend on having a balance of transmissive, interactive and collaborative activities, however the efficacy of lessons where student contributions are not made on a frequent basis should be carefully considered. In such circumstances it is not possible to ascertain the level of student engagement or understanding. This is particularly pertinent in online environments because of the lack of non-verbal cues such as body language. Relevant tasks require students to integrate different levels of knowledge and encourage collaboration through their goal oriented and problem solving nature.
4. As the major influence upon the types of activity that transpires, teachers need to take responsibility for establishing the environment and guiding collaboration
Establishing clear expectations about the nature of the collaboration required allows students to concentrate on the task rather than how to interact using the mediating technology. Strategies include providing clear task specifications, allocating students to roles, ensuring students have the required technological competencies to perform the task, and suggesting ways to represent concepts using the technology. By pre-empting areas where students will require explicit instructions on how an exercise should be performed or providing support in-situ, students are able to dedicate more time to developing their design and problem solving abilities unimpeded by overheads incurred by trying to learn collaboratively in online mode. Providing collaborative space (both virtual and temporal) increases the amount of contribution students can make during a learning episode. Along with providing student-centred authentic tasks, providing students with adequate collaborative space was critical to revealing their mental models so that the teacher could more accurately diagnose misconceptions and provide appropriate feedback.
5. Environment should be adaptively redesigned to meet the emerging cognitive and collaborative needs of learning episodes
In conversational approaches to learning the direction of discourse is negotiated (Laurillard, 2012). This means that the cognitive and collaborative requirements of the interface may change based on the interpretations, actions and feedback of participants. This makes it imperative that the interface is redesigned during learning episodes in order to meet the evolving representational and interactional needs, wherever possible. Examples include using whiteboards to represent emerging requirements for conceptual discussion, using screen broadcasting to perform group programming processes, using notes pods to work with larger quantities of factual information. At times text-chat was sufficient for a few students to contribute factual responses, however for more extended, tightly coupled interactions involving intensive use of another visual medium then audio was more suitable to allow more rapid contribution and more cognitively efficient information processing.
6. Students can play an important role in adaptive re-design of learning environments
Students can an excellent source of redesign ideas, often contributing suggestions and adjusting the interface themselves when given the opportunity. For instance, in Iteration 2 and Iteration 3 of the student-centred Circle Applets programming activity, students decided to enlarge the note-pod of the main program. Once the other programs are integrated into the main program they become obsolete, and enlarging the combined program allowed a more complete view of the code. Resizing pods allowed their relative level of importance in the shared cognitive process to be represented. The fact that students are frequent users of the environment for learning means that they can often generate environmental adaptations that had not been considered by the teacher.
While these principles have been derived from studying teaching and learning via web-conferencing, it is proposed that they can be transferred to some extent to other online learning environments.
Framework for adaptive learning design in the web-conferencing environment
The analysis conducted in this study implies a framework for adaptive design in web-conferencing environments based upon the level of interaction to be facilitated and the level of knowledge being addressed. Table 1 contains thumbnail summaries of the techno-pedagogic patterns that comprise the web-conferencing learning design framework. The summaries are patterns in so far as they provide a description of how the learning environment may be redesigned depending on certain attributes (knowledge and interaction type) but generalizable to a range of learning situations (for instance, different domain of study). The patterns are atomic in so far as they operate at the level of individual learning activities rather than larger session, module or program patterns. By defining a series of nine design patterns for different types of activity and different levels of knowledge the framework supports rapid redesign based on the emerging requirements of the learning episode.
The teacher uses audio and presents pre-prepared artefacts in a large share pod to provide students with factual knowledge. Students comment using a small text-chat pod if required. See Figure 1.
Modelling Process (e.g. Programming)
The teacher uses audio and a large screen-sharing pod to describe how to perform a process. Students comment using the small text-chat space if required. See Figure 3.
Teacher uses audio and pre-prepared diagrams (either on documents or whiteboard) to explain concepts. Students comment using a relatively small text-chat if required. See Figure 10.
The teacher uses audio and visual stimulus to prompt students for responses to factual questions. Students are provided with an enlarged text-chat pod to respond (or use audio for more extensive responses). See Figure 2.
The teacher uses audio and screen-sharing to prompt students for directions about how to perform a process. Students are offered an enlarged text-chat pod to respond (or use audio for more extensive discursive contributions). See Figure 11.
The teacher uses audio to guide students through the construction of a conceptual representation on the whiteboard. The concept may be static (see Figure 8) or dynamic (see Figure 9). Students use audio to interact but may choose to use text-chat to contribute thoughts while the teacher is speaking.
Students use note-pods and audio to collaboratively compose sets of definitions or factual information. The teacher uses audio to address the class or a particular group (or text-chat to address individuals within a group). See Figure 4
Collaborative Process (e.g. Programming)
Students use note-pods with audio to perform a co-constructive process (e.g. write a computer program). See Figure 6 and Figure 7. As the product approaches finality it can be further refined using specialised software and screen-sharing (see Figure 13). The teacher uses audio to address the class or a particular group (or text-chat to address individuals).
The students use a whiteboard and audio to collaboratively construct a conceptual representation. The teacher uses audio to address the class or a particular group (or text-chat to address individuals). See Figure 12. A note-pod may be used instead of a whiteboard if the information is textual rather than visual.
Some of the patterns such as the ‘Question-Response’ and the ‘Instructed Teacher’ were used from the first iteration of this study, while others such as the ‘Teacher-led Representation’ and the ‘Student Representation’ did not evolve until the end of the design research process. These nine patterns are not an exhaustive set of learning designs for teaching in web-conferencing environments, but rather an essential collection of atomic patterns which can form the basis of a teacher’s adaptive design repository. They reflect the accumulated understanding derived by the end of the three iterations analyzed in this study. The patterns could also be used to inform the design of more extensive learning designs that occur across an entire lesson or module of work.
If students have no or weakly formed understanding of content matter then a teacher-centred approach allows fundamental mental model forming information to be transmitted. If students have acquired an understanding of individual items of information then a teacher-led approach allows students to learn other knowledge chunks and observe how the pieces may be synthesized. If students understand components of knowledge then student-centred group-work approaches allows them to collaboratively interrelate knowledge by negotiating solutions with their peers. While it is acknowledged that amongst a class or even within an individual student the level of knowledge cannot be precisely defined at a single level, this fading approach to scaffolding provides a general framework for considering how to redesign the environment to accommodate appropriate levels of student-centredness.
As the level of student control is increased they are provided with greater collaborative space; transmissive interfaces provide a small text-chat area, interactive interfaces increase the text-chat area and provide audio for more substantive discursive contributions, and collaborative interfaces provide audio to enable students to hold discussions at the same time as they construct solutions.
The level of knowledge influences which representational modality is included in the interface. Text (prepared documents, text-chat, or note pods) are provided when working with factual knowledge. Screen-broadcasting is used to share procedural knowledge (with note pods providing an alternative at early stages of co-authoring processes), and whiteboards are used to share conceptual representations.
Transitioning to the optimal interface design is a matter of diagnosing the level of knowledge to be addressed and the degree of interaction that will take place, and then adapting the interface accordingly. To some extent this can be anticipated in advance based on the learning activities that have been prescribed and an appreciation of the students’ level of ability. At other times the level of interaction and knowledge level may unexpectedly change based on student questions, activity or feedback, in which case the framework in Table 1 provides a guide for rapid adaptive redesign to meet the emergent collaborative and cognitive needs. Thus the framework provides support for Learning Design Pedagogy, but also relies on the pedagogical insights of the teacher to appropriately improvise.
It should be noted once again that the above framework provides standard techno-pedagogic patterns for different levels of interaction and knowledge levels. It is anticipated that there are many other possible designs depending on the specific collaborative and cognitive needs of the specific context. As well, it may be possible to represent hybrids of the aforementioned designs in order to support integration of different levels of knowledge or types of collaboration. The way in which a whiteboard is integrated into the interface shown in Figure 8 to facilitate interrelation of procedural and conceptual knowledge is a case in point.
More detailed descriptions of each of the patterns are provided in earlier work (Bower, 2008).
Adaptive design of learning environments can enable more effective interaction and knowledge representation, which in turn enables the teacher to more accurately gauge student understanding and provide appropriate intervention. As the technologies at educators’ disposal become more flexible, frameworks to guide adaptive design will become increasingly important.
This study derived a framework for adaptive re-designing of a web-conferencing environment to more effectively cater to the level of knowledge being addressed and the type of interaction that transpired. Text was used to support factual knowledge, screen-sharing and notes areas supported sharing of procedural knowledge, and whiteboards were used to exchange conceptual knowledge. As the degree of student contribution increased the size of text-chat area was enlarged, and in the case of student-centred collaborative tasks students used audio to enable them to exchange larger amounts of text while working on their co-constructive tasks. As they are provided with greater control over the learning experience, students (as end users of the environment) may become important participants in this dynamic design process.
The design-based research methodology allowed the effect of different interface designs to be gauged and validated over several cycles of iterative refinement (both within semesters and between them). Rather than being prescriptive or exhaustive, the nine patterns in the framework provide a design repository that can be used by teachers to support rapid interface redesign and context specific adjustment. By including teacher-led and student-centred approaches to developing procedural and conceptual understanding it is hoped that the framework encourages educators to transition from more transmissive approaches to more effective application of interactive and collaborative pedagogies in online multimodal synchronous environments. Thus, while the framework contains a degree of pedagogical neutrality in so far as it does not prescribe how teachers should use it, the Learning Design Pedagogy that teachers apply when using the framework will determine the extent to which it positively affects student learning.
The field of technology-based learning is continually evolving, and if teachers are expected to move to new technologies and rapidly redesign the environment to facilitate more effective learning then frameworks will be needed to support this process. These frameworks could take the form of guides for teachers, either within or external to learning technology systems. In the future it is possible that a level of intelligence could be integrated into educational technologies that processes participant contributions and provides educators with advice about how to adapt the environment in light of emerging requirements. ‘Big data’ and learning analytics may mean that the adaptation of learning environments may be informed by collective analysis of learning episodes across a wide range of institutions and contexts. As learning technologies become more refined, knowledge derived from this analysis may be presented to teachers in the form of a simple option to ‘apply’ a redesign, or even be automatically implemented.
In each instance the development and deployment of adaptive design frameworks will require an analysis of the affordances of the technologies, the types of activity desired and the types of knowledge being addressed, as was the case in this study. The development of adaptive design frameworks for a variety of technologies will also allow the field to determine more general principles for adaptive technology-based learning design.
The author declares that they have no competing interests.