Group-work in the design of complex adaptive learning strategies

Keywords: complex learning, technology enhanced learning, adaptive learning, computer supported collaborative design

Introduction

In the field of Technology Enhanced Learning (TEL), the process of transformation from the initial conceptualisation of an ICT-infused learning design, its development using technological means and its enactment with learners plays a central role (Muñoz-Cristóbal, 2012). This transformation process, all important from the designer's standpoint, constitutes a demanding and complex task. It requires the mastery of an integrated set of competencies in order to effectively address issues that may range from learning theory to software or knowledge engineering (Mor & Winters, 2007; Sims & Koszalka, 2008; Hardre & Kollmann, 2013; Paquette et al., 2006). The alignment of these competencies with a multidisciplinary approach towards learning design is important also due to the lack of commonly- and cumulatively-built understandings within the field of Technology Enhanced Learning (TEL) community (Mor & Winters, 2007). In the "real world" that awaits the Learning Design practitioners after the completion of their studies, they will probably need to deal with "a range of interconnected issues including technological, organisational and pedagogical" (Conole, 2008). Among others, a learning designer in the field of TEL will be called to make decisions about: strategy, interactions, interfaces, content and its delivery (Sims, 2006),as well as, to accomplish tasks such as (Kenny et al, 2005): writing learning objectives, identifying the types of learning outcomes, selecting media formats, delivery modes etc

The intervention described in this paper was enacted as part of a 14-week graduate course on "Learning Design and Design of Educational Software" which was included in an MSc in Information and Communication Systems (ICS) program during the first semester of the academic year 2012-2013. The course is offered online and it combines weekly synchronous sessions (using the web conference technique) with asynchronous activities implemented through the university learning management system and via email. The asynchronous activities mainly facilitated a) instructor-to-student and student-to-student communication and b) access to learning materials, assignments, assignment grades and so on. To a lesser extent, the learning management system facilitated the students' familiarisation and experimentation with certain learning technologies, like Web 2.0 technologies.

Twelve students, aged from 25 to 45 years old, participated in the course. Students taking this course specialise in Educational Technology, usually for professional reasons. Six of them were active educators in different settings (elementary, primary, secondary and continuous/adult education), teaching diverse subject matters (informatics, mathematics, physics and others). Two of them were pre-service teachers and the remaining four had various non-teaching professions involving ICT. Four live in Cyprus and eight live in Greece. None of them had participated before in a lesson about LD or TEL. All of them had completed their studies in tertiary education. During the academic semester, students submitted ten short weekly and two in-depth assignments.

The intervention discussed in this paper attempts to exploit the Learning Design approach in order to help novice instructional designers to:

  • adopt a more creative and holistic view towards the design of a TEL design artefact (Conole, 2010)
  • realise the interplay between the various dimensions and approaches that influence the Learning Design process and its enactment with learners using technological means
  • liaise effectively in a multidisciplinary team and communicate the needs of all the critical stakeholders of the field within the team

Learning Design & IMS-Learning Design specification

In the recent literature, the meaning of Learning Design is twofold (Conole, 2008; Donald et al., 2009) since it can be viewed both: a) as a process of designing for learning (i.e. lessons, learning activities, lesson plans) and b) as a product that contains elements such as descriptions of learning tasks, resources and scaffolds. Dobozy (2011) expands this typology by adding the notion of Learning Design as "a standardised (re)presentation of technology enhanced learning sequences and prescribed design based procedures that are content independent". In (Conole, 2008) it is stated that the term in recent years is being used almost synonymously to "course design". For the scope of this paper LD is defined as "the act of devising new practices, plans of activity, resources and tools aimed at achieving particular educational aims in a given situation" (Mor and Craft 2012).

The term came into usage almost a decade ago with the development of the IMS Learning Design (IMS-LD) specification (IMS GLC, 2003a; Conole 2008). It aims to provide a means of formalisation of the teaching-learning process through an educational modelling language that employs the metaphor of a theatrical play (Koper & Tattersall, 2005). Using this metaphor, the IMS LD specification provides a pedagogical neutral representation of the teaching-learning process (Koper & Olivier, 2004) comprised of actors, roles and sequences of activities. From this perspective, the Learning Design specification is a notational system which IMS-LD compliant editors and players employ, along with instructional engineering methods (Paquette et al., 2006), in order to create a Learning Design for sharing and future re-use (Kordaki, 2007).

  • The information model of the IMS-LD (IMS GLC, 2003b) has three levels of implementation, each a superset of the precedent one:
  • level A provides the core elements of the modelling language (roles, activities, services, resources etc)
  • level B adds properties and conditions and serves as the adaptation basis since, in its simplest form, enables the creation of rule-based (simple "IF-THEN-ELSE" rules) adaptation: an event is triggered when a property satisfies a condition. For example, the average grade of the student ("property") is calculated ("event") when the last question of a quiz is answered ("condition"). Adaptation rules are created by the designer during authoring time of the resulting Unit of Learning (i.e. the adaptive course)
  • level C adds notifications of events to users, as well as, notifications between system parts

Adaptive educational systems are model-based systems that adapt some of their key functional characteristics - for example, content presentation and/or navigation support (Brusilovsky, 2003) - taking into account learner characteristics like learner needs and preferences (Magnisalis & Demetriadis, 2012). The IMS-LD specification was frequently used as the modelling basis for adaptation and personalisation (Burgos et al., 2007; Paramythis & Loidl-Reisinger, 2003; Magnisalis & Demetriadis, 2012; Specht & Burgos, 2007) in a range of TEL examples that include: an adaptive learning management system (Boticario & Santos, 2007), authoring tools to define re-usable adaptive learning designs (Berlanga & García, 2005; Miao, 2005; Sampson et al., 2005), adaptive learning designs (De-la-Fuente-Valentín et al., 2011; Berlanga & García, 2006; Burgos et al., 2007; Mavroudi & Hadzilacos, 2012(a),(b)), Computer Supported Collaborative Learning (Hernández-Leo et al., 2006; Valdivia et al., 2009; Magnisalis & Demetriadis, 2012). The adaptive learning strategies that can be fully or partially implemented with the use of the IMS-LD methodology can be summarised as follows (Burgos & Barak, 2009; Burgos et al, 2006): learning flow adaptation (fully supported), content-based adaptation (fully supported), interactive problem-solving support (fully supported), adaptive user grouping (partially supported), interface adaptation (partially supported), adaptive evaluation (partially supported) and adaptive information filtering and retrieval (not supported).

Complex learning & 4C/ID model

The design of the intervention is based on the Four Components Instructional Design model (4C/ID), a model originally developed by Van Merriënboer et al. (2002), who suggested that environments for complex learning can have four interrelated components:

  1. Learning tasks that engage students in activities suitable for the development of the needed constituent skills, as opposed to activities where students need to study general information related to the skills.
  2. Supportive information that bridges what students already know to their work on the learning tasks. Tutors typically refer to this type of information as "the theory", often presented in lectures and textbooks.
  3. Just-In-Time (JIT) information relates to the constituent skills that should be performed similarly in different problem situations. It offers students information about the procedural knowledge they need to obtain in order to perform the recurrent skills. Examples of this type of information include instructions provided during students' practice, where tutors are acting as an "assistant looking over your shoulder" (Van Merriënboer et al., 2002).
  4. Part-task practice which is required if a very high level of automaticity of particular recurrent aspects is necessary. Examples of part-task practice are "drilling children on multiplication tables and playing scales on musical instruments" (Van Merriënboer et al., 2002). An example of part-task practice in training design of an air traffic controller might involve critical recurrent constituent skills in terms of safety, for example, identifying risky air traffic situations from a radar screen (Van Merriënboer et al., 2002).

Two main principles of the model are: 1) scaffolding and fading (i.e. withdrawing help as the learner progresses) and 2) that in each learning task, the complexity of the sub-tasks should gradually increase. Finally, the model aims at the automation of the recurrent aspects of the task, while promoting deep learning for its non-recurrent aspects. According to Van Merriënboer et al. (2003) complex learning involves "the integration of knowledge, skills and attitudes, the coordination of qualitatively different constituent skills and the transfer of what is learned to daily life or work settings". According to Merrill (2002), the model is based on problem-based learning and also focuses on a learning activity that employs complex cognitive skills.

The basic aim of our intervention was to make the students appreciate the interdisciplinary nature of the field of educational technology and the interplay of the various perspectives and dimensions involved in the design of a TEL artefact through a problem-based learning approach. On top of that, the use of IMS-Learning Design specification provided a formalisation of the teaching-learning process through the metaphor of the theatrical play, leaving all the design decisions up to the learning designer. In parallel, the focus was also placed on Learning Design systems that, in the context of this paper, are defined as "workflow engines for collaborative activities" (Udas, 2009). Two IMS-LD compliant systems were used by the students in order to author and preview their adaptive e-courses. The resulting learning environment was an Open Learning Environment (OLE) with tools, resources and activities suitable for the promotion of divergent thinking in a learning situation where multiple perspectives are valued (Hannafin et al, 1983). Learning Design is an open-ended and ill-defined problem. Thus, the learning tasks were designed so as to provide enough opportunity to the students for practice both the non recurrent (study LD theory, design learning scenario, employ an adaptive learning strategy and so on) and the recurrent aspects of the complex skill (drill the most commonly used functionalities of the tools and so on). That is why such a learning ecology was appropriate and why the 4C/ID model was a suitable model for our students' collaborative tasks.

More specifically, our students had to deal with the following tasks:

  1. read the respective LD theory, which, for the scope of this intervention, comprised two chapters from a previous master thesis on the topic, the IMS-LD Information model (IMS GLC, 2003b), the IMS-LD Best Practice and Implementation Guide (IMS GLC, 2003c), one chapter from the text book (Koper & Tattersall, 2005) and lesson slides
  2. study worked-out examples of adaptive learning scenarios and their incarnations in the form of Units of Learning; worked-out examples and their corresponding UoLs were administered to the learners (two examples of level A and two examples of level B and their corresponding UoLs)
  3. conceptualise and design a learning scenario for adaptive learning close to their interests (i.e they could select any subject matter); towards this end, a learning scenario template accompanied with the description of the semantics of all its fields was administered to the learners
  4. draw a UML activity diagram to demonstrate the different phases, actors, interactions, synchronisation points between actors and learning strategies used in the learning scenario-see examples in (IMS GLC, 2003c)
  5. develop learning materials and adaptive learning strategies that can be implemented fully or partially with the use of the IMS-LD methodology
  6. install and use the authoring tool (i.e. the MS-LD compliant editor), as well as, a tool to preview their work (i.e an IMS-LD compliant player); user guides were provided
  7. implement the respective adaptive Unit of Learning (i.e. adaptive e-learning lesson) using the authoring tool and frequently preview the results using the player
  8. discuss the difference concerning the implementation difficulty between a non-adaptive (Level A-compliant) UoL and an adaptive (Level-B compliant) UoL
  9. reflect on the concepts of "re-usability", "learning content", "learning services", "learning activities" and the use of widgets (see Annex 2)
  10. reflect on their experience by proposing a model of guidance and support in an imaginary scenario where they had to lead a multidisciplinary team in order to construct adaptive Units of Learning -this task was included as a topic in the students' final exams
  11. map textual descriptions of a set of events (what constitutes an event was described in the previous section) against the type(s) of adaptation functionality they implement (e.g. learning flow adaptation, content-based adaptation, interactive problem-solving support, adaptive user grouping etc), this task was also included as a topic in the students' final exams

Methodology

The contextual settings of the Learning Design intervention

The intervention took place between the 10th and the 13th week of the course (from the time that the assignment was announced to the students to the time that the students' products were uploaded to the Moodle LMS). The two last sub-tasks mentioned in the previous section were included as sub-topics in the final exams, due one week after the end of the 14th week of the course. The participating roles in this intervention were: a) the tutor, who was the main person responsible for the course in general and was basically responsible for the support of the students with the more "theoretical aspects", b) the assistant, who was the main person responsible for the more practical parts i.e. of the design of the worked-out examples and completed scenarios and c) the students. In terms of scaffolds, except those mentioned above, one web conference meeting (where the students, their tutor and the assistant participated) was devoted to the enlightenment of the difficult parts of the theory. Additionally, support and advice was provided by the assistant via a dedicated online forum, Skype and email. It revolved mostly around tool functionality issues, but secondarily also involved other issues like ideas about adaptation parameters. Towards this end, the paper by Economides (2008) was also included in the suggested readings. The support included synchronous as well as asynchronous discussions. Student evaluation rubrics and performance standards were designed collaboratively by the tutor and the assistant (the students' evaluation rubric is shown in Annex 1).

The twelve students who participated in the course were asked to work collaboratively and consequently, three groups were formed. In each group there was at least one member who was an educator (and, as such, he/she was familiarised with lesson planning procedures) and at least one member that was an informatics professional.

In terms of the IMS-Learning Design compliant tools used, the ReCourse LD editor and the Astro LD player were pre-selected by the tutors (both available at: http://tencompetence-project.bolton.ac.uk/ldauthor/)

Data collection methodology

Empirical results were gathered from the following data collection sources:

  1. the web-conference recording session,
  2. posts in the dedicated forum in the university LMS,
  3. answers on an online survey questionnaire and follow-up semi-structured interviews
  4. the final products of the students i.e. their designed artefacts
  5. the answers of the students in the hypothetical scenario (a final exams question)
  6. the log-files of the LMS system and the IMS-LD player (the online version)

Some remarks concerning the data collection methodology:

  • The online survey questionnaire was distributed to the students immediately after the end of the semester and the semi-structure interviews followed a few weeks afterwards. The interviews duration was 20 minutes in average and they were conducted through electronic means (VoIP and the Elluminate live! TM web-conference software, according to the student's preference).
  • The log files provide some evidence about the type and the frequency of students' interactions with the system, but it is not a clear and reliable indicator concerning their engagement and participation. This is due to the fact that, concerning the LMS forum, it was only used for the "official story" of the students' interactions, since from the interviews it became evident that the students used SkypeTM to collaborate on their assignment. As far as the IMS-LD complaint player is concerned, each student had access both to the online version but also to their local, offline version of the player that they used to preview and test their UoLs.
  • Participation in the online survey, in the discussion forums or in the personal interviews was voluntary. Also, none of the above contributed towards the students' final grades. The return rate of the online survey was 100%, since all the students completed the survey. The return rate of the semi-structured interviews was 67%, since eight students participated (the remaining four students were either not able or willing to participate). Finally, the students were aware of the fact that the web conference sessions and the semi-structured interviews were recorded.

Data analysis and results

Collaboration

The students felt that the aspect of genuine collaboration was addressed satisfactorily (n=12, mean=4.6, standard deviation=0.6 in a five-point Likert scale where the respective question in the online survey was "This task favoured genuine collaboration". A score of 'one' indicated failure in satisfying the goal and a score of 'five' indicated success).

In the online survey questionnaire the students were asked to reason about the score that they had assigned in the previous question. The open-ended question was used in the survey in order to provide the students with the opportunity to make suggestions, comments and complaints related to their learning experience (Kanuka & Anderson, 2007). The transcripts of the open-ended questions were analysed using the grounded theory approach (Glaser & Strauss, 1967; Strauss & Corbin, 1994), a content analysis method that allows key themes to be emerged mostly from textual data through coding cycles. The code, concepts and categories emerged were tested against the interview transcripts. The grounded theory was selected as opposed to an interpretivist approach (e.g. theory-driven approach) of qualitative data analysis, since the research goal was not to test any preconceived hypothesis about the students' collaboration and engagement but rather to try to understand their ontological meanings in a contextual way (Allan, 2003). The results are shown in the textbox, where concepts are presented in the form of bulleted lists and the emerged themes are in capital letters.

Figure 1. Concepts and themes concerning the collaboration aspect

TEAM MEMBERS COLLABORATED IN ORDER TO RESOLVE PROBLEMATIC SITUATIONS

  • Collaboration as a means of resolving tool issues and providing help
  • Collaboration in order to compensate of lack of supportive material

ACTIVE TEAM MEMBERS HAVE A POSITIVE ATTITUDE TOWARDS COLLABORATION

  • Positive attitude/comments towards collaboration
  • Collaboration perceived in terms of equal contribution by team members

COLLABORATION WAS FOSTERED THROUGH A SCHEDULE

  • Having a schedule (task schedule, meeting schedule)
  • Collaboration via frequent meetings between team members
  • Conflictions with other students' obligations hinder participation

COLLABORATION WAS SUPPORTED BY A RANGE OF CSCL TOOL AND PRACTICES

  • Collaboration via online meetings
  • Collaboration perceived in terms of efficient communication
  • CSCL mediated via students' designed artefacts (lesson scenario, UoLs)
  • Tools and practices covered various aspects of CSCL (skype meetings for synchronous communication, forum for asynchronous communication, google docs for theoretical parts, desktop sharing for technical or practical issues)

COLLABORATION SCHEMATA

  • Limited collaboration between teams
  • Role/task distribution within teams
  • Frequent controls and corrections

Concerning the collaboration via the forum, five discussion forums (each addressing a separate aspect of the students' assignment) were set up on the Moodle LMS. Two of them ("forum 1" and "forum 2") were created and initiated by the tutor and the remaining three by one particular student ("forum 3", "forum 4" and "forum 5"). The data presented in the table below indicate the frequency of student and instructor participation, the total number of messages in each thematic forum (Hara et al, 2000) and the number of students that participated in each forum discussion.

Table 1. Participation in the online forums

Name
(created &
initiated)

Total number
of messages

Total number
of instructors'
messages

Number of
students'
participants

Forum 1
(by the instructor)

20

9

4

Forum 2
(by the instructor)

9

3

4

Forum 3
(by the student)

4

1

2

Forum 4
(by the student)

29

8

7

Forum 5
(by the student)

11

10

1

The dimensions of content analysis using CSCL tools, such as an online forum, that are commonly employed are: participation, cognitive processing and social interactions (Sing and Khine, 2006). The data of the table above gives a rough idea of the participation aspect. The five discussions included altogether 73 postings. The proportion of postings by the instructors was 42% (31 messages). This relatively large proportion can be attributed to the fact that the forum was perceived by the students as the means of getting help and support primarily from the professor and the assistant. In the context of this distance education setting, communication with them exclusively took place online. From the online survey and the semi-structured interviews it became evident that the students frequently consulted the forum. This is confirmed by the web analytics that enable the user tracking functionality of the LMS, which revealed 763 views of forum discussions (in the forums 1-5 and during the time period that the intervention took place).

Concerning the types of postings, various content analysis schemes to analyse transcripts of online asynchronous discussions exist (Hara et al, 2000; De Wever et al, 2006). For the scope of this paper, the theoretical framework of Järvelä and Häkkinen (2002) seemed most appropriate, since it proposes a categorisation of the postings which discriminates between: (a) theory, (b) new point or question, (c) experience, (d) suggestion, and (e) comments. The message served as the unit of analysis for this categorisation. The students' messages in the forum were analysed and coded independently by the assistant and the author of the master thesis included in the students' readings. The inter-rater agreement percentage was at first 81%, which reached 95% after discussions about the disagreements between the two coders. The final results are depicted in the figure below. Out of the 42 students' messages posted in the forum, 1 message (2%) concerned LD theory, 9 messages (21%) introduced a question or a new point, 10 messages (24%) suggested a solution, 10 messages (24%) reflected students' learning experience and 12 messages (29%) mostly commented on various other aspects related to the intervention at stake.

Figure 2. Distribution of posting types

The forum served as a common place of reference that included a significant number of suggestions concerning the above issues. It should be noted however that communication between the assistant tutor and the students was also to a lesser extent enacted through email and Skype. From the online survey it can be concluded that, overall, students felt that the support they received was very satisfactory (n=12, mean=4.4, standard deviation= 0.8 in a five-point Likert scale where the respective question in the online survey was "The support I received via the forum/Skype/email was satisfactory". A score of 'one' indicated failure in satisfying the goal and a score of 'five' indicated success).

Bridging the gap between LD theory and IMS-LD practice

The degree in which the intervention accomplished the goal of bridging the gap between theory and practice was perceived by the students as very satisfying (n=12, mean=4.4, standard deviation= 0.9 in a five-point Likert scale where the respective question in the online survey was "The task helped me in bridging the gap between theory and practice". A score of 'one' indicated failure in satisfying the goal and a score of 'five' indicated success).

Two indicators were evaluated positively on this criterion:

  1. Consistency between the learning scenario, the UML workflow diagram and its incarnation in the form of a Unit of Learning (see Annex 1).
  2. Evidence that the students could distinguish the difficulties that arose due to the user-friendliness of the tools they used (or the absence of it) from those that would remain the same regardless of the IMS-LD compliant tool being used. The latter implies difficulties inherited in the complexity of the conceptual framework of the IMS-LD (the workflow, the theatre metaphor, the adaptation elements).

Concerning the first indicator, all the groups managed to achieve consistency between the learning scenario and the UoL, whereas none of the groups managed full consistency between the UML activity diagram and the learning scenario (or the UoL), although the students were familiarized with UML activity diagrams before their enrolment in the course. As far as the second indicator is concerned, all the students that were interviewed showed explicit evidence that they could distinguish the IMS-LD specification from the tools that implement it. An example of what constituted explicit evidence concerning the issue is depicted below:

(Student) If you have understood the IM-LD specification and you know that you have to cope with activities, roles, environments etc, and furthermore, if you can distinguish levels A, B & C, then -using the ReCourse editor or any other IMS-LD compliant tool -you won't be having any issues, from the specification perspective. If you have conducted a good planning on how to implement it (i.e the learning design) then it's a purely technical matter and it depends on how familiarised you are with the program.

In addition to the indicators mentioned above, the authors also detected some student statements that on the one hand exemplified deep understanding concerning the fact that the UoL is the incarnation of a learning design, but, on the other hand, they did not fall into either of the two categories mentioned above. An example is the following:

(Student) After this assignment everything was clear to me, I didn't need to study (for the final exams), only some lectures. The only thing I took with me in the exams (students were allowed to bring with them any learning materials they wished for the final exams) was the printed form of the manifest.xml of the UoL we had created. The entire course is included in this file, what else did I want? [..] declarations, metadata, properties, activities, roles-parts, roles.., everything is in there. If you can conceive this, this XML file can tell you all you need to know.

Additionally, from the discussions held with the students it became evident that they could abstract their learning experiences. By the time that the interview took place one student (an informatics professional) had already managed to transfer the knowledge gained from the intervention into her workplace. In particular, she mentioned that she had applied the design philosophy behind the IMS-LD compliant tool into a new e-business system in her work, in order to better conceptualise its information model since she felt that it was similar to the IMS-LD Information Model (i.e. resembling the metaphor of a theatrical play). Another student, who used two different IMS-LD compliant editors, could effectively communicate to the assistant during the semi-structured interview their different affordances and intended uses.

Learning progress

As mentioned in section "Complex learning & 4C/ID model", two sub-topics about adaptive learning were included in the students' final exams, aimed at checking how the students' learning about the topic at stake (i.e. adaptive learning designs) had progressed over time. The score of each team (i.e. the average score of the team members) both for their assignments and for the final exams was calculated. The results are depicted in the table below.

Table 2. Scores & learning progress

Team

Assignment scores
(out of 10)

(Average) exam
scores
(out of 10)

Team A

8.5

7.3

Team B

9

9.3

Team C

9.5

9.7

As it is shown in the table, there is a strong positive correlation (r=.92, p=.25) between the scores in the assignment with the scores (in the topics about adaptive learning) in the final exams. The correlation between the students' scores in the assignment and the students' scores in the final exams is depicted in the table below. The t-test still reveals a positive correlation between the two variables (r=.501, p = .01). Note that Kolmogorov-Smirnov tests were performed to test the normality presupposition.

Table 3. Correlation between performance in the assignment
and performance in the exams
at the level of the individual student

With regards to the hypothetical scenario where the students had to lead a multidisciplinary team in order to construct adaptive Units of Learning, most of them reproduced their own variations of the 4C/ID model.

Complex learning in today's digital world: CSCL or CSCD?

Background and rationale

There is a growing body of research on co-design for learning purposes approached from two major perspectives:

  • In terms of specific-purpose tools; for example in (Hernández-Leo et al, 2011) a web-based authoring environment, "LdShake", is presented as a tool that enables the co-edition and social network-oriented sharing of learning designs created using a general rich text editor.
  • In terms of collaborative design of ICT-infused learning scenarios by teachers, as a form of their professional development. This approach sees teachers as designers (Voogt et al, 2011) and is aligned with the well-established "learning by design" notion (Kolodner et al., 2003), while it acknowledges that the active involvement of the teachers in the LD process might have a positive impact on their professional development and in turn, on student learning (Kali & McKenney, 2012).

In parallel, the question of how we could apply to LD insights derived from the discourse with other design disciplines is an interesting aspect that has begun to flourish and to attract the attention of the stakeholders (Mor & Craft, 2012; Mor et al., 2013). Examples of initiatives that embrace this aspect include:

  • The Learning Design studio (LDS), an effective manifestation of the Design Inquiry of Learning (DIL) model. The latter combines an inquiry-based learning approach with a design-based scientific paradigm. The former is modelled after the tradition of studio-instruction in arts and design disciplines, such as architecture, its main characteristic being the students' on-going group work on design challenges in a domain of practice (Mor & Mogilevsky, 2013).
  • The Design Principles Database (DPD), which aims at synthesizing emerging design knowledge about the use of technologies for education (Kali,2006).
  • The International Journal of Designs for Learning, a journal dedicated to publishing descriptions of artefacts, environments and experiences created to promote and support learning in all contexts by designers in any field.

In this study the design of the collaborative activities and, consequently, the evaluation of students' designed artefacts were focused on two challenges that students needed to face through group work, e.g.:

  • the formulation of an adaptive learning strategy which poses several challenges for the novice designers, since it involves the interplay between the adaptation parameters (prior student knowledge, learning objectives, student performance in learning tasks etc.) with the adaptation methods (like: sequencing of learning activities, feedback, student grouping etc.).
  • The interplay between the different LD representations in the lifecycle of the students' design artefacts. The learning experience involved the design of a learning scenario and the development of the corresponding unit of e-learning. Literally speaking, the Unit of Learning is the incarnation of their learning scenario.

Creation of artefacts and collaboration were the two most pivotal design patterns, while the following design principles were incorporated in the intervention:

  • use open-ended construction tools
  • engage learners in a complex project
  • use multiple representations
  • provide knowledge representation and organisation tools
  • promote autonomous lifelong learning
  • provide students with templates to help reasoning
  • provide just-in-time data to students
  • connect to personally relevant contexts
  • encourage reflection

Implications

The purpose of this section is to provide insights concerning the interactions between the students with their digital artefacts, as well as their social interactions in order 1) to discuss educationally effective strategies in a complex learning task where Computer Supported Collaborative Learning (CSCL) and Computer Supported Collaborative Design (CSCD) took place and 2) provide future directions on how they can be incorporated in the design of an IMS-LD environment.

In particular, focus is placed on the role of the mediating artefacts viewed from a distributed cognition framework perspective, since the latter "is specifically tailored to understanding interactions among people and technology" (Hollan et al, 2000) in terms of not only what people know, but how they go about using what they know to do what they do (Hollan et al, 2000). For the scope of this paper (where the participants' observation was not possible) distributed cognition is being viewed from a situative perspective (rather from a cognitive one) in which knowledge exists in the way that social groups communicate, make use of symbols, tools and designed artefacts and understanding is a process of negotiating the meaning of these objects with others (Hewitt and Scardamalia, 1998). Interesting points in the intervention at stake are: 1) the use of student-created designed artefacts and 2) the students' spontaneous use of CSCL tools and techniques, in order to mediate their learning and reduce their cognitive load. The discussion is mostly based on what is presented in the section titled "Data analysis and results", sub-section titled "Collaboration". As already mentioned in that section, CSCL emerged mostly within teams, whereas among teams it has commonly happened in the forums. Also, with regards to the collaboration schemata: roles and subsequent tasks distribution among team members: additionally frequent controls and corrections took place. CSCL was perceived by the team members in terms of efficient communication (i.e. how well ideas and experiences were communicated among team members) and equal contribution to the tasks. The students' constructed mediated artefacts were: the lesson scenario and the UoL. As already mentioned, the students used a variety of CSCL tools and practices: Skype meetings for synchronous communication, forum for asynchronous communication, Google docs for theoretical parts (i.e. related to the creation of the learning scenario), desktop sharing for technical or practical issues (related to the creation of the UoL) in a spontaneous mode. Concerning the organisational and social settings that superseded the learning process, students acted as lifelong learners, since they took full responsibility of their progress by preparing a shared plan of action (task schedule, meeting schedule) and organising frequent online meetings. As it was expected, participation was hindered by students' obligations (family obligations, work obligations). On the other hand, having a positive attitude towards collaboration and being an active member within a team seem to be interrelated. Finally, collaboration between students was practiced as a means of resolving tool issues, providing help and compensating the lack of supportive material.

In conclusion, we argue that the incorporation of a CSCL tool in a way that fosters a variety of practices in today's digital world where learners create and share their artefacts could be an indispensable characteristic of an all-encompassing IMS-LD compliant environment. More specifically, concerning the desktop sharing feature in the case of CSCD of the UoL, it will enable the ''what you see is what I see" (WYSIWIS) design principle which is suggested in previous work for designing at a distance via real-time designer-to-designer interaction (Scrivener et al, 1993). The benefits of Web 2.0 tools (such as forum, chat, Google docs etc.) have been extensively discussed in the CSCL-related literature in general and especially with regards to professional development (Cochrane & Narayan, 2013). Finally, a shared view of the designed artefacts along with the use of Web 2.0 tools would foster the communication and the support coming from the experts, something that seems to be vital in the case of novice developers.

Discussion

This paper is a design case where CSCL and CSCD took place, in which it is evident that the students were not pre-occupied with the Learning Design aspects that are closely related to pedagogy. On the one hand, it has been argued that students who follow Learning Design courses offered via computer-science related programs of study (in this case, the ICS program) face difficulties in the design of learner-centred courses which revolve around the design of appropriate lesson plans (Kordaki et al, 2007). On the other hand, the creation of learner-centred e-courses was not the focus of this particular intervention, although exemplifying sound pedagogical choices was considered an advantage (as one can see in Annex 1).

With regards to our methodology, the idea of conducting semi-structured interviews with the possible stakeholders for eliciting design requirements is not new, since they can provide direct access to 'experience' (Silverman, 2000). For example, Luck (2000) suggests requirements about the inclusive design of a building based on interviewing people with disabilities. Where there were conflicting requirements, these were resolved during dedicated steering group meetings. Yet, a building is a physical entity and as such it can have only one facade, whereas this is not true for a digital environment intended for collaborative design of Units of Learning. Since multiple views in such an environment would be possible, future plans include further work in the requirements analysis by identifying possible different requirements for the two main groups: teachers and informatics professionals. Yet, no conflicting requirements were noted in our case.

Finally, the benefits of adaptive learning to students' cognitive development have long been discussed (Lee & Park, 2007; Kim, 2012; Hwang et al., 2012), thus excluding the teachers from the creation of rich adaptive learning designs may hinder their professional development.

Acknowledgements

The scenario template was a revised version of previous similar work conducted for the scope of GRAPPLE project ( http://www.grapple-project.org), which was an EU funded IST FP7 project (from 2008 to 2011). The intended usage of the GRAPPLE learning scenario template was meant for learning designs that exploit the IMS-LD specification.

Annex 1: The students' evaluation rubric

A. Answer in the question of Annex 2 (15 points)

Clear and concise answers concerning the concepts: re-usability, learning material, learning services, learning activities, widgets

Β. Unit of Learning (40 points)

1.1. Technical integrity/excellence (15 points)

1.2. In accordance with the learning scenario (25 points)

C. Learning scenario (30 points)

1.3. Are all the fields satisfactorily completed/addressed? (10 points)

1.4. Do the students address thoroughly the topic of each field? Is it evident that they have understood the semantics of the field? (10 points)

1.5. Is the workflow diagram correct? Is it fully consistent with the learning scenario? (10 points)

D. Overall impression (articulated answers, authenticity, within word limits, pedagogical rationale and roles, complexity of adaptive strategies: does it combine more than one adaptive strategy? If yes, how many? The combination concerns the same or different phases of the scenario?) (15 points)

Annex 2: A question that was included in the students' assessment

Upload and run the completed (ready-made) UoLs to the LD player in order to preview the respective e-course and to formulate an initial idea about their design and enactment, as well as, the degree and the nature of their re-usability. In order to answer to the latter think that the reusability might entail processes and justifications like the following:

- remove this learning content, keep the lesson structure intact and insert my own content

- remove this learning content, change the lesson structure by adding or removing phases or learning activities and also insert my own learning content,

- …. etc

Please describe (in no more than 150 words) your learning experience. Having practiced the above, what do you think about the concepts of "learning content", "learning services", "learning activities"?

By now, you must have read two chapters from the master thesis. The process of learning design described in a chapter of the master thesis integrates some widgets. Have you understood how they can be used?

References

Allan, G. 2003. A critique of using grounded theory as a research method. Electronic Journal of Business Research Methods, 2(1): 1-10.

Berlanga, A. J., & García, F. J. 2005. IMS LD reusable elements for adaptive learning designs. Journal of Interactive Media in Education, 2005(1).

Berlanga, A. J., García, F. J., & Carabias, J. 2006. Authoring adaptive learning designs using IMS LD. In Adaptive Hypermedia and Adaptive Web-Based Systems, 31-40. Springer Berlin Heidelberg.

Boticario, J. G., & Santos, O. C. 2007. An open IMS-based user modelling approach for developing adaptive learning management systems. Journal of Interactive Media in Education, 2: 1-19.

Brusilovsky, P. 2003. Adaptive navigation support in educational hypermedia: the role of student knowledge level and the case for meta-adaptation. British Journal of Educational Technology, 34: 487-497.

Burgos, D., & Barak, N. 2009. Analysis, modelling and implementation of IMS-LD Units of Learning. Grapple Public Deliverable, 5. http://www.grapple-project.org/public-files/deliverables/D5.3a-WP5-LDUnitsOfLearning.pdf [accessed 10 June 2013]

Burgos, D., Tattersall, C., & Koper, R. 2007. How to represent adaptation in e-learning with IMS learning design. Interactive Learning Environments, 15(2): 161-170.

Burgos, D., Tattersall, C., Koper, E.J.R. 2006. Representing adaptive and adaptable Units of Learning. How to model personalized eLearning in IMS Learning Design. In: Fernández Manjon, B., Sanchez Perez, J.M., Gómez Pulido, J.A., Vega Rodriguez, M.A., Bravo, J. (eds.) Computers and Education: E-learning - from Theory to Practice. Kluwer, Germany.

Cochrane, T., & Narayan, V. 2013. Redesigning professional development: reconceptualising teaching using social learning technologies. Research in Learning Technology, 21.

Conole, G. 2010. Learning design - Making practice explicit. In: ConnectEd 2010: 2nd International conference on Design Education, 28 June - 1 July 2010, Sydney, Australia

Conole, G. 2008. Capturing practice: the role of mediating artefacts in learning design.In Lockyer, L., Bennett, S. Agostinho, S. and Harper, B. (Eds) Handbook of Research on Learning Design and Learning Objects: Issues, Applications and Technologies, 187-207, Hersey PA: IGI Global.

De-la-Fuente-Valentín, L., Pardo, A., & Kloos, C. D. 2011. Generic service integration in adaptive learning experiences using IMS learning design. Computers & Education, 57(1): 1160-1170.

De Wever, B., Schellens, T., Valcke, M., & Van Keer, H. 2006. Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers & Education, 46(1): 6-28.

Dobozy, E. 2011. Typologies of learning design and the introduction of a "LD-Type 2" case example, eLearning Papers, 27(27): 1-11. [online] Available from http://www.openeducationeuropa.eu/en/node/111472 [accessed 5 August 2013]

Donald, C., Blake, A., Girault, I., Datt, A., & Ramsay, E. 2009. Approaches to learning design: past the head and the hands to the HEART of the matter. Distance Education, 30(2): 179-199.

Economides, A. A. 2008. Context-aware mobile learning. In The Open Knowlege Society. A Computer Science and Information Systems Manifesto, 213-220. Springer Berlin Heidelberg.

Glaser, B. G., & Strauss, A. L. 1967. The discovery of grounded theory: Strategies for qualitative research. Aldine de Gruyter.

Hannafin, M., Land, S., & Oliver, K. (1983). Open learning environments: Foundations, methods, and models. Instructional design theories and models, 2, 115-140.

Hara, N., Bonk, C. J., & Angeli, C. 2000. Content analysis of online discussion in an applied educational psychology course. Instructional science, 28(2): 115-152.

Hardre, P., & Kollmann, S. 2013. Dynamics of Instructional and Perceptual Factors in Instructional Design Competence Development. Journal Of Learning Design, 6(1): 60. doi:10.5204/jld.v6i1.106

Hernández-Leo, D., Villasclaras-Fernández, E., Jorrón-Abellán, I. M., Asensio-Pérez, J. I., Dimitriadis, Y., Ruiz-Requies, I., et al. 2006. Collage, a collaborative learning design editor based on patterns. Educational Technology & Society (Special Issue on Learning Design), 9(1): 58-71.

Hernández-Leo D., Abenia P., Moreno P., Chacón J., Blat J. 2011. Let's shake on it: can we support co-edition and sharing using diverse existing learning design editors within the same platform?. ASLD Workshop 2011, 13-14 October, London, UK.

Hewitt, J., & Scardamalia, M. 1998. Design principles for distributed knowledge building processes. Educational Psychology Review, 10(1): 75-96.

Hollan, J., Hutchins, E., & Kirsh, D. 2000. Distributed cognition: toward a new foundation for human-computer interaction research. ACM Transactions on Computer-Human Interaction (TOCHI), 7(2): 174-196.

Hwang, G. J., Sung, H. Y., Hung, C. M., Huang, I., & Tsai, C. C. 2012. Development of a personalized educational computer game based on students' learning styles. Educational Technology Research and Development, 60(4): 623-638.

IMS Global Learning Consortium. 2003a. IMS learning design specification. [online] Available from http://www.imsglobal.org/learningdesign [accessed 5 August 2013]

IMS Global Learning Consortium 2003b. Learning Design Information Model. Version 1.0 Final Specification. [online] Available from http://www.imsglobal.org/learningdesign/ldv1p0/imsld_infov1p0.html [accessed 5 August 2013]

IMS Global Learning Consortium 2003c. IMS learning design best practice and implementation guide. Final Specification, 1. [online] Available from http://www.imsglobal.org/learningdesign/ldv1p0/imsld_bestv1p0.html [accessed 5 August 2013]

Järvelä, S., & Häkkinen, P. 2002. Web-based cases in teaching and learning-the quality of discussions and a stage of perspective taking in asynchronous communication. Interactive learning environments, 10(1): 1-22.

Kali, Y. (2006). Collaborative knowledge building using the Design Principles Database. International Journal of Computer-Supported Collaborative Learning, 1(2), 187-201.

Kali, Y., & McKenney, S. (2012). Teachers as designers of technology enhanced learning. In J. van Aalst, K. Thompson, M. J. Jacobson, & P. Reimann (Eds.), The future of learning: Proceedings of the 10th international conference of the learning sciences (Vol. 2, pp. 582-583). Sydney, NSW, Australia: International Society of the Learning Sciences.

Kanuka, H., & Anderson, T. 2007. Online social interchange, discord, and knowledge construction. The Journal of Distance Education/Revue de l'Éducation à Distance, 13(1): 57-74.

Kenny, R., Zhang, Z., Schwier, R., & Campbell, K. 2005. A Review of What Instructional Designers Do: Questions Answered and Questions Not Asked. Canadian Journal Of Learning And Technology / La Revue Canadienne De L'Apprentissage Et De La Technologie, 31(1). [online] Available from http://cjlt.csj.ualberta.ca/index.php/cjlt/article/view/147/140 [accessed 5 August 2013]

Kim, C. 2012. The role of affective and motivational factors in designing personalized learning environments. Educational Technology Research and Development, 60(4): 563-584.

Kolodner, J. L., Camp, P. J., Crismond, D., Fasse, B., Gray, J., Holbrook, J. & Ryan, M. (2003). Problem-based learning meets case-based reasoning in the middle-school science classroom: Putting learning by design (tm) into practice. The journal of the learning sciences, 12(4), 495-547.

Koper, R. & Tattersall, C. 2005.Learning Design: A Handbook on Modeling and Delivering Networked Education and Training. Berlin: Springer-Verlang.

Kordaki, M., Papadakis, S., & Hadzilacos, T. 2007. Learning Design: the views of prospective computer professionals. In Proceedings of Informatics Education Europe II Conference, 2-11.

Lee, J., Park, O. 2007. Adaptive instructional systems. Handbook of Research for Educational Communications and Technology, pp. 469-484. J. M. Spector, M. D. Merill, J. Merrienboer van, and M. P. Driscoll (Eds.), New York: Routledge/Taylor & Francis Group

Luck, R. 2000. Does 'inclusive design' require an inclusive design process?. In Collaborative Design, 71-79. Springer London.

Magnisalis, I., & Demetriadis, S. 2012. Extending IMS-LD Capabilities: A Review, a Proposed Framework and Implementation Cases. In Intelligent Adaptation and Personalization Techniques in Computer-Supported Collaborative Learning (pp. 85-108). Springer Berlin Heidelberg.

Mavroudi, A., & Hadzilacos, T. 2012a. Broadening the use of e-learning standards for adaptive learning. In Advances in Web-Based Learning-ICWL 2012, 215-221. Springer Berlin Heidelberg.

Mavroudi A., Hadzilacos Th. 2012b. Implementation of adaptive learning designs, Annals of the University of Craiova, Series: Automation, Computers, Electronics and Mechatronics, 9 (2).

Merrill, M. D. 2002. First principles of instruction. Educational technology research and development, 50(3): 43-59.

Miao, Y. 2005. CoSMoS: Facilitating learning designers to author units of learning using IMS LD. In Proceediongs of the International Conference on Computers in Education (pp. 275-282). Singapore: IOS Press.

Mor, Y., & Winters, N. 2007. Design approaches in technology-enhanced learning. Interactive Learning Environments, 15(1): 61-75.

Mor, Y.; Craft, B. & Hernández-Leo, D. (2013). The Art and Science of Learning Design: Editorial. Research in Learning Technology 21

Mor, Y., & Mogilevsky, O. (2013). The learning design studio: collaborative design inquiry as teachers' professional development. Research In Learning Technology, 21.

Muñoz-Cristóbal, J. A., Prieto, L. P., Asensio-Pérez, J. I., Jorrín-Abellán, I. M., & Dimitriadis, Y. 2012. Lost in translation from abstract learning design to ICT implementation: a study using moodle for CSCL. In 21st Century Learning for 21st Century Skills (pp. 264-277). Springer Berlin Heidelberg.

Paquette, G., Léonard, M., Lundgren-Cayrol, K., Mihaila, S., & Gareau, D. 2006. Learning design based on graphical knowledge-modeling. Journal of Educational technology and Society, 97-112.

Paramythis, A., & Loidl-Reisinger, S. 2003. Adaptive learning environments and e-learning standards. In Second European Conference on e-Learning, 369-379.

Sampson, D., Karampiperis, P., & Zervas, P. 2005. ASK-LDT: a web-based learning scenarios authoring environment based on IMS learning design . International Journal on Advanced Technology for Learning (ATL), 2(4): 207-215.

Scrivener, S. A., Harris, D., Clark, S. M., Rockoff, T., & Smyth, M. 1993. Designing at a distance via real-time designer-to-designer interaction. Design Studies, 14(3): 261-282.

Silverman, D. 2000. Analyzing talk and text. Handbook of qualitative research, 2, 821-834.

Sing, C. C., & Khine, M. S. 2006. An Analysis of Interaction and Participation Patterns in Online Community. Educational Technology & Society, 9 (1): 250-261.

Sims, R. 2006. Beyond instructional design: Making learning design a reality. Journal of Learning Design, 1(2): 1-7.

Sims, R. C., & Koszalka, T. A. 2008. Competencies for the new-age instructional designer. Handbook of research on educational communications and technology, 3.

Specht, M., & Burgos, D. 2007. Modeling adaptive educational methods with IMS Learning Design. Journal of Interactive Media in Education, 2007(1).

Strauss, A.L., & Corbin, J. 1994. Grounded theory in methodology: An overview. In N. Denzin & Y. Lincoln (Eds.), Handbook of qualitative research, 273-285. Thousand Oaks, CA: Sage.

Valdivia, R., Nussbaum, M., & Ochoa, S. 2009. Modeling a collaborative answer negotiation activity using IMS-based learning design. IEEE Transactions on Education, 52(3): 375-384.

Van Merriënboer, J. J., Clark, R. E., & De Croock, M. B. 2002. Blueprints for complex learning: The 4C/ID-model. Educational Technology Research and Development, 50(2): 39-61.

Van Merriënboer, J. J., Kirschner, P. A., & Kester, L. 2003. Taking the load off a learner's mind: Instructional design for complex learning. Educational psychologist, 38(1): 5-13.

Voogt, J., Westbroek, H., Handelzalts, A., Walraven, A., McKenney, S., Pieters, J., & De Vries, B. (2011). Teacher learning in collaborative curriculum design. Teaching and Teacher Education, 27(8), 1235-1244.

Udas, K. 2009. James Dalziel - Learning Design and Open Source Teaching. Connexions Web Site. Available from http://cnx.org/content/m14733/1.5/ [accessed 5 August 2013]