A QTI editor integrated into the netUniversité web portal using IMS LD

Ecaterina Giacomini Pacurar1, Philippe Trigano2, Sorin Alupoaie3

1 Université de Technologie de Compiègne UMR CNRS 6599 Heudiasyc BP 20529

60205 Compiègne FRANCE


2 Université de Technologie de Compiègne UMR CNRS 6599 Heudiasyc BP 20529

60205 Compiègne FRANCE


3 Université de Technologie de Compiègne UMR CNRS 6599 Heudiasyc BP 20529

60205 Compiègne FRANCE


Commentary on: Chapter 10: Integrating Assessment into E-learning Courses. (Joosten ten Brinke,Gorissen & Latour, 2005)

Abstract: This paper presents our research work concerning the development of a QTI player and QTI editor that are integrated in a web portal, called netUniversité, to support teachers in desiging educational Web sites based on pedagogical scenarios represented in IMS LD. In the first part of this paper, we introduce the context of our research work. Then we present a general overview of the netUniversité web portal. Finally, web describes the development process of the QTI player and editor as well as the QTI player integration mechanism into the netUniversité navigator module.

Keywords: assessment using LD, exercises templates, XSL, educational web sites by using IMS LD

1. Introduction

In this paper we present our research work concerning the development of a QTI (IMS QTI, 2003) player and QTI editor that are integrated in a web portal, called netUniversité, to support teachers in desiging educational Web sites based on pedagogical scenarios represented in IMS LD (IMS LD, 2003). Our application is a tool that helps staff having few skills in web development to develop their educational Web sites.

Our work complements Joosten-ten Brinke, Gorissen and Latour's research work (Joosten ten Brinke et al., 2005) regarding assessment and its relation with learning design and how QTI (Question and Test Interoperability) and Learning Design can be integrated in an application.

In the first part of this paper, we introduce the context of our research work. Subsequently, the second part of this paper presents a general overview of the netUniversité web portal. The third part describes the development process of the QTI player and editor as well as the QTI player integration mechanism into the netUniversité navigator module (IMS LD player).

2. The context of our work

Our work finds its origin in project CEPIAH (Design and Evaluation of Interactive Documents for Human Learning), developed within our research team. Before presenting our objectives and other issues, we make a short presentation of CEPIAH. In this project we developed an interactive guide accessible on the Web, composed of three modules: Design help, Evaluation help and web sites Pedagogical Models. The objective of this interactive environment is to help the authors of educational Web sites in the design and the evaluation of their prototypes. The main role of these three modules is to guide users towards web development skills (Giacomini and Trigano, 2004).

After a first validation of CEPIAH guide (Trigano and Giacomini, 2004) we had some discussions with teachers of various disciplines to the Université de Technologie de Compiègne (UTC). These helped us know their practices of teaching and their opinion regarding the software tools used in web development. We observed that the majority of the teachers expressed a reserve in using these tools. This attitude arose either because of a lack of technical skills, or because of a lack of the time necessary to develop online courses (from teaching and technical point of view).

All these aspects convinced us to evolve the functionalities of our interactive guide CEPIAH into an environment that assists the design of educational web sites (online courses) in order to support teachers who do not have competences in web development. The new functionalities are realized in a web portal called netUniversité, integrated in CEPIAH guide. This new direction of research led to some questions.

Which functionalities should we integrate with netUniversité to allow the easy development of online courses? Once that we decided on this first aspect of functionalities, other questions remain to be posed: How to generate these structures? How to represent the pedagogical scenarios? Which standard would represent better these scenarios? How can we give to the user the possibility of choosing the pedagogical elements that he wishes to integrate in his web site? In order to respond to these problems we have defined our research objectives, some of them described in this paper.

A first objective was to develop a tool that will allow the user: to automatically generate educative web site structures based on pedagogical scenarios, to add and edit the pedagogical content and to administrate the created courses. This application must also allow the teacher to consult the activities performed by learners for a specific course. A second objective was to determine the types of pedagogical scenarios and the types of graphical interface in order to conceive two interactive questionnaires (a pedagogical one and another related to the graphical interface) used in the process of automatic generation of course structures.

Two other important objectives were to make an analytical study concerning, on the one hand, the subject of standards of representing pedagogical content and, on the other hand, the theories of learning used to set-up the theoretical framework for the development of pedagogical scenarios. Therefore, in order to facilitate the user interaction, our main objective was to develop an intuitive graphical interface for the generation of web site structures, the content editor and course administration.

3. netUniversité - architecture

As we mentioned in the first section of this paper, one of our objectives was to conceive an application that supports the development of educative web sites (online courses).

The target users of our tool are teachers who have few IT skills, particularly in web development. Thus, we believe that one of characteristics of this tool is its intuitive interface and usability. Its functionality must be transparent in order to avoid disturbing the user in the process of course building. We took all these aspects into account during the process of design and development of our application. In the following sections we present a general description of netUniversité and we justify our choices concerning the standard of representation of the pedagogical content.

One of our objectives is to take into account and to respect the standard that we consider more adapted for representing the pedagogical content in the context of our research project. In order to achieve this goal we made a bibliographical study on the various norms and standards like LOM (IEEE LTSC, 2001) and IMS LD (IMS LD, 2003). The results of this study influenced us to choose IMS Learning Design in order to have a standardized representation of the education web sites managed with netUniversité.

General overview of our application

In order to help any course author who has no competences in computers to conceive their educational web sites (online courses) we developed a tool made up of three principal parts presented in figure 1: a module of dynamic generation of educational web site structures, a module for editing and administrating the generated courses as well as a module of navigation (IMS LD player) allowing the users to visualize and participate to the courses. We put together the functionalities of these modules in only one tool, the web platform netUniversité.


Figure 1. General presentation of netUniversité portal

By using this web platform, the teacher has the possibility to generate structures of educational web sites and publish the course content into these structures, then visualize and manage his courses. The students can visualize and participate to courses using the navigator (IMS LD player) integrated in netUniversité. This application is to be used by the teachers in universities (high education). It can be considered as support for a normal course, but also as support for distance learning.

One of our problems mentioned in the section §1 was related to the way we generate the structures of online courses. In agreement with our research objectives, we chose to generate these structures starting from the responses to two interactive questionnaires (a questionnaire concerning the pedagogical aspects of the course and another questionnaire related to its graphical interface attributs). Thus, as shown in figure 2, the system uses the responses to these questionnaires in order to generate the pedagogical structures desired by the user (the teacher). In consequence, we represent each online course managed in netUniversité by two XML files: one representing the pedagogical structure of the course (with respect to IMS LD standard) and other containing the GUI (graphic user interface) attributes of the course.

Figure 2. Generating from two questionnaires

Our pedagogical questionnaire makes possible for the user to dynamically choose the pedagogical elements that will be integrated in his or hers educational Web site. This questionnaire is structured on three levels of granularity: a question itself, its reformulation as well as a deepening related to this question (see figure 2). In the deepening, we present a short synthesis of the theoretical concepts of teaching and training, connected to this question, and also we explain the consequences of choosing this answer on the final course structure.

The analysis of 175 educational web sites (accessible on UTC's intranet) enabled us to determine various types of possible interfaces. Several criteria are taken into account: navigation starting from the menus, the shapes of the buttons of navigation within the various pages of the web site, the colors of the menus, the buttons, etc. Starting from this typology we designed a questionnaire, called web site interface questionnaire, that facilitates the choice of the final web site GUI attributes,.

The problem of the complexity of the conditions used to generate this structure led us to a knowledge-based system. The number of entries (represented by the possibilities of responding to these two questionnaires) and the great number of possibilities concerning the teaching models determine us to develop a rules-based system. Knowledge is stored into an XML file, in a format similar to a JESS rule. It will be transformed into this language and will be added in the inference engine before being treated by the generator. In order to generate the pedagogical web sites structures we use a list of primary pedagogical models (i.e., basic models) also named building blocks. These building blocks are used to build the final course structures. The inference rules in the knowledge base specify the way in which the blocks are put together (figure 4). These blocks are in fact IMS Learning Design elements defined on multiple levels (plays, learning activities, environments etc.), which can be created by default at the time of the generation or conceived before and saved in XML files with respect to this standard.

Each resulting model of course structure is associated with an identification number. We have developed three levels of rules corresponding to three deep levels of IMS LD. The rules used to define the root and the identification number of a model represent the first activation level. The rules of the second level are used to associate a play to the root, depending on the responses to the questions concerning the pedagogical elements of the web site. The third level of rules associates an activity structure (or a simple activity) to a play.

In the case of the rules that constitute the first level of activation, the root is specified by a fact named root that includes the identification number of the root and the ID of the root pedagogical model (a Learning Design object) (figure 3).

Figure 3. Definition of a fact « root »

The rules of this type have as premises the answers to the questions. According to these answers the system creates the facts root as well as the facts that specify the pedagogical concepts used in order to build the course (figure 4). These are used by the rules of third level to choose the primary pedagogical models of activity structure.

Figure 4. Rule for the first level

Below we give an example of elementary brick (building block) that can be integrated into a course generated from the responses to the questions of two questionnaires referred in the preceding paragraphs of this paper. The play "Course" is composed of only one pedagogical act that associates an activity structure to a role (figure 5).

Figure 5. Representation of online courses

The section of the XML diagram referring to the components contains static and reusable elements of a unit of learning: roles, learning/support activities, activity structures and environments. In our example (figure 6) the activity structure is built of another two activity structures representing the chapter 1, respective chapter 2. The two structures are connected in sequence. This means that in order to pass to chapter 2 (activities of type description of the course content, resolution of simple tests, simulation, demonstration) one must initially finish the activities of chapter 1 (activities of type introduction, resolution of simple tests, description of the course content).

Figure 6. Components represented into a unit of learning

The IMS LD document includes the organization of the pedagogical content. The resources representing this content are referenced by elements of type <item>. The resources are declared in the section <resources> which specified the URL that indicates their location (figure 7).

Figure 7. Representation of resources

4. A simple QTI editor integrated in netUniversité

4.1 netUniversité: user-end overview

After presenting the conceptual and methodological framework that we used for the implementation of the module of dynamic generation, this section gives a short presentation of the netUniversité platform from the user point of view.

As we can observe in figure 8, once connected to the system the teacher can generate a new course and visualize the list of the courses he created. From the home page, the teacher can create a new course by clicking on the link "Génération d'un site web pédagogique". The effect will be the execution of the two questionnaires necessary for the generation of a pedagogical web site.

Figure 8. Home page for staff

Once the course structure is generated, the teacher can edit the course by modifying its structure and adding/modifying the pedagogical content. A screenshot of the editor interface is presented in figure 9.

Figure 9. Editor home page

Thus, using the editor interface the user modifies in fact the XML file describing the unit of learning in IMS LD. When the teacher interacts with the interface, the module "Editor" modifies and saves interactively the XML file.

4.2 QTI editor: description of our approach

Essential functionality from a pedagogical point of view contained in netUniversité is the possibility of integrating interactive exercises in the courses. In order to obtain this functionality we conceived and developed a QTI Player and a QTI editor of the exercises based on standard IMS Question & Test Interoperability. At the beginning we specified six patterns of exercises: multiple choice with "fill in the blank", matrices based on multiple answers, simple multiple choices, fill in the blank, true/false and open answer (large text area).

The access to this QTI editor is done starting from the interface of the course IMS LD editor (see figure 10).

Figure 10. An example of the course content IMS LD editor

4.2.1 Standard IMS Question&Test Interoperability - representing the interactive exercises

The QTI language, defined by IMS, is a part of the same family of specifications as IMS Learning Design. It describes the basic structure necessary to represent in XML the questions and tests of evaluation. In our tool we implemented the "lite" version of this specification that allows the representation of the six types of exercises: text areas, multiple choice, true/false, matrices for "single choice", open answer (large text area), etc. All the tests represented in this personalized version of QTI are also valid for the complete version.

In the QTI "lite" version implemented in our tool, a test corresponds to a QTI structure "assessment" (see figure 11.a). The test is composed from one or more sections of exercises/questions (items).

Figure 11. a) Structure of QTI test. b) QTI exercise

From the QTI point of view, a question/exercise (item) is composed of two parts: a "presentation" part and a " result processing" part (figure 11.b). The "presentation" contains all information necessary to represent the content of the question (texts, buttons, zones of text etc). The "result processing" defines the method of evaluation of the score obtained for this question.

The content of a QTI question (figure 12a) is represented as a succession of text paragraphs (material), response-button (response_lid) and response-text (response_str). The response-button can be of type: checkbox (multiple choice), simple radio button (one choice) and matrix radio buttons (multiple single choices). We have also the possibility of grouping these elements in a sub-section (flow) (figure 12b). If the content of a question is integrated into a sub-section, the content elements will be presented in the same paragraph. Thus, we can build for example exercises with "fill in blank" responses integrated into a phrase.

Figure 12. a) The content of QTI question. b) The structure of QTI sub-section

The QTI language also offers the possibility of indicating in the "result processing" (resprocessing) section of each item, a method to calculate the score related to the question (figure 13). The key concepts used are the variables and conditions. The variables are declared in the part "outcomes". In the implemented version only one variable is defined - by default the score obtained to this question.

Figure 13. The score calculation for a QTI question

The general structure of a condition (respcondition) is: IF reponse=x or no answer THEN set or add a value to a variable. Thus, the conditions give the possibility of interactively calculating the score for a question according to the responses of the user to this question. The correct answers are specified in the section "conditionvar".

In the implemented QTI version we have two types of evaluation: evaluation by question and evaluation by response. In the case of evaluation by question we consider a total score attributed when all the answers are correct. This situation is represented from the QTI point of view by only one condition (section resprocessing) that sets a certain value to the variable score (if the answers are correct). In the other case (evaluation by answer) for each answer we add a partial score to the final score. From the QTI point of view we define a condition each answer to be evaluated. Thus, if an answer is correct we add a certain value to the final score of the question.

4.2.2 QTI Player - interpreting the standard IMS QTI

The QTI player is implemented as an XSL style-sheet. The goal is to display the test content in an HTML format conforming to the QTI specification.

Figure 14 a) Question represented in QTI b) The output of the QTI player for this question

An example of this kind is illustrated in figure 14 which illustrates the way the QTI player display (14.b) a question represented in QTI format (14a). We can see that the element "material" is used to represent the text. Each answer is specified by the element response_label integrated into a response_lid (used for the answers-button). The role of the XSL style-sheet is to transform the content represented in XML into a displayable web format such as HTML.

Currently, the types of the exercises, we propose, are: fill in the blank, open questions, true/false, multiple choices, etc. Figure 15 shows a screenshot with a part of this QTI editor.

Figure 15. An example of a part of QTI editor

After the user includes the desired element into the exercise, he can edit the content and specify the correct responses. In the case of evaluation by response, he will add also the scores for each of the correct answers. Figure 16 represents the case of this type of evaluation. In order to specify the score the teacher must click on the small icon associated with the correct answer. A gray zone will be displayed, which includes: an option to define the type of answer (incorrect or correct) and an option to edit the score (in the case of correct answers) as shown on figure 16.

Figure 16.Editing the content for an exercise (evaluation by response)

4.2.3 Integrating the QTI player into the netUniversiténavigator (IMS LD player)

The IMS LD player considers the QTI XML files defining the evaluation tests as simple resources. Thus, these files are referred in "resources" section of the unit of learning like all the resource documents. The IMS LD player differentiate this type of resource by using the attribute type (in our case "qti") specified where the resource is declared (figure 17).

Figure 17.Declaring a QTI resource into th unit of learning

Therefore, the navigator knows (using the resource type) that in the central page of the course it must display the test represented in QTI file, and uses the XSL style-sheet in order to transform XML content into HTML.

After the user answers to all questions, the result of the test is obtained (figure 18) starting from its answers and from the computed score for each question specified in QTI file. The module "Test Processing" which implements this calculation process checks firstly if the user did not already complete the test. This situation is indicated by the value of the score stored in the database.

Figure 18.Computing the resulted score for a test represented in QTI

If the score is valid (positive or zero), then the user has already completed the test. If the score is negative the test is not yet completed. In the first case the user cannot complete the test the second time and the player only displays his final result (figure 19). In the other case, the user is invited to complete the test. After the test is finalized the result is stored in the database as an IMS LD locpers property (specific for a user participating to a course). The title of the property used to indicate the score is built using the test title.

Figure 19. The case when the test was already completed

5. Conclusions

In this article, we presented our research work concerning the design and the development of the QTI player and QTI editor both integrated in the netUniversity web portal. We are continuing the development of this first version of the QTI editor by adding the news kinds of exercises as matching, ordering, etc. Also, we are going to integrate a new content edition functionality that will enable teachers to write mathematical equations. Moreover, this last option can be used by teachers in order to propose adequate exercises for their students.

We have created a first experiment by testing our web portal with the teachers near of our university as well as the University "Aurel Vlaicu" in Romania. In this first experimentation stage, our goal consists of presenting the netUniversity portal to them and explaining its use. Then, in order to find out their opinion about the ease of use, we ask them to answer an evaluation questionnaire. The results of this evaluation can help us to detect the deficiencies of our prototype in order to address these shortcomings.

Acknowledgements: This thesis benefits from the financial support of the Pole of research STEF (Systems and Technology for Education and Training) of the Picardy Region, within the framework of the State/Region plan.

6. References

Giacomini, E., and Trigano, P. (2004). Flexible navigation for the pedagogical hypermedia design and evaluation improvement. In proceedings World Conference on E-Learning in Corp., Govt., Health., & Higher Education.

IMS LD (2003). IMS Learning Design Best Practice and Implementation Guide. IMS Global Learning Consortium, Inc.

IMS QTI (2003). IMS Question and Test Interoperability, Information model, Best Practice and Implementation Guide, XML Binding, Schemas, Version 1.2.1 Final Specification IMS Global Learning Consortium Inc. Retrieved January 19, 2004 from http://www.imsglobal.org/qti/

IEEE LTSC (2001). (IEEE P1484.3 GLOSSARY WORKING GROUP). IEEE Learning Technology Standards Committee Glossary, draft standard, copyright 2001 by the Institute of Electrical and Electronics Engineers, Inc.

Joosten ten Brinke, D., Gorissen, P., and Latour I. (2005). Integrating Assessment into E-learning Courses, Chapter 10, in Learning Design: A handbook on Modelling and Delivering Networked Education and Training. Rob Koper & Colin Tattersall (Eds.), ISBN 3-540-22814-4 Springer Berlin Heidelberg New York, 2005.

Koper, R., and Olivier, B. (2004). Representing the Learning Design of Units of Learning, In Journal of Educational Technology & Society, IEEE Learning Technology Task Force, Vol. 7, n° 3.

Trigano, P., and Giacomini E.(2004). Toward a Web based environment for Evaluation and Design of Pedagogical Hypermedia. In journal of Educational Technology & Society, IEEE Learning Technology Task Force, Vol. 7, n° 3.