Citation Details: Conole, G. (2002). Systematising Learning and Research Information. Journal of Interactive Media in Education, 2002 (7). ISSN:1365-893X []
Print: [PDF] [HTML]

Published: 25 July 2002

Editor: Martin Oliver (UC London, UK) []

Reviewers: Gordon Doughty (U. Glasgow, UK), Albert Ip (U. Melbourne, AUS), Peter Twining (Open U., UK)

Systematising Learning and Research Information

Grainne Conole

Research and Graduate School of Education
University of Southampton
Highfield Campus
SO17 1BJ

Abstract: This paper considers the ways in which information of relevance to the learning and research communities is organised and used. It contends that there is considerable overlap between the different types of online resources and information currently available within education. It describes some of the structured environments and data stores that have emerged in recent years, along with standards which are attempting to define the properties of discrete learning objects, through the specification of Learning Object Metadata (LOM). The paper contends that current developments of structured learning environments such as Managed and Virtual Learning Environments (MLEs and VLEs) are occurring on the whole in parallel to resource data stores, such as information gateways and portals. This discrepancy has arisen in part because these developments have occurred independently of one another and in part because there has to date been no rigorous definition of the underlying theoretical models. Furthermore, it argues that these predefined structured environments are unlikely to be sufficient to meet the information needs of users in different contexts. The paper goes on to describe an information toolkit, which provides a way of systematising information handling in learning and research, which helps users articulate information plans within specific contexts. The paper concludes with a description of two case studies which illustrate how this toolkit can be used.

Keywords: Toolkits, learning technology, information, managed and virtual learning environments, information gateways and portals, learning object metadata

1 Introduction

The rationale for this paper comes from a recognition that the increasing sophistication of online resources and virtual environments - far from making things easier for learners and researchers - can provide a biased view or filter out information that might be of relevance to the user. Furthermore, there is a need for users to think critically about their information requirements in a specific context and then assess the relevance of different online resources to meet these needs. This paper outlines a resource, which guides and supports the user from the process of thinking about their information needs to mapping these to a specific range of resources.

Learners and researchers have a potentially vast range of resources available to them through the Internet and associated technologies. But, as the Internet increases in size and complexity, so do the associated usability and navigational issues. A range of strategies has been used to manage online information and provide different searching protocols, navigational aids and maps. Nonetheless, information overload persists. This information overload problem is well recognised and a number of structured resources and environments attempt to address it, such as information gateways and portals, digital libraries and virtual learning environments. In principle, these managed environments, with tailored views for specific individuals (or types of users), are valuable, provided that they do not limit, restrict or omit information. Even so, it is unlikely that any one of these tools will be sophisticated enough to meet the needs of every individual. In reality, users may need to adopt more of a ?mix and match? strategy, selecting individual features from relevant sources to meet specific needs. What users need is a specific personal set of information, hand picked from this general set and tailored to their own requirements.

At the first meeting of the Learning Technology theory group (LT-Theory 2000), there was considerable debate about the use of definitions within the area of learning technology research. In particular, the discussion focused on the fact that, by its nature, this area of research draws on a range of research disciplines: cognitive science, educational theory, computing, psychology, organisational theory, etc. Ely has recently suggested that instructional technology is not yet a discipline (Ely 1999), stating that

the term, discipline, is usually reserved for areas of inquiry and application that have been established over time and follow established paradigms. There is likely to be a consistency in the basic beliefs, rationales and common principles that define the scope and structure of the discipline.

Multi-disciplinarity, in new emergent research areas, is in many respects a strength; however it also means that there is often no shared common language or set of definitions. This is evident currently in the learning technology domain, where the definition and scope of different types of online environments and resources are indistinct and, in some instances, overlapping. For example, the latest glossary of learning technology terms from the Institute of Electrical and Electronic Engineers (IEEE) states: ?the vocabularies of learning technology, education and information technology share an ever growing number of terms yet are far from unified? (IEEE 2002). This leads to confusion for the end users in terms of being able to clearly articulate their information needs and uses. In particular users are generally unclear about how to translate their information needs into the correct terminology in order to be able to locate the information they need. This problem is caused, in part, by the lack of consistency in the vocabularies used by different information providers. This paper provides a practical approach to addressing such needs, through the use of a structured decision-making resource for information handling, which enables users to develop tailored information plans for specific contexts. This paper will describe the philosophy behind the development of this information toolkit and report on the evaluation of its use to map the information needs of two different users.

The paper is divided into two distinct parts. The first half of the paper looks at the different ways in which information is organised and discusses some of the current issues of information overload and multiple information sources and in particular it outlines the problem of the complexity of locating information. The second half suggests some guidance on one approach to helping users address this complexity and describes a toolkit to help academic uses of information to access and manage their information needs.

2 The information landscape

We are all too aware of the impact of increasing information overload and in principle would agree that providing tailored views for specific individuals or types of users is valuable, provided that this is not at the expense of limiting or restricting information and omitting items which may be of importance. This information overload problem is well recognised and a number of tools and resources exist; such as search engines, information gateways and portals, digital libraries and online datasets and virtual learning environments. This paper contends that there is considerable overlap between these different systems and that what users really need is a specific personal set of information, hand picked from this general set and tailored to their own needs and requirements. The rationale for this paper comes from recognition that the increasing sophistication of online resources and virtual environments - far from making things easier for learners and researchers - can provide a distorted or biased view or filter out information that might be of relevance to the user. Furthermore it is clear that there is a need to enable users of online tools to think critically about their requirements and use of these resources in this context. This paper attempts to provide this type of guidance by presenting a framework that guides the user from the process of thinking about their information needs to mapping these to a specific range of online resources. It begins by contextualising this work and illustrating examples of online resources for learning and research.

2.1 Categorising online resources and information

In recent years developments in Information and Communication Technologies have led to a growth in the range of Internet-tools which can be used for learning and research. Some have now gained wide scale acceptance (for example, email); others seem to find either niche applications or are less pervasive than one might have first imagined (for example, video conferencing).

Research and development into the creation of digital libraries and online portals has developed to a large extent in isolation from research and development into the creation of online learning resources and virtual learning environments. Developments are now emerging into the creation of institution-specific online environments or resources and the development of tailored Managed or Virtual Learning Environments (MLEs and VLEs), information portals and gateways, and personalised desktops. An obvious question in relationship to these developments is, what defines and distinguishes a MLE from a VLE from an information portal or gateway? What are the components that make up a M/VLE or gateway? How much do we know or understand about how they are used or how well they meet user needs?

This lack of clarity and clear definitions of what differentiates VLEs, MLES, gateways and portals is a in part characteristic of learning technology as a new and emerging area of research, but is also systematic of a lack of understanding within the community and in particular the lack of a shared language with which to develop and consolidate ideas and definitions. The toolkit described in the second half of this paper is an attempt to address this by providing a mechanism for creating a shared meta-language and a means of structuring and categorising different information processes. The value of meta-language through the use of toolkits was also evident in the use of Media Advisor with a group of teachers from different subject areas. The study found that the toolkit allowed the practitioners to share and interpret each others understanding of learning and teaching within their subject domains (Oliver, MacBean et al. 2002).

This paper contends that we need to gain a better understanding of the information tools and methods that make up MLEs, VLEs, gateways and portals, in order to be able to fully utilise their potential and in particular how they can be used in different contexts by different users. The paper suggests adopting more of a ?pick and mix? strategy for creating personalised environments, rather than trying to produce an ?all-embracing? solution to meet the needs of everyone. The paper describes an Information Toolkit to facilitate this, which helps the user map information tools and methods to information activities. Use of this toolkit, supports users in critically evaluating resources according to particular needs. It can help the user to gain a better understanding of the potential of current technologies to support particular learning and research activities. The explicit mapping of information tools and methods to particular activities should also inform the debate on the development of future structured learning environments for particular groups of users or specific needs. This section describes the different types of structured learning environments and their constituent components. It outlines the way in which these are now being specified at both the macro-level, via a Learning Technology Systems Architecture (LTSA) and at the micro-level, through the definition of Learning Object Metadata (LOM).

2.2 Managed and virtual learning environments

In the last five years, Virtual Learning Environments (VLEs), which integrate communication tools (email, discussion boards, etc), with learning resources and assessment tools, have emerged in order to support learning. This has lead to the development of a range of home-grown institutional solutions, whilst offerings of ?Commercial Off The Shelf? (COTS) solutions (WebCT, Blackboard, Lotus Learning Space) abound. Although initially targeted to support learners, many of these are also being used as intranets or for information storage. Many of these purport to the notion of providing a ?holistic? learning environment and intimate that these types of environments are well suited to supporting student-centred learning and constructivist ideals. Whilst, Laurillard?s conversational model (Laurillard 2001) and her mapping of methods with educational activities, are not out of step with these notions, this should by no means be taken as read. These environments are equally able to support bad practice as well as good. Laurillard et al. have espoused the importance and role of narrative structure in supporting learning and, in particular, an investigation of the design features that ?afford? activities that generate learning (Laurillard, Stratfold et al. 2000). Put simply, it is not the environments themselves that support or enhance learning, but the ways in which they are used. Narratives to aid understanding, navigational guides and scaffolding strategies are all crucial, but can only be put in place once there is a clear understanding of the underlying components and how they can be used. This work suggests there is a need for a much clearer understanding of VLEs and the ways in which they are be used to support learning and that coherence will not arise until these components are understood. Many universities are now looking beyond the development of VLEs to more encompassing Managed Learning Environments (MLEs) and this introduces an additional level of complexity. The components of a Managed Learning Environment (MLE), which integrate or ?join-up? the VLE with the university?s management systems, might include a wide range of functional components - administrative information about courses, resources, support and guidance, collaborative information, assessment and feedback, and evaluation (taken from a report by JISC (ASSIST 2000). Whilst, as described above, VLEs at least purport to support a constructivist approach to learning, it is unclear what theoretical basis MLEs align to. Therefore, here, too, there is a need to understand the relationship between these components and the strategies which can be included to better support their use.

2.3 Information gateways and portals

A number of niche information gateways have developed. In the UK the main gateway for Higher Education is the Resource Discovery Network (, similar gateways exist in Europe and elsewhere (Renardus 2001). The term ?gateway? is used to describe a range of Internet sites that in some way provide access to other, predominantly Internet-accessible, resources. Gateways are intended to facilitate resource discovery by their target audience, to help their users find what they need via the Internet. ?Information Gateway? is a generic term that refers to a whole range of Web-based resource guides pointing to Internet information resources, whereas ?Subject Gateways? are subject-based resource discovery guides that provide links to information resources (documents, collections, sites or services), predominantly accessible via the Internet (definitions taken from (Renardus 2001). Resource description and subject classifications are the most important characteristics of such guides. In addition, it is becoming more common for publishers to make journals and books available online, as well as online data archives, collections, etc. Considerable thought is being given to how users can access and use these resources in a coherent way; a good example of this is a recent definition of a portal architecture by Powell. In his paper, he defines a subject portal as something that ???brokers? end-user access to a range of services, relevant to a particular subject area, over a number of application protocols (LDAPP, Whois++, Z39.50, ILL, FTP) [and]?provides ?discover?, ?locate?, ?request? and embedded ?use? ?functionality? (Powell 2000). Powell goes on to describe the architecture for a portal. He defines six types of ?player?; the end-user, presenters, coordinators, mediators, communicators and providers, and describes the roles and relationship between them. At the ?bottom? of the architecture are the information providers, such as subject gateways, data archives, service providers and collections. Clearly, these systems represent a rich variety of structures, supporting a range of applications. As with VLEs and MLES outlined above, there is a need to better understand the fundamental components of these systems and their affordances.

It is also evident that the boundaries between VLEs for learning and portals for research are blurring and that many of these systems are starting to provide additional functionality beyond their original scope. For example, the Social Science research gateway, SOSIG, primarily focuses on the development of a quality-assured set of resources of relevance to the social science research community. However it also includes a facility to develop personal profiles, ?My SOSIG?, and the ability to network with ?likeminds? using its grapevine facility. SOSIG is also linked closely to a set of online tutorials which provide an introduction to the Internet (Virtual Training Suite, available online at Similarly VLEs are often now used as gateways or portals to specific types of resources of relevance to a particular learning programme. These developments suggest that an evolution is in process, namely that an information source, once established, starts to adapt according to user needs.

The description above illustrates the rich variety of structured information environments that have developed over the past five years or so. It is evident that despite addressing different initial needs (for example resources for the research community or support for a distance learning programme), these systems are now converging in terms of functionality. In an attempt to better understand and define the different forms of structured learning environments, an overarching framework (the Learning Technology Systems Architecture, LTSA) has been developed as part of the IEEE standards work (IEEE 2002). The LTSA specification covers a wide range of systems (learning technology, computer-based training, electronic performance support systems, computer-assisted instruction, intelligent tutoring, education and training technology, metadata, etc.) and is intended to be pedagogically neutral, content-neutral, culturally neutral, and platform-neutral. It aims to provide a framework for understanding existing and future systems and promoting interoperability and portability by identifying critical system interfaces. The purpose of developing system architectures is to

discover high-level frameworks for understanding certain kinds of systems, their subsystems, and their interactions with related systems? An architectures isn?t a blue print for designing a single systems, but a framework for designing a range of systems over time and for the analysis and comparison of these systems? By revealing the shared components of different systems at the right level of generality, an architecture promotes the design and implementation of components and sub-systems that are reusable, cost-effective and adaptable (IEEE 2002)

In theory, therefore, it should be possible to map all MLEs, VLEs, portals and gateways to this overarching LTSA and in so doing see the relationship between these different systems. By more clearly articulating this underlying architecture and identifying the relationship between the different expressions of this architecture, we should be able to gain a better understanding of these systems and more importantly what features ?afford? activities that support learning or facilitate the gathering or using of information and resources.

2.4 Learning objects

At the other extreme to the concept of all-encompassing online environments is the idea is that one can define ?objects? as the basis for combining and re-purposing resources and data according to needs and interest, or for categorisation of information. The IEEE have developed a set of guidelines for definition Learning Object Metadata (LOMs) and an associated specification (LTSC 2001). This standard:

specifies the syntax and semantics of learning object metadata, defined as the attributes required to fully and adequately describe a learning object. A learning object is defined here as any entity, digital or non-digital, that can be used, re-used or referenced during technology-supported learning. Examples of technology-supported learning applications include computer-based training systems, interactive learning environments, intelligent computer-aided instruction systems, distance learning systems, web-based learning systems and collaborative learning environments. (LTSC 2001)

However, as Wiley points out the IEEE definition is very broad (Wiley 2002). Various attempts have been made to narrow the scope of the definition down to something more specific. The proliferation of definitions for the term ?learning object? makes communication confusing and difficult. Wiley concludes that:

An in depth discussion of the precise meanings of each of these terms would not add to the main point of this discussion: the field is still struggling to come to grips with the question, What is a learning object?

Similarly one of the reviewers of this paper comments:

I always feel a bit uneasy if there is not a conceptual distinction between the terms objects and resources. To me, "objects" implies more than just resources. For instance, consider a multiple choice item. The item itself (the wordings in the stems, the options and answer) is "resources". When the system can present this with interactivity (i.e. a learner can indicate a choice and submit the choice to some VLE), it is a learning object. Some may like to call the software component which provides the interactivity "object" and still makes a distinction between the "interactive" part and the "resource" part. ?. LOM can and should be used to describe ANY resource and hence enabling resources to be used effectively.

The LOM standard focuses on the minimal set of properties needed to allow learning objects to be managed, located, and evaluated. Relevant properties of learning objects include type of object, author, owner, terms of distribution, and format. Where applicable, learning object metadata may include pedagogical properties, such as teaching or interaction style, grade level, mastery level and prerequisites. This approach in essence assumes a positivist, rational view of information and knowledge. Underlying this approach is the vision that by having clear and specific definitions, it will be possible to search, acquire, evaluate, use, share, and combine different learning objects. At a practical level, this suggests that a student could build up their own tailored set of learning resources gathered from a range of sources and adapted to their particular learning needs. Educational establishments would be able to re-purpose their materials and exchange resources with other institutions. In essence these conventions could give us a ?grammar? for information, with MLE, VLEs, portals and gateways becoming expressions using this.

Nonetheless, there are still a number of unresolved issues with this positivist, rational approach. Firstly, there is concern over the conflict between the degrees of granularity that will be necessary for different learning objects to provide a sufficiently detailed description, whilst also providing sufficient flexibility of use. The scope of what defines a learning object is not clear, ranging from little more than raw digital assets through to large ?chunks? of learning materials or courses. Secondly, this static set of rules does little to tell us about what these LOMs mean in terms of what they can ?afford? for learning or research (Gibson 1979). If we can gain a better understanding of these objects and the ways in which they can be combined, we can go some way towards grouping and tailoring information to specific user needs. Then it would not matter if at the macro-level we talked about a VLE or a portal: the underlying architecture would be the same.

2.5 Summary

Thus information can be categorised on the one hand into some form of higher-level structure, designed to meet the needs of a specific type of user, or alternatively can be categorised in terms of learning objects, with tightly defined associated properties. However, the increasingly rich array of sources and types of information now available means that it is unlikely that any one VLE, MLE, gateway or portal is going to provide all the information needs for a particular user within a specific context. It is also unlikely that just defining resources in terms of a set of component learning objects, is going to provide enough support to enable users to derive maximum benefit from these resources or to support them in using these resources to meet particular needs. Furthermore, simply grouping these learning objects into different contexts to build tailored VLEs, MLEs, portals or gateways, won?t work adequately either. In reality, users need to construct their own information needs using a variety of strategies, which include aspects of the above as appropriate. The information toolkit outlined in the remainder of this paper is designed to help users to articulate their information needs and construct personalised information strategies for a particular need. It is a technique to enable users to engage critically and questioningly with the messy world of information in a way that reflects their own values, needs and beliefs.

Furthermore, Bruce' argues that there's more than one way to think about what counts as "effective use" of information, and that if we concentrate on technological issues (of access, architecture and functionality) we miss arguably the more important issues of building new knowledge or wise, authentic and personalised use of information (Bruce 1997). The information toolkit outlined in the next section attempts in part to address this.

3. The information toolkit

The previous section illustrates the complexity and range of online environments and highlights the need for guidance and support mechanisms to help users navigate, find and utilise online resources. This section describes one approach to providing this form of expert guidance, through the use of a decision-making resource, an information toolkit, which can be used to facilitate information handling and processing.

A range of resources to facilitate decision-making processes has been developed to support the use and integration of learning technologies. In essence, decision-making resources range from highly restrictive ?templates? or ?wizards?, which provide high levels of support and step-by-step guidance but little possibility of user-adaptation, through to ?theoretical frameworks?, which provide a context and scope for the work but leave the user to devise their own strategy for implementation. Between these extremes lie a range of resources, including checklists, guidelines and step-by-step tutorials. This paper describes a toolkit for systematising information needs. A detailed definition of our use of the term toolkit is defined and illustrated elsewhere (Conole 2000; Conole and Oliver 2001).

The information toolkit adopts a similar approach to two related toolkits, ?Media Advisor? (Conole and Oliver 1998; Oliver and Conole 2000) and ?Evaluation Toolkit? (Conole, Crewe et al. 2001; Oliver, MacBean et al. 2002). It provides a means of mapping information resources against types of information activity (grouped in the toolkit into four categories: gathering, using, communicating and evaluating). This gives the user a clearer view of the resources they are using, why they are using them, and allows them to form their own tailored information plan. The toolkit guides the user through the process of articulating their information needs and results in the production of an information plan for a particular task. The ?scope? of the task is one of the first stages of working through the toolkit; a task could range from considering all the information needs for a course module, a research programme or a development project.

The information toolkit consists of the following key steps:

  • Scope. Articulation of the scope of the information plan ? who is this for, at what level, how long will it be used for?
  • Purpose. In this step the user considers the four information activities and specifies the relevance (or not) of each to this particular plan and the specific purpose of each activity.
  • Mapping. This final stage involves mapping the tools and methods to the information activities to produce the Information plan.

3.1 Scoping the plan

In this section of the toolkit the user articulates the scope of the information plan. This scoping is important because it will help the user to filter out inappropriate information sources, which may be too difficult, too rudimentary, or of secondary importance. The user may choose to describe the scope in some detail or with a rough sketch, whichever is appropriate. The scope helps them to focus on what they are gathering and using the information for; are they seeking information for a review, writing a report or essay? or conducting a piece of research work?

Table 1: The Scoping Table: Defining the scope of the information plan


Illustrative examples


At a minimum the scoping stage should provide a description of the area of interest. Is this plan to be produced to support a student on a particular module? Is it information being gathering to support a research project, or relevant material for a journal review? Is it concerned with all the information needs associated with a consortium-based development project, with a view to forming a shared project resource?

Primary stakeholder

This is most likely to be the person designing the information plan. Details here could include something about the person; are they a researcher, an undergraduate, a project manager? These details could be important in terms of given an indication of the quality and the level of the information plan, if it were reviewed by anyone else, for example as a case study for other users.

Secondary stakeholder(s)

These are others who might be interested in the plan. Examples might include fellow students on a course. For example, students might divide the information ?mining? for a set of modules on a course and agree to use them together as a shared resource. Another example of a secondary stakeholder could be the tutor for a course or a reviewer for a research project.


What level of difficulty is the information to be pitched at, for example degree or graduate level? What type of audience is it aimed at, for example learners, researchers or a commercial audience?


Is this for a particular task over the next six months or is it a general information plan for an ongoing research interest?


Who will be gathering and using the information apart from the end-user? Is there a research assistant involved in data collection, for example?


In this section the user is encouraged to articulate any other relevant information to include in the scope of the plan, which will help them to focus on particular uses and sources of information in the next section.

3.2 Defining information activities

This section considers the different types of information activity that are important in the plan. The classification used here is similar to the approach used in the Media Advisor toolkit (Conole and Oliver, 1998; Oliver and Conole, 2000), where four groupings were used to classify educational activities (delivery, discussion, feedback and activity). Of course this approach is a simplification: information activities can be classified in a number of different ways and to a greater degree of granularity (indeed Laurillard?s conversational framework identifies twelve tutor-student interactions (Laurillard, 1993)). However, as with Media Advisor, the purpose here is not to provide a rigid classification, but to give enough guidance for the user to make informed decisions. A reasonable starting point therefore is that manipulation of information can be grouped into the following four categories:

  • Seeking or gathering
  • Processing or using
  • Communicating or disseminating
  • Evaluating or monitoring

The Learning Technology Systems Architecture specification described earlier defines information in terms of data processes, stores and flows (see Figure 4, page 23 of The four information categories outlined above map to this as outlined in the figure below. Data stores act as both vehicles for storing or receiving information. A user can seek, gather, evaluate or monitor information from a data store, or alternatively can communicate or disseminate information back (for example by depositing information in a data store). Therefore data stores are linked to different data processing activities, via gathering, communicating or evaluating.

Subject learning resource

Table 2 illustrates the four information categories defined above in terms of their typical roles in both learning and research. (For simplicity these classifications will be referred to in future as ?seeking, using, communicating and evaluating?.) The user will then use this table to begin to identify which of the four information activities are relevant to their own information plan. In some cases all four of the activities will be involved - for example in the production of an undergraduate research project, where the user needs to gather background information on the research topic, process relevant information and communicate the results in the form of a dissertation paper. The evaluation in this case will be by the tutor in terms of marking the report. In other cases, a plan might focus on only one or two activities, such as carrying out a literature review at the start of a research project.

Table 2: The Classification Table: Classification of information activities





Gathering or seeking

To gather information to support learning or specific activities.

Information about a course, the university, other resources, relevant related materials, examples of introductory guides.

To gather information for a literature review, to support a research paper, or to keep abreast of current developments. Information about research centres or individuals of relevance to a particular research activity.

Using or processing

Manipulation of data or resources to carry out a specific learning task.

Analysis of data collected using standard qualitative and quantitative research tools and methods.

Communicating or disseminating

Student-tutor: for checking, for support, administration or feedback.

Student-student: collaboration, sharing of ideas, or clarification.

Communication with research peers to share or explore ideas, or with project partners, or to the wider research community to disseminate findings.

Evaluating or monitoring

Assessment of students, monitoring and feedback on progress.

Research assessment exercise, peer review through journals and conferences, success in securing funding.

3.3 Mapping the tools and methods to information activities

This part of the toolkit gets the user to brainstorm what information will be gathered and for what purpose. The user is presented with a range of information sources and resources (mailing lists, email, journals, books, seminars, etc) and asked to focus on the four types of information activities (seeking, using, communicating and evaluating). They are asked to consider:

  • What types (or specific) information they need to gathering
  • What they will do with it
  • How they will communicate it and to whom
  • How they intend to monitor/evaluate or be monitored/evaluated

In Media Advisor, users then map the learning and teaching methods (including both ICT and traditional approaches) against educational activities, to give a picture of their own learning and teaching approach. Similarly, Table 3 outlines how different types of ?information? can be classified under four main headings (seeking, using, communicating or evaluating). By providing a map of the relationship between information activities and sources, the user can apply this knowledge to articulate their own individual map for specific types of activity. The map is not restrictive: the user can define additional tools or methods and can choose to adapt the information activities if they wish. The idea is to give them some structure to their thinking. It aims to guide the user through these stages and helps them to articulate different resources.

Table 3: The Mapping Table: Mapping of tools and methods with information activities

Tools and Methods

Information activities





Subject-gateways/digital libraries

The user includes in each cell details

of the way in which each of the tools

and methods are used with respect

to each of the information activities.

Only those cells, which are relevant

to the user?s information needs,

are completed. This map gives a quick

visual representation of the information

plan and can be updated as necessary.

Online resources (course materials, tutorials, CAL-packages)

Mailing lists, Newsgroups, Discussion boards

Conferences, seminars, Presentations/workshops/lectures

Online Journals, books, etc

Paper-based books, leaflets, etc

Search engines




Tracking or assessment software

Data gathering using research instruments (questionnaires, observation, video logs, focus groups, etc)

Databases or data archives

User defined tools and methods

On a first iteration through this table the user is likely to identify generic sources of information, highlighting the types of tools and methods which are likely to be of most use and eliminating irrelevant ones. On subsequent iterations the user can start to build a picture of specific examples (eg ?this journal?, such and such a person), until they are satisfied that they have a comprehensive plan.

3.4 Case studies

This section describes the results of using the toolkit in two case studies. The methodology used was to guide each user through the toolkit, prompting additional explanations where necessary and following up with a semi-structured interview (to access the users? impressions of the toolkit and to gain insight into its value). The case studies are intended only to be illustrative of the ways in which the toolkit might be used. It would be interesting to extend these studies to different types of users and different information searching tasks. In particular a case study on a first-year undergraduate student starting a piece of learning that could be more ?surface? than ?deep? would be a good example of the use of the toolkit.

3.4.1 Case study 1: online resources for a learner

The learner described here is a student studying for a MSc in Information and Library management. The toolkit provides a mechanism for the student to articulate the different types of information resources being used to support the course and the ways in which each of these is being used. It can be used as a starting point to gather information, but the more significant benefit is that it can be used as an ongoing tool for learning, which can reflect the students? changing work pattern. By sub-dividing information use and processing, the toolkit helps the student to think clearly about each type of information and its relevance to them, whilst also allowing them to articulate the ways in which their work will then be represented.

The student was first briefed that the aim of working through the toolkit is to provide structured guidance on the gathering and use of information, and that by the end the student should have developed a comprehensive information strategy to support a particular learning activity or course. The four fundamental aspects of the types of information activity that the toolkit supports are then introduced and explained, using the Scoping Table (Table 1) as an illustration. The different types of information sources, resources and channels are then presented as outlined in the Classification Table (Table 2). The student then brainstorms the types of ways in which (s)he gathers and uses information and, using the Mapping Table (Table 3), begins to complete an information plan. An example of the partially completed information plan for this student is illustrated in Table 4. In this example the table does not include specific details of links for reasons of anonymity, but in reality this would be an important part of the plan. The information plan forms a summary of links to relevant resources, details of conference and event dates, and deadlines for papers or abstracts. In essence therefore the plan is a form of extended ?bookmarking? system which brings together online and other sources of information.

The process took about 30 minutes to complete; the student had little trouble working through the toolkit or completing the tables and appeared to need minimum clarification. She was clear about the function and value of the toolkit and could see its relevance to her own working practice.

This is a helpful tool, because you are working on your own as a part-time student and don?t necessarily have the same support as a full-time or undergraduate student might.

In particular, she commented that having completed case studies of the toolkit to refer to would be valuable, although she did express some reservation that these might hinder creativity and impose some restrictions on the way a user would then complete their own plan.

Having the case study approach gives you ideas of where to look, but you would need to be careful as this could also limit or restrict your scope. Anything that makes things easier is good!

In her opinion, the toolkit would require a small investment of time initially, but this would be beneficial in the long term as it would provide a structured record of relevant resources which can easily be updated or adapted.

It would take longer to work through this tool initially and quicker after, however this is time well spent as this is the most important part of a research project like this.

Finally, although she could see the benefit of the toolkit and the final information plan, she was unconvinced that it would fundamentally alter her strategies for searching and using information.

Useful to have the information all together in this format. I don't think this tool would have made me do the research any differently though.

3.2.2 Case study 2: mapping information resources for a researcher

This case study was carried out with a PhD researcher in philosophy. As with the first case study, the researcher was provided with an initial overview of the purpose of the toolkit, its components and the expected outcomes of the process. Guidance was provided through each stage of completing the information plan, prompted with further explanations or examples where necessary.

Table 5 shows the researcher's partially completed information plan. The researcher began by defining the scope of his information plan, using Table 1 as a guide. After brainstorming the ways in which he used different types of information, using Table 2, he mapped his tools and methods with the information activities to produce an information plan, as illustrated in Table 5. Running through the toolkit with this researcher took about thirty minutes, and was followed by a short semi-structured interview to gain first impressions on its potential value. The researcher found the toolkit easy to use and was clear about its purpose; for example, when he was introduced to the second part on identification of information activities, he said:

As a researcher I can relate very easily to the examples of different types of information activities given as examples for research.

In some places he was unsure which cell to put an entry in:

A difficulty with this is that each resource plays so many roles, it can be difficult to attach a resource to an individual cell.

However, he did not feel it obstructed his information plan. One strategy he used to get round this was to combine cells across one or more information activity. This user adaptation is an important underlying assumption of these types of decision-making resources. This process of guiding the user through a set of choices, whilst maintaining user freedom by allowing them to customise or add to entries is an integral feature. He clearly found the information toolkit valuable and observed the following:

Part of the role of a researcher is as a ?librarian? to the subject area, to introduce other researchers to the area. This toolkit helps to divide up and recommend relevant resources according to the four categories. It makes me think of how to prioritise and the overview helps me to articulate how to prioritise the balanced use of these different resources. I could imagine giving conditional recommendations, i.e. using my knowledge to fill in the gaps of other researcher?s information plans. I could also imagine using this system as a basis for organising my online bookmarks and address books into these categories.

4 Conclusion

This paper has argued that we do not yet have a clear enough understanding of the way in which information can be best structured to support particular learning or research activities. It has reviewed the different ways in which information can be structured or defined both at a macro-level (in the form of the Learning Technology Systems Architecture) and a micro-level (by tightly defined learning objects and associated metadata). However, it contends that there is still a need for users to critically assess their own information needs in context. The paper has described an information toolkit which can be used to guide learners and researchers through a structured process of thinking about the ways in which they gather and use information to support particular activities. The toolkit has been piloted with users from both domains and initial evaluation feedback has been positive. By specifying the terms of reference for the area in the form of a toolkit the user is guided through the thought processes and sequence stages which are relevant to the problem. An additional benefit is that the toolkit can be used to generate a suite of these types of information plans. These plans can be used as templates for new users, which they can then adapt and update for their own needs.

Evaluation with users so far suggests that there are three potential levels of use for this toolkit. Firstly, it can be used by an individual to articulate an information plan for a particular task, however large or small. An initial plan can be developed quickly and then iteratively build up over time. Secondly, information plans can be shared with peers or aggregated to form a series of templates for particular information areas, in the same way that the Evaluation Toolkit can be used to generate evaluation case studies for common types of evaluation. Thirdly, construction of these types of information plans gives an insight into the information requirements of different types of learners and researchers, and a critical analysis and evaluation of these patterns would give a fascinating insight into the cultures of different information users and the development of new user interfaces to existing information resources. It is also important not to neglect associated issue of quality in relation to the use of the toolkit. For example at what point does the user judge the quality of the resources identified and do they have the necessary skills to be able to make those judgements?

One of the benefits of toolkits is that they make it possible to move away from an "expert knows best" mindset to one in which each user is viewed as an expert in their context, which helps to empower the user to develop the necessary skills to make their only locally-situated judgements. Furthermore it is important to stress that toolkits are not solutions, they are designed to be educative in nature. The user has to build their own representation of knowledge sources and therefore they need to think about what's available, why it might be good, and how it all relates - tasks that are automated in an expert system (cf. wizards). So toolkits require more effort by the user, but there are longer-term educative pay offs. One potential shortcoming of this approach is that it assumes that the user is aware of all possible sources of information that may be relevant to them. One solution would be to develop a more interactive form of the toolkit which would include reference materials and guidance on different sources of information which the user could dip into as needed. This is similar to the approach adopted in the Evaluation Toolkit, where the user is guided through the processes of data collection and analysis through a series of filtered resources and further information (Conole, Crewe et al. 2001). The issue of whether or not a ?pick and mix? strategy for gathering and using information as proposed in this paper is better than the alternative one stop shop of a unifying point for information as purported in many portal developments and in the recommendations from the SoURCE evaluation (Beetham, Taylor et al. 2001) is not yet clear and will need to be investigated further. Clearly one of the strengths of the one-stop approach is that it provides a simple and structured interface for the users. A disadvantage is that the scope has to by definition be predefined. An advantage of the toolkit approach is that it allows the user to tailor their information plans and as the model is flexible the user can adapt and update as appropriate. A disadvantage is that this presupposes that the user has the necessarily information seeking and handling skills to best utilise this and that they are aware of different information sources. It is worth ending this paper by noting the perspective of one of the reviewer of this paper who states that:

In the user needs analysis for the Main SoURCE Evaluation Report on the Re-usable Educational Software Library (reported in Beetham, Taylor & Twining 2001) a clear demand was expressed for one integrated access point for resources (relating to learning technology practice). This seems to be in conflict with the approach being advocated in this paper. Clearly, in the short term, a pick and mix approach is the only viable one. However, I would argue that we need to be looking to move towards much greater integration of information sources in the future (though I acknowledge the complexity of this - particularly in the light of the context specific nature of metadata).

Whether or not this is true remains to be seen, but it is clearly evident that a substantial amount of research and developed is still needed before the dream of a comprehensive rich use of the wealth of information now available on the Web and elsewhere is realised and but to best effect.

Acknowledgements: I would just like to note my thanks to the referees for their very detailed and helpful comments on this paper. The paper has been updated significantly as a result of their comments and is much better as a consequence. My thanks also to Martin Oliver for providing lots of valuable feedback and for, as always, making me think about what I haven?t written (too!) hard.

Table 4: Case study 1: partially completed information plan for a masters student

Information Plan: To support a masters dissertation on user education on the Web.

This plan will focus on the information plan for the final dissertation module. This was a critical module in the masters, as failure to pass this would have meant failure In the overall masters

Scoping details

Primary stakeholder: The learner completing the masters programme. This is a p/t course of study. The learner is a full time project manager of a large online resource for Business and Economics students. Secondary stakeholders: A major secondary stakeholder was a fellow colleague, who the learner collaborated extensively with and discussed findings of mutual interest. Also the tutors and peers at work (through the perceived benefit of completing a relevant professional qualification), Timescale: 90 hour module, part time over 9 months, Level ? postgraduate/masters level, Resources: The researcher, the supervisor and a core of related PhD researchers.

The subject matter was based on Internet research and hence there were very few formal books and journal articles, a significant body of the information was in a more informal format, via mailing lists and through anecdotal evidence, which made traditional research data gathering difficult.


Information activities





The purpose of gathering information was to find out what was going on in the field to test the hypothesis of the research project and to compare findings, and the learners perspective with the wider body of information in the field.

The learner did all her own primary research (which consisted of the development and use of a questionnaire and semi-structured interviews). In addition, she used the findings from the research literature and from colleagues to compare her results.

Production of a final report for the dissertation, summarising the results and findings.

Assessed by course tutors and external examiner for the programme as part of the masters.

Tools and Methods

Information activities





Subject-gateways/digital libraries

PICK (now defunct)

Online resources (course materials, tutorials, CAL-packages)

CTI Library and management information resources

Examples of tutorials that other university had development in this area

The Internet detective tutorial

The VTS suite of tutorials

Mailing lists, Discussion boards, Newsgroups




Conferences, seminars, Presentations/workshops/lectures

IFLA and the online papers from the conferences

Online Journals, books, working papers, etc

Managing information

Aslib proceedings

Journal of academic librarianship

Academic libraries as high tec gateways

Journal of Information Science

Paper-based books, journals, leaflets, etc

Colleagues dissertation report on a related topic


Library associate record

Search engines

Google, yahoo


Distributed research instruments via email to the targeted group of subjects


The course Intranet ? including submission details and dissertation guidelines


A colleague?s (who had completed a similar masters the year before) bibliography on a related topic

Validation of initial analysis with peers

Update on progress with course advisor

Tracking or assessment software

Data gathering using research instruments (questionnaires, observation, video logs, focus groups, etc)

Developed own research instruments ? questionnaire and semi-structure interview

Databases or data archives



User defined tools and methods

Excel to analysis data and identify emergent themes

Production of dissertation report

Submission of report for assessment

Note: the acronyms used in this table are not expanded as it is not the resources themselves that are of interest in this paper,

but the concept of the ways in which information resources can be mapped to the information toolkit categories.

Table 5: Partially completed information plan for a postgraduate research student

Information Plan: To support a thesis on ?Epistemic values? and related research papers

Scoping details

Primary stakeholder: the researcher, secondary stakeholders: PhD supervisor, other PhD researchers in the field, Level: postgraduate, Timescale: 3 or more years, Resources: The researcher, the supervisor and a core of related PhD researchers


Information activities





Primarily a literature search, identifying related researchers in other institutions, also online papers and relevant journals. Looking for illustrative examples from other disciplines.

Application of a range of qualitative research techniques, summarising and comparing information, evaluating arguments and critiques.

Submitting ideas or works in progress to other active researchers in the field, acquiring further literature recommendations, submitting to relevant formal and informal publications and dialogue.

Finding out about journals and conferences to which this work can be submitted. Preparation for the writing of the final PhD thesis and subsequent viva.

Tools and Methods

Information activities





Subject-gateways/digital libraries

SOSIG and other gateways to journals of relevance and working paper. Dedicated search engine for Philosophical research ? Hippias

?My sosig? entry describing the researcher?s interest and calling for interdisciplinary work with Psychologists

SOSIG and other gateways to journals of relevance and working paper.

Online resources (course materials, tutorials, CAL-packages)

Course materials on this area of research help to form an overview of the research area and the information collected, from selected graduate level course on topics such as the philosophy of science.

Mailing lists, Discussion boards, Newsgroups

Philosophy news service ? table of contents of major journals;

Local mailing lists on the philosophy and theory of decision making

Conferences and event information and call for papers

Philos-L mailing run from Liverpool.

Conferences, seminars, Presentations/workshops/lectures

British Society for Philosophy of Science

Royal Institute of Philosophy

Local one day conferences

Online Journals, books, working papers, etc

Regular sources include: Syntheses, British Journal of the Philosophy of Science

Working papers found through Hippias

Online book: Probability ? the logic of Science

Paper-based books, journals, leaflets, etc

Definitive texts include: Logic of decision; Betting on theories, Philosophy of Science

Search engines

Google, altavista, local search engines at institutions of interest


Key contacts to whom researcher emails idea ? related researchers in the area ? both local and international



Members and other postgraduates in the Philosophy department

Tracking or assessment software

Data gathering using research instruments (questionnaires, observation, video logs, focus groups, etc)

Databases or data archives

Online catalogues of other universities

Episteme links ? a gateway which has an online conference database

User defined tools and methods

University online journals listing,

Note: the acronyms used in this table are not expanded as it is not the resources themselves that are of interest in this paper,

but the concept of the ways in which information resources can be mapped to the information toolkit categories.


ASSIST, J. (2000). Final Report on the Managed Learning Environments Workshop. [cited]

Beetham, H., J. Taylor, et al. (2001). SoURCE Evaluation Report. Milton Keynes, The Open University. [cited] [cited]

Bruce, C. (1997). The Seven Faces of Information Literacy, Adelaide, Auslib Press. [cited]

Conole, G. (2000). Resources for Supporting Decision Making. First Research Seminar of the Learning Technology Theory Group. ALT-C 2000, Manchester. [cited]

Conole, G., E. Crewe, et al. (2001). A Toolkit for Supporting Evaluation. ALT-J, 9, (1), 38-49. [cited] [cited] [cited]

Conole, G. and M. Oliver (1998). A Pedagogical Framework for Embedding C and IT into the Curriculum. ALT-J, 6, (2), 4-16. [cited] [cited]

Conole, G. and M. Oliver (2002). Embedding Theory into Learning Technology Practice with Toolkits. Journal of Interactive Media in Education, 2002, (8): Special Issue on Theories for Learning Technologies. <>

Ely, D. (1999). Towards a Philosophy of Instructional Technology: Thirty Years On. British Journal of Educational Technology, 30, (4), 305-310. [cited]

Gibson, J. J. (1979). The Ecological Approach to Visual Perception, Houghton Mifflin. [cited]

IEEE (2002). IEEE Learning Technology Standards Committee. [cited] [cited] [cited]

Laurillard, d. (2001). Rethinking University Teaching: A Framework for the Effective Use of Educational Technology. London, Routledge. [cited]

Laurillard, D., M. Stratfold, et al. (2000). Affordances for Learning in a Non-Linear Narrative Medium. Journal of Interactive Media in Education, 2002, (2), 1-17. [cited]

LTSC, I. L. T. S. C. (2001). Standard for Information Technology - Education and Training Systems - Learning Objects and Metadata. LTSC. [cited] [cited]

LT-Theory (2000). First Research Seminar of the Learning Technology Theory Group. ALT-C 2000, Manchester. [cited]

Oliver, M. and G. Conole (2000). Assessing and Enhancing Quality Using Toolkits. Quality Assurance in Education, 8, (1), 32-37. [cited] [cited]

Oliver, M., J. MacBean, et al. (2002). Using a Toolkit to Support the Evaluation of Learning. Journal of Computer Assisted Learning 18, (2), 199-208. [cited] [cited]

Powell, A. (2000). DNER Portal Architecture. Positional Paper for the Distributed Network of Electronic Resources (DNER). [cited]

Renardus (2001). Gateways Defined. Positional Paper from the European Consortium. [cited] [cited]

Wiley, D. A. (2002). Connecting Learning Objects to Instructional Design Theory: A Definition, a Metaphor, and a Taxonomy. The Instructional Use of Learning Objects. In D. A. Wiley (Ed.), Association for Instructional Technology and the Association for Educational Communications and Technology. [cited]