The University of Wollongong’s first locally developed and hosted Massive Open Online Course (MOOC) “The Reluctant Mathematician” was run in the last four weeks of our summer holiday, prior to Autumn session in 2014. It was developed to lift maths skills at our university and also in the community – where maths skills continue to be a challenge and in some cases a source of stress. Internally the MOOC provided an alternative online way to support students who struggle with mathematics at university level, as a complement to the existing face to face small group workshops and individual consultations.

This paper describes the aims, rationale of the learning design, and evaluation of the MOOC. Data sources include enrolment/demographic data, analytics data of student patterns of use, assignment submission data, and qualitative feedback via online forms. It provides some support for recent non-binary definitions of MOOCs proposed by Lane (

The lowering of maths skills in our students and the community, and the removal of maths pre-requisites for university entrance are all part of a ‘maths skills crisis’ which has been on the Australian national education policy agenda for some time.

“The mathematics skills crisis is creating a vicious cycle that is slowly impacting on mathematics education. That is, fewer high school students are studying advanced or intermediate mathematics, which means fewer students are enrolling in university mathematics classes leading to a reduction in the number of mathematics teaching staff in universities and leading to lower numbers of enthusiastic, mathematics-qualified teachers in schools. The cycle will ultimately result in a shortage of skilled professionals in the fields requiring tertiary mathematics education, including engineering, science, finance and the actuarial profession, all of which are areas on which our society and economy depend for continued prosperity.” (

What this also means for universities, is that there is an increase in the number of students enrolling in university courses who are short on maths skills, and who struggle in the classroom across a wide range of disciplines including nursing, economics and finance, education and even engineering.

Staff in Mathematics and Statistics at the University of Wollongong (UOW) have been proactive and innovative since at least 2005 in progressing maths education to support students’ transition to university maths, and to lower failure rates in maths-related subjects at the university. There has been a chain of internal and then externally funded and collaborative grants that have built on this experience. (

From 2008–2011 the University of Wollongong and Central Queensland University as its partner institution undertook an ALTC Leadership project

The University of Wollongong developed a new eLearning Strategy in 2012–13 with the mission: “To connect technology-rich learning environments with transformative curriculum renewal to contribute to UOW’s goal of being a top 1% international university by 2025.” Open-Education was a significant new focus area of this Strategy. Open-education or open-learning refers to activities that either enhance learning opportunities within formal education systems or broaden learning opportunities beyond formal education systems (

Much has been written about the rise and definition of Massive Open Online Courses (MOOCs) and in particular the debate about content vs collaboration-focussed MOOCs ie xMOOCs v cMOOCs (Beavan et al. 2014;

The constructivist and / or connectivist cMOOC (

On the other side of the debate, there are also reports of students floundering, becoming overwhelmed and dropping out of MOOCs that have no teacher presence, minimal content or suggested learning pathways (

However it is timely to extend the debates about definitions of a MOOC to look at the fundamental question as to whether the MOOC is fit for purpose. In other words, is the design suitable and aligned with the aims of the MOOC, its learning outcomes and intended audience? What role do particular discipline challenges play in the design? And what motivates the intended students to succeed?

That really depends on who you think your intended learners are. Downes puts forward a case for MOOCs as something for advanced rather than novice learners.

“What we are trying to do with a MOOC is to

The ‘Reluctant Mathematician MOOC’ took a different approach. We designed for the non-expert and/or stressed maths learner/reviser and provided a highly scaffolded and structured learning space. However, the design and resources were also flexible enough to allow experts to dip in and choose the more challenging material. As we will see later in reviewing the student demographic data – the attraction of this MOOC to maths lovers and experts and their use to practice and maintain skills and provide encouragement for others was one of the surprise outcomes of the MOOC. The MOOC aimed to address maths weaknesses among students, similarly to other examples of using online learning via MOOC to enhance understanding in a particular STEM topic (

The learning outcomes were particular to the individual – we wanted students to leave the MOOC with the ability to successfully complete four types of mathematical problems that are foundational to further study in numerous university disciplines. In other words, ours was a task-based MOOC scaffolded and supported by video content, quizzing/feedback and optional engagement with peers via email or a Facebook set up for this MOOC if they chose. This is consistent with the new MOOC taxonomy put forward by Lane in 2012 where content, tasks and networking/social learning are not mutually exclusive but complementary, even if there is one main focus (

In addition, high degrees of scaffolding are suitable and helpful to deal with the situation of low self-efficacy and stress in the learners (

Albert Bandura introduced the notion of self-efficacy in learning in 1977 building on a strong foundation of educational psychology research throughout the 1960s and 70s. Self-efficacy was found by Bandura to be the greatest predictor of student success in learning i.e. not the student’s intelligence or IQ, but their belief in their ability to succeed and the consequential decision to apply sufficient effort to the learning tasks (

As Brennan has more recently summarised:

“High efficacy students tend to try harder, for longer. They overcome obstacles, can cope with failure, and continue to strive. They are less easily discouraged. They will be more ambitious, and more likely to achieve those ambitions in their learning.

Low self-efficacy learners tend to try less hard, and for shorter periods of time. They are more likely to blame themselves for lack of success, they are easily discouraged by failure, they strive less hard to overcome obstacles, are less ambitious, and achieve less.” (

Unfortunately, there exist (and persist) in the general community negative attitudes to mathematics, and anxiety and low self-efficacy to learning mathematics (

David Wiley’s four ‘R’s model (re-use, redistribute, revise, and remix) provides a useful definition of what an open education resource is, and this broad definition has been our aim in our consideration of the ‘openness’ of our MOOC and its resources (

We were lucky to already have a collection of Creative Commons licenced video lesson resources to begin with – as discussed previously; this collection was a key reason to choose the topic in creating our first MOOC. We linked through to a free and openly licensed text-book from the South African Siyavula Project (

The MOOC was built over 2 months of work by a single staff member (the author) with input from Dr. Caz Sandison from the School of Mathematics and Applied Statistics. Dr Sandison also provided a $2000 budget to hire a casual staff-member (and also a maths tutor) to create a series of 3–4 video quiz resources for each of the 4 maths topics covered by the MOOC. The MOOC can be viewed at the following URL

The idea to develop the Camtasia video quizzes came about in response to a problem – we did not have a quiz tool in our externally hosted and supported version of Wordpress. However as we thought through and planned the quizzes, we realised that the solution allowed for a greater degree of scaffolding and formative feedback than a standard quiz may have provided. In the end the video quizzes gave a re-cap of the technique to solve a maths problem, and broke subsequent questions down into parts asking the student to solve each step (i.e. answer a quiz question) before proceeding, with the video continuing after each quiz question was submitted and providing a correct method of thinking and working out that step before moving on. There were around 3–4 “stop points” in each video quiz for the student to test their understanding (i.e. complete a quiz question) before the video continued.

The materials were structured with four topics, one per week. Each topic had the same learning sequence, as can be seen in Figure

Starting with a theory refresher video lesson including recapping terms,

Moving to a series of short video lessons with a mathematician solving maths problems at the white board starting with easy ones before moving on to more complex problems

Next were the video quizzes – a half-half hybrid resource/quiz where the learners need to try their skills.

Lastly, a series of 50 maths problems – the “test yourself: practice questions” which eventually became to be considered “the assignment”. This task was substantial, and it required 2–3 hours to complete, and in the case of the novices it could take them pretty much a whole day, with breaks for reviewing lessons, text-book and/or resources.

We also designed a final optional assignment task asking the students to make their own video lesson, inspired by some recent advances in research in maths education (Hoban et al., 2009). We provided a list of links to tutorials in various methods of media production. This aspect of the design is outside the scope of this study.

The MOOC homepage showing pull-down menu navigation to the items in the learning sequence.

The learning sequence shows scaffolding in action, the first item is teacher demonstrates maths, the last item is student practices maths on their own, and there is a gradual reduction in support by the teacher during the course of the sequence such that the student gets used to doing aspects of the maths on their own until they can do it all on their own.

In a free open course where students engage for a wide range of reasons, we did not expect that all students would do all the assignments. Individuals could decide on how much they wanted to do, depending on what they wanted to get out of it. The use of a simple pull-down menu to list the links to all the items in the learning sequence made it simple for students to pick and choose between the various items. We expected that some students would start with the quizzes or perhaps the worked examples/lessons and go back to the theory refresher if they got stuck for example.

We needed to be creative with the marketing of the MOOC as for this pilot we had no access or facility to email current or potential students, and no marketing budget, beyond limited quota of an internal colour photocopier. Choosing a catchy title and humorous graphics targeting stressed maths students (Figure

The digital signage and posters used to promote the MOOC.

Small bookmark sized flyers were handed out to Science, Medicine and Health students at enrolment time by academic advisors.

In the month prior to the MOOC start-date we utilised a range of external and then internal marketing strategies including:

Media Releases aimed at parents of school leavers and new university students– gained attention on syndicated Radio, local TV News and large regional newspaper

Direct email – we are Foundation Partners in the Open Education Resource University (OERu) who are pushing towards improving access globally for education – we e-mailed colleagues in this international network who spread the word to their students

All staff email – “Do your students need maths skills to succeed in your course?” meant that some staff referred relevant students to the MOOC

Digital signage – targeting current students needing to refresh maths skills ahead of careers with maths dependencies

Flyers at Science, Medicine and Health enrolment days – including conversations with Faculty advisors who validated the need for the extra maths support

The title “The Reluctant Mathematician” worked well and coupled with a humorous website graphic used in all the promotions i.e. having a strong brand was key to gaining attention and interest.

We were expecting around 50 students might enrol in the fully online maths refresher course, however we were pleased to find around 200 students were enrolled at the start of the MOOC and in the following 2 weeks a further 50 students enrolled which we presume was due to positive word of mouth referrals as no additional marketing or promotional activities took place that could account for an additional burst of interest.

The online enrolment form gathered a range of demographic data.

About 85% of participants had English as their first language, with around 15% stating English was their second language. About 60% of participants were tertiary students refreshing skills ahead of study, with the remaining 40% made up of community members. After a flurry of overseas enrolments from South Africa (courtesy of email promotion to partners in the Open Education Resource University) the local media promotion took off and Australia/Oceania enrolments ended up at over 80%. There was no national or international marketing budget for this first MOOC, so we did not expect a significant international cohort.

The group who studied maths more than 20 years ago made up the largest group of students (at just under 30%), with a further 15% of students in each of the groups who had studied 3–5, 5–10 and 10–20 years ago. Recent school leavers made up around 10% of the cohort.

Open text comments provided at enrolment time gave extra information as to the motivation for doing the MOOC. Many students noted their stress and/or recent failure in compulsory maths subjects required for their course, and there were a number of parents and grandparents trying to get up to speed to help their kids/grandkids. There were a number of training teachers who expressed passion for teaching and wanting to look at different ways to help children learn. And there were also those who just loved maths and some who were “testing the water as a pre-decision activity to retrain in an area I should have chosen earlier in life.” Many students expressed reluctance to engage in maths learning, and one particular comment exemplified the degree of this reluctance, stating that their inability to follow mathematical instructions “really undermines my self-worth especially as a woman”. If we weren’t 100% sure of the need and demand for such a course at the time of starting this project, by now we felt a big responsibility to support quite a diverse range of learners.

All participants were emailed an eight page Guide to the MOOC, however by the end of Week 1 nobody needed help using the website and nobody took up the online “drop-in” coaching options that were heavily promoted to the cohort via email. The only emails received from two stressed students were those unable to login. This was because they are so used to logging in to eLearning products but in fact our MOOC was open to the extent that no login was required.

A Facebook page and an email group was set up to satisfy the needs of learners who like to communicate with peers, and about 25 students ‘followed’ the updates to the Facebook page, and about eight in the email group. The email group was private to those that signed up so we cannot see what happened there – however it is referred to in the open-text feedback so it was useful to at least one person. The Facebook page conversation consisted of a couple of stressed learners communicating with a couple of keen and experienced maths educators, and the encouragement they gave and received seemed to help keep the participants on track. One student commented that they had spent a couple of hours on the practice questions and only got one third of the way in, but “I won’t give up!”

Enrolments continued to roll in, and to satisfy students’ desire for instant feedback on their practice questions, at the end of the first week we added a facility to allow for electronic submission of an indicative sample of questions. Previously we thought we would just upload the solutions at the end of the first week – however with students enrolling late and moving ahead at their own pace there would be no one time we could upload the answers that would be suitable. We wanted to provide a facility for students to be automatically emailed the link to download the worked examples/answers rather than waiting for it to be posted online at a particular time. However we wanted them to make a serious attempt at the questions before getting the answers, so we used a standard web-form and asked for the submission of an indicative sample of questions.

Over the first few weeks it emerged that students considered the “Test yourself practice questions i.e. the 50 maths questions to be ‘the assignment’. The Facebook page and email discussion centred around this task. Email and Facebook feedback indicated that once the 50 practice questions had been attempted, the students felt satisfied. This early observation played out for the following weeks of the MOOC.

We collected demographic data via an online form at the time of enrolment which provided us with data about the nature of the cohort such as age, location and English skills. There was an open-text box on the form inviting students to tell us about their past maths experience or reasons for doing the course. These initial comments demonstrated the diversity of students, ranging from those with a high degree of expertise in mathematics, to parents wanting to further support their children’s learning in schools, and those considering mid-life career changes and needing re-training. The initial responses also showed anxiety or stress in many of the participants. Google analytics, YouTube statistics and media use data were gathered to gauge the students’ patterns of use.

A further pool of data was sourced through qualitative feedback via online forms presented at the time of assignment submissions.

The questions we were hoping to find answers for included:

Can students get out of it what they signed up for, i.e. does it match their intentions and meet their expectations?

Does being able to attempt maths problems in private provide a face-saving way to learn, reducing the sense of “I’m dumb” that can be reinforced in class settings when peers succeed but you don’t?

What percentage of students submitted the assignment (practice questions) per module? Is it somewhere around the 10% mark that we might expect based on current experiences with other MOOC providers?

Is there a demand for fully online learning opportunities for maths?

Assignment results: what spread of marks and average marks did the students achieve with their assignments? Did they succeed in mastering the topic?

Table

Student views of the content and the number of assignment submissions.

Module One | Module Two | Module Three | Module Four | Total | |
---|---|---|---|---|---|

Theory Refresher | 275 | 100 | 63 | 41 | 479 |

Worked Examples | 181 | 86 | 52 | 31 | 350 |

Assignment (50 Practice Questions) | Views: 234 |
Views: 143 |
Views: 68 |
Views: 73 |
Views: 518 |

Video Quiz | 169 | 56 | 21 | 18 | 264 |

Create a Maths Video | Views: 105 |
Views: 47 |
Views: 15 |
Views: 13 |
Views: 186 |

The marks achieved by the MOOC students (see Table

Assignment marks.

% students getting 60% or higher in the assignment | % students getting 70% or higher in the assignment | Median Assignment Mark | |
---|---|---|---|

Module One | 62% | 62% | 100% on 7 qs |

Module Two | 77% | 72% | 75% on 8 qs |

Module Three | 75% | 66% | 75% on 12 qs |

Module Four | 100% | 73% | 86.6% on 13 qs |

We decided to compare the whole cohort to those who submitted the practice questions (“The assignment cohort”) across all the demographic data. Figures

Compares the location of the students in the overall cohort to those who submitted the assignment.

Compares the students’ English language in the overall cohort to those who submitted the assignment.

Looks at the students’ main reason for doing the MOOC, and compares the % between the overall and the assignment cohort.

Looks at the time gap between taking the MOOC and studying Maths at school students, and shows the % between the overall and the assignment cohort.

Looks at how stressed the MOOC learners are about studying maths, and compares the % between the overall and the assignment cohort.

The majority of students were from Australia and Oceania, with English as their first language. Just over 10% of students had English as their second language, and a similar percentage of the English as 2^{nd} language students submitted an assignment. Only a tiny percent of students had English as their third language, and they did not submit an assignment. The sample size is too small to draw any conclusions from this. 60% of students enrolled in the MOOC to brush up on skills ahead of university or TAFE (vocational) further education, with around 20% doing it out of general interest with those concerned for employment and supporting their kids’ learning coming in at just under 5%. The general interest group was slightly up on the assignment submission rate, but not by much. Overall we can see that on these first 3 themes, there is no major difference between the whole cohort and those who submitted an assignment.

When we look at the last 2 themes however, some interesting differences are noticeable. Over 40% of the assignment submitters were from roughly the 38 plus age group, having studied maths more than 20 years ago, whereas this group represents less than 30% of the whole cohort. The very recent school leavers also submitted assignments at a higher rate. And pleasingly, 55% of students who submitted assignments were “a bit” stressed about learning maths as compared to just over 20% who were “not at all” stressed. And the “a bit stressed” learner group was also slightly over-represented in the assignment sample as compared to the overall cohort. There were some assignments submitted from those who were “quite stressed” and “extremely stressed” but at lower rates overall and as compared to the whole cohort.

The last set of data we had was the open-text comments for feedback submitted by students as they submitted their assignments. Apart from one student who didn’t like having to submit their assignment answers prior to getting the results (‘makes me feel like I’m back at school’), they were overwhelmingly positive about the learning experience, and valued different aspects as the following selection shows:

From the beginning of the marketing period this MOOC proved more popular than we expected. “Who would want to spend the last month of their summer holiday doing maths?” we wondered. We were really hoping to get the kind of numbers to make the effort worthwhile, i.e. more than the 10 or 20 that might attend a face-to-face series of maths skills workshops on campus. To have over 200 enrol far exceeded our expectations, and shows that there is demand for fully online learning opportunities, even within a campus-based university community where face-to-face learning experiences are the norm, particularly for undergraduates.

Getting almost an extra 52 students joining the group, making a total of 252 enrolled participants, after the course started, could only be put down to positive word of mouth feedback from students already in the MOOC. This and the lack of requests for support were also very good news for the design of the course and the website. That plus the significant number of hits on the website pages give us some indication that the course was easy to engage with and to use.

Participants submitted assignments at an overall rate of 18.3% (averaged across the four modules), significantly higher than the 10% completion rates expected of current MOOCs. The raw numbers are also impressive – a total of 98 assignments were submitted, indicating that that the program worked for a higher number of students than we would expect to be able to support in a face-to-face workshop. We can see from the comments made at time of submission that it worked to allow students to develop skills in the four maths topics for a diverse range of students, but particularly mature-age learners having a 10–20 year gap since studying maths, and those who were a bit stressed about learning maths.

The fact that just under half of our learners were “a bit” stressed about learning maths was in itself an achievement. It tells us that there is a demand for fully online learning, even for those who are not confident in learning the topic. The open-text comments noted that the fully online program allowed students to move through at whatever pace suited them, and that the video resources which allowed for stop-points were “better than face to face” in that you could really think things through in your own time before answering. These are promising findings not just for MOOCs but potentially for all online learning designs.

However we are not clear as to whether the more stressed students found these aspects as helpful. The data set is too small and the issue needs additional research.

We have a small budget to further develop this MOOC and the challenge is to decide what direction to take. Based on the feedback from the numerous staff who contacted us when we began to promote the MOOC we feel that there is potential to expand the number of topics, and customise for different disciplines adding more contextualised information about why the selected maths skills to be refreshed and practiced are important for that particular profession. Our next steps will be to hold some further discussions with these members of staff representing the various professions that depend on maths, to find out where the priorities lie in extending the MOOC.

Our evaluation showed that task and resource-based MOOCs can be appropriate designs for learning outcomes in foundational skills based courses, and that fully online learning opportunities can work to engage the non-confident and non-expert learner and help them work towards skills mastery.

The Reluctant Mathematician MOOC took as its starting point on the one hand, some issues and challenges that were particular to maths learners and on the other, many years of experience in developing audio-visual resources and bridging programs to support those struggling with maths at university. We designed for the non-expert and/or stressed maths learner/reviser and provided a highly scaffolded and structured learning space that was well received. However, the design and resources were also flexible enough to allow experts to dip in and choose the more challenging material.

This provided an engaging learning experience for a wide variety of university and non-university learners from recent school leavers through to those who had studied maths 10–20 years ago. The fully online format including the new video-quiz resources allowed students to move through the program at their own pace and in their own time, and for some this can be more effective than face-to-face study where you can stop and really think and try out a skill before the lesson proceeds.

Those who submitted the assignment generally did well and those who submitted their work were typically ‘a little stressed’ about learning maths indicating that fully online self-paced learning can work well for these students. The data is inconclusive about what works or does not work with the MOOC for those who are highly stressed about learning maths, and this is an area for future study.

Lastly, we feel it is timely to move away from binary definitions of MOOCs to a more nuanced conversation as to whether a MOOC is fit for purpose, whether the learning objectives can be met by the intended learners, and whether they can assist with outstanding issues particular to the academic discipline.

The author declares that they have no competing interests.

I wish to acknowledge and thank Dr Caz Sandison, from the School of Mathematics and Applied Statistics at the University of Wollongong without whom the Maths MOOC would not have gone ahead. I greatly appreciate her enthusiasm to try a new approach to teaching maths in fully-online mode and to try new resources and techniques which had emerged in commercial MOOCs. Dr Sandison had a significant role in the creation of a set of OER video resources as part of a previous project, and that set of resources and the Summertime Maths website (