## Introduction

The lowering of maths skills in our students and the community, and the removal of maths pre-requisites for university entrance are all part of a ‘maths skills crisis’ which has been on the Australian national education policy agenda for some time.

“The mathematics skills crisis is creating a vicious cycle that is slowly impacting on mathematics education. That is, fewer high school students are studying advanced or intermediate mathematics, which means fewer students are enrolling in university mathematics classes leading to a reduction in the number of mathematics teaching staff in universities and leading to lower numbers of enthusiastic, mathematics-qualified teachers in schools. The cycle will ultimately result in a shortage of skilled professionals in the fields requiring tertiary mathematics education, including engineering, science, finance and the actuarial profession, all of which are areas on which our society and economy depend for continued prosperity.” (**Professions Australia 2008**).

What this also means for universities, is that there is an increase in the number of students enrolling in university courses who are short on maths skills, and who struggle in the classroom across a wide range of disciplines including nursing, economics and finance, education and even engineering.

Staff in Mathematics and Statistics at the University of Wollongong (UOW) have been proactive and innovative since at least 2005 in progressing maths education to support students’ transition to university maths, and to lower failure rates in maths-related subjects at the university. There has been a chain of internal and then externally funded and collaborative grants that have built on this experience. (**Porter 2005**; **Porter 2007**; **Aminifar et al. 2005**; **Aminifar et al. 2006**). This work has focussed on developing methods for producing video-based maths lessons and the creation of the Summertime Maths website (University of Wollongong, 2015), which were used to deliver a range of maths bridging courses.

From 2008–2011 the University of Wollongong and Central Queensland University as its partner institution undertook an ALTC Leadership project *Building leadership capacity in the development and sharing of mathematics learning resources across disciplines across universities* (LE8–783). An outcome of this grant and additional funding by the University of Wollongong Teaching and Learning grants 2008 has been the development of a collection of peer-reviewed maths video resources (**Open UOW, 2013**) that can be freely shared. All resources are licensed under creative commons (licence: Attribution-NonCommercial-Share Alike 3.0 Australia) to enable lecturers and educators to use and adapt the resources with the permission of the developers but with recognition (**Porter 2014**).

The University of Wollongong developed a new eLearning Strategy in 2012–13 with the mission: “To connect technology-rich learning environments with transformative curriculum renewal to contribute to UOW’s goal of being a top 1% international university by 2025.” Open-Education was a significant new focus area of this Strategy. Open-education or open-learning refers to activities that either enhance learning opportunities within formal education systems or broaden learning opportunities beyond formal education systems (**D’Antoni 2009**). Goal 1 was to be Openly Connected to our communities – the first of 4 high-level goals. A new role Manager, Open Education was created, focusing on building capacity for open-education at the University, and working with the Open Education Resource University (OERu) to develop full length and accredited open subjects. The University also moved to pilot Massive Open Online Courses (MOOCs), or shorter open online courses. Two MOOCs were developed with external partner Open2Study and in order to test internal capacity and use of existing eLearning tools for Open-Learning a decision was made to develop one in-house MOOC using up-cycled Open Education Resources (OERs) addressing maths skills issues. This became *The Reluctant Mathematician* (**Lambert, 2013**) which can be found at http://blogs.uow.edu.au/reluctantmathematician/.

## MOOC taxonomies

Much has been written about the rise and definition of Massive Open Online Courses (MOOCs) and in particular the debate about content vs collaboration-focussed MOOCs ie xMOOCs v cMOOCs (Beavan et al. 2014; **Ross et al. 2014**; **Siemens 2012**). The argument for cMOOCs is about getting beyond a transmission and content-based model of teaching and placing the locus of control for student learning with the student, rather than with the teacher (**Siemens 2012**).

The constructivist and / or connectivist cMOOC (**Koutropoulos, 2013**) supporters have noted how different a student centred, collaborative MOOC can be, and discuss the different types of skills that they foster. “So while students who have grown up with the typical command-mode style of learning, it is not unreasonable to assume that students raised on MOOCs will have mastered the different set of skills. Students are adept at learning to follow orders when they are given a steady diet of orders; it is reasonable to assume they will learn to take responsibility when they are given responsibilities.” (**Downes 2012**).

On the other side of the debate, there are also reports of students floundering, becoming overwhelmed and dropping out of MOOCs that have no teacher presence, minimal content or suggested learning pathways (**Kop 2011**, **Kop et al. 2011**). These are what I think of as “choose your own adventure” MOOCs. But as Kop et al. have shown, too much choice can be de-motivating for some learners, in the way that too rigid a structure can be for others.

However it is timely to extend the debates about definitions of a MOOC to look at the fundamental question as to whether the MOOC is fit for purpose. In other words, is the design suitable and aligned with the aims of the MOOC, its learning outcomes and intended audience? What role do particular discipline challenges play in the design? And what motivates the intended students to succeed?

That really depends on who you think your intended learners are. Downes puts forward a case for MOOCs as something for advanced rather than novice learners.

“What we are trying to do with a MOOC is to *create an environment* where people who are more advanced reasoners, thinkers, motivators, arguers, and educators can practice their skills in a public way by interacting with each other. In such an environment, people can learn by watching and joining in.” (**Downes 2012**). This is similar to a definition of a Community of Practice where experts rub shoulders with novices, and provide a kind of community education service by mentoring and supporting them (**Beaven et al. 2014**).

The ‘Reluctant Mathematician MOOC’ took a different approach. We designed for the non-expert and/or stressed maths learner/reviser and provided a highly scaffolded and structured learning space. However, the design and resources were also flexible enough to allow experts to dip in and choose the more challenging material. As we will see later in reviewing the student demographic data – the attraction of this MOOC to maths lovers and experts and their use to practice and maintain skills and provide encouragement for others was one of the surprise outcomes of the MOOC. The MOOC aimed to address maths weaknesses among students, similarly to other examples of using online learning via MOOC to enhance understanding in a particular STEM topic (**Jiang et al. 2014**).

The learning outcomes were particular to the individual – we wanted students to leave the MOOC with the ability to successfully complete four types of mathematical problems that are foundational to further study in numerous university disciplines. In other words, ours was a task-based MOOC scaffolded and supported by video content, quizzing/feedback and optional engagement with peers via email or a Facebook set up for this MOOC if they chose. This is consistent with the new MOOC taxonomy put forward by Lane in 2012 where content, tasks and networking/social learning are not mutually exclusive but complementary, even if there is one main focus (**Lane 2012**).

In addition, high degrees of scaffolding are suitable and helpful to deal with the situation of low self-efficacy and stress in the learners (**Klinger 2005**). Low self-efficacy is a well-known phenomenon with maths skills and indeed there are over a decade’s worth of research data involving all OECD countries that measure the degree to which students’ beliefs in their own mathematics skills manifests in themselves feelings of helplessness and anxiety around mathematics which impact their motivation and effort as well as their performance (**OECD 2013**).

Albert Bandura introduced the notion of self-efficacy in learning in 1977 building on a strong foundation of educational psychology research throughout the 1960s and 70s. Self-efficacy was found by Bandura to be the greatest predictor of student success in learning i.e. not the student’s intelligence or IQ, but their belief in their ability to succeed and the consequential decision to apply sufficient effort to the learning tasks (**Bandura 1977**).

As Brennan has more recently summarised:

“High efficacy students tend to try harder, for longer. They overcome obstacles, can cope with failure, and continue to strive. They are less easily discouraged. They will be more ambitious, and more likely to achieve those ambitions in their learning.

Low self-efficacy learners tend to try less hard, and for shorter periods of time. They are more likely to blame themselves for lack of success, they are easily discouraged by failure, they strive less hard to overcome obstacles, are less ambitious, and achieve less.” (**Brennan 2013**).

Unfortunately, there exist (and persist) in the general community negative attitudes to mathematics, and anxiety and low self-efficacy to learning mathematics (**Biller 1996**; **Coben 2003**; **Klinger 2005**). So we were keen to see if we could design and deliver a MOOC that might help us meet some of these challenges, and engage a diverse variety of maths learners – including those stressed about learning maths.

##
*The MOOC* design process

David Wiley’s four ‘R’s model (re-use, redistribute, revise, and remix) provides a useful definition of what an open education resource is, and this broad definition has been our aim in our consideration of the ‘openness’ of our MOOC and its resources (**Hilton III et al. 2010**)

We were lucky to already have a collection of Creative Commons licenced video lesson resources to begin with – as discussed previously; this collection was a key reason to choose the topic in creating our first MOOC. We linked through to a free and openly licensed text-book from the South African Siyavula Project (**Siyavula Technology Powered Learning, N. D.**)

The MOOC was built over 2 months of work by a single staff member (the author) with input from Dr. Caz Sandison from the School of Mathematics and Applied Statistics. Dr Sandison also provided a $2000 budget to hire a casual staff-member (and also a maths tutor) to create a series of 3–4 video quiz resources for each of the 4 maths topics covered by the MOOC. The MOOC can be viewed at the following URL http://blogs.uow.edu.au/reluctantmathematician/.

The idea to develop the Camtasia video quizzes came about in response to a problem – we did not have a quiz tool in our externally hosted and supported version of Wordpress. However as we thought through and planned the quizzes, we realised that the solution allowed for a greater degree of scaffolding and formative feedback than a standard quiz may have provided. In the end the video quizzes gave a re-cap of the technique to solve a maths problem, and broke subsequent questions down into parts asking the student to solve each step (i.e. answer a quiz question) before proceeding, with the video continuing after each quiz question was submitted and providing a correct method of thinking and working out that step before moving on. There were around 3–4 “stop points” in each video quiz for the student to test their understanding (i.e. complete a quiz question) before the video continued.

The materials were structured with four topics, one per week. Each topic had the same learning sequence, as can be seen in Figure 1:

- Starting with a theory refresher video lesson including recapping terms,
- Moving to a series of short video lessons with a mathematician solving maths problems at the white board starting with easy ones before moving on to more complex problems
- Next were the video quizzes – a half-half hybrid resource/quiz where the learners need to try their skills.
- Lastly, a series of 50 maths problems – the “test yourself: practice questions” which eventually became to be considered “the assignment”. This task was substantial, and it required 2–3 hours to complete, and in the case of the novices it could take them pretty much a whole day, with breaks for reviewing lessons, text-book and/or resources.
- We also designed a final optional assignment task asking the students to make their own video lesson, inspired by some recent advances in research in maths education (Hoban et al., 2009). We provided a list of links to tutorials in various methods of media production. This aspect of the design is outside the scope of this study.

The learning sequence shows scaffolding in action, the first item is teacher demonstrates maths, the last item is student practices maths on their own, and there is a gradual reduction in support by the teacher during the course of the sequence such that the student gets used to doing aspects of the maths on their own until they can do it all on their own.

In a free open course where students engage for a wide range of reasons, we did not expect that all students would do all the assignments. Individuals could decide on how much they wanted to do, depending on what they wanted to get out of it. The use of a simple pull-down menu to list the links to all the items in the learning sequence made it simple for students to pick and choose between the various items. We expected that some students would start with the quizzes or perhaps the worked examples/lessons and go back to the theory refresher if they got stuck for example.

### Interest in the MOOC and early observations

We needed to be creative with the marketing of the MOOC as for this pilot we had no access or facility to email current or potential students, and no marketing budget, beyond limited quota of an internal colour photocopier. Choosing a catchy title and humorous graphics targeting stressed maths students (Figure 1) worked very well. Not only did it appeal to students directly when applied to on campus Digital Signage and flyers at enrolment (Figures 2 and 3), it also proved popular with the mainstream media.

In the month prior to the MOOC start-date we utilised a range of external and then internal marketing strategies including:

- Media Releases aimed at parents of school leavers and new university students– gained attention on syndicated Radio, local TV News and large regional newspaper
- Direct email – we are Foundation Partners in the Open Education Resource University (OERu) who are pushing towards improving access globally for education – we e-mailed colleagues in this international network who spread the word to their students
- All staff email – “Do your students need maths skills to succeed in your course?” meant that some staff referred relevant students to the MOOC
- Digital signage – targeting current students needing to refresh maths skills ahead of careers with maths dependencies
- Flyers at Science, Medicine and Health enrolment days – including conversations with Faculty advisors who validated the need for the extra maths support

The title “The Reluctant Mathematician” worked well and coupled with a humorous website graphic used in all the promotions i.e. having a strong brand was key to gaining attention and interest.

We were expecting around 50 students might enrol in the fully online maths refresher course, however we were pleased to find around 200 students were enrolled at the start of the MOOC and in the following 2 weeks a further 50 students enrolled which we presume was due to positive word of mouth referrals as no additional marketing or promotional activities took place that could account for an additional burst of interest.

The online enrolment form gathered a range of demographic data.

About 85% of participants had English as their first language, with around 15% stating English was their second language. About 60% of participants were tertiary students refreshing skills ahead of study, with the remaining 40% made up of community members. After a flurry of overseas enrolments from South Africa (courtesy of email promotion to partners in the Open Education Resource University) the local media promotion took off and Australia/Oceania enrolments ended up at over 80%. There was no national or international marketing budget for this first MOOC, so we did not expect a significant international cohort.

The group who studied maths more than 20 years ago made up the largest group of students (at just under 30%), with a further 15% of students in each of the groups who had studied 3–5, 5–10 and 10–20 years ago. Recent school leavers made up around 10% of the cohort.

Open text comments provided at enrolment time gave extra information as to the motivation for doing the MOOC. Many students noted their stress and/or recent failure in compulsory maths subjects required for their course, and there were a number of parents and grandparents trying to get up to speed to help their kids/grandkids. There were a number of training teachers who expressed passion for teaching and wanting to look at different ways to help children learn. And there were also those who just loved maths and some who were “testing the water as a pre-decision activity to retrain in an area I should have chosen earlier in life.” Many students expressed reluctance to engage in maths learning, and one particular comment exemplified the degree of this reluctance, stating that their inability to follow mathematical instructions “really undermines my self-worth especially as a woman”. If we weren’t 100% sure of the need and demand for such a course at the time of starting this project, by now we felt a big responsibility to support quite a diverse range of learners.

All participants were emailed an eight page Guide to the MOOC, however by the end of Week 1 nobody needed help using the website and nobody took up the online “drop-in” coaching options that were heavily promoted to the cohort via email. The only emails received from two stressed students were those unable to login. This was because they are so used to logging in to eLearning products but in fact our MOOC was open to the extent that no login was required.

A Facebook page and an email group was set up to satisfy the needs of learners who like to communicate with peers, and about 25 students ‘followed’ the updates to the Facebook page, and about eight in the email group. The email group was private to those that signed up so we cannot see what happened there – however it is referred to in the open-text feedback so it was useful to at least one person. The Facebook page conversation consisted of a couple of stressed learners communicating with a couple of keen and experienced maths educators, and the encouragement they gave and received seemed to help keep the participants on track. One student commented that they had spent a couple of hours on the practice questions and only got one third of the way in, but “I won’t give up!”

Enrolments continued to roll in, and to satisfy students’ desire for instant feedback on their practice questions, at the end of the first week we added a facility to allow for electronic submission of an indicative sample of questions. Previously we thought we would just upload the solutions at the end of the first week – however with students enrolling late and moving ahead at their own pace there would be no one time we could upload the answers that would be suitable. We wanted to provide a facility for students to be automatically emailed the link to download the worked examples/answers rather than waiting for it to be posted online at a particular time. However we wanted them to make a serious attempt at the questions before getting the answers, so we used a standard web-form and asked for the submission of an indicative sample of questions.

Over the first few weeks it emerged that students considered the “Test yourself practice questions i.e. the 50 maths questions to be ‘the assignment’. The Facebook page and email discussion centred around this task. Email and Facebook feedback indicated that once the 50 practice questions had been attempted, the students felt satisfied. This early observation played out for the following weeks of the MOOC.

### Evaluation methodology

We collected demographic data via an online form at the time of enrolment which provided us with data about the nature of the cohort such as age, location and English skills. There was an open-text box on the form inviting students to tell us about their past maths experience or reasons for doing the course. These initial comments demonstrated the diversity of students, ranging from those with a high degree of expertise in mathematics, to parents wanting to further support their children’s learning in schools, and those considering mid-life career changes and needing re-training. The initial responses also showed anxiety or stress in many of the participants. Google analytics, YouTube statistics and media use data were gathered to gauge the students’ patterns of use.

A further pool of data was sourced through qualitative feedback via online forms presented at the time of assignment submissions.

The questions we were hoping to find answers for included:

- Can students get out of it what they signed up for, i.e. does it match their intentions and meet their expectations?
- Does being able to attempt maths problems in private provide a face-saving way to learn, reducing the sense of “I’m dumb” that can be reinforced in class settings when peers succeed but you don’t?
- What percentage of students submitted the assignment (practice questions) per module? Is it somewhere around the 10% mark that we might expect based on current experiences with other MOOC providers?
- Is there a demand for fully online learning opportunities for maths?
- Assignment results: what spread of marks and average marks did the students achieve with their assignments? Did they succeed in mastering the topic?

## Results

Table 1 shows the number of unique webpage views i.e. individuals who viewed the various MOOC webpages during February 2014 when the MOOC was active. At this time we had 252 students enrolled. There is a drop-off in motivation from the high numbers of Week 1 to lesser figures of Week 4. The assignment submissions also dropped off as the weeks progressed, but not to the same degree. The view of the Assignment pages remained relatively high, indicating that at least some of the students getting the hang of the learning sequence skipped over the early video lessons and opted to test their skills with the assignment first.

Module One | Module Two | Module Three | Module Four | Total | |
---|---|---|---|---|---|

Theory Refresher | 275 | 100 | 63 | 41 | 479 |

Worked Examples | 181 | 86 | 52 | 31 | 350 |

Assignment (50 Practice Questions) | Views: 234 Submitted: 42 (17.9%) |
Views: 143 Submitted: 23 (16.1%) |
Views: 68 Submitted: 14 (20.6%) |
Views: 73 Submitted: 16 (21.9%) |
Views: 518 Submitted: 95 (18.3%) |

Video Quiz | 169 | 56 | 21 | 18 | 264 |

Create a Maths Video | Views: 105 Submitted: 0 |
Views: 47 Submitted: 0 |
Views: 15 Submitted: 0 |
Views: 13 Submitted: 0 |
Views: 186 Submitted: 0 |

The marks achieved by the MOOC students (see Table 2) are very pleasing – distinction and high-distinction figures by university standards. And this lead us to the following important research question: were the ‘stressed about maths’ students part of this high-achieving cohort of assignment submitters? Or did we just manage to provide a fun learning experience for those with already high skills and confidence in maths?

% students getting 60% or higher in the assignment | % students getting 70% or higher in the assignment | Median Assignment Mark | |
---|---|---|---|

Module One | 62% | 62% | 100% on 7 qs |

Module Two | 77% | 72% | 75% on 8 qs |

Module Three | 75% | 66% | 75% on 12 qs |

Module Four | 100% | 73% | 86.6% on 13 qs |

We decided to compare the whole cohort to those who submitted the practice questions (“The assignment cohort”) across all the demographic data. Figures 45678 show the results.

The majority of students were from Australia and Oceania, with English as their first language. Just over 10% of students had English as their second language, and a similar percentage of the English as 2^{nd} language students submitted an assignment. Only a tiny percent of students had English as their third language, and they did not submit an assignment. The sample size is too small to draw any conclusions from this. 60% of students enrolled in the MOOC to brush up on skills ahead of university or TAFE (vocational) further education, with around 20% doing it out of general interest with those concerned for employment and supporting their kids’ learning coming in at just under 5%. The general interest group was slightly up on the assignment submission rate, but not by much. Overall we can see that on these first 3 themes, there is no major difference between the whole cohort and those who submitted an assignment.

When we look at the last 2 themes however, some interesting differences are noticeable. Over 40% of the assignment submitters were from roughly the 38 plus age group, having studied maths more than 20 years ago, whereas this group represents less than 30% of the whole cohort. The very recent school leavers also submitted assignments at a higher rate. And pleasingly, 55% of students who submitted assignments were “a bit” stressed about learning maths as compared to just over 20% who were “not at all” stressed. And the “a bit stressed” learner group was also slightly over-represented in the assignment sample as compared to the overall cohort. There were some assignments submitted from those who were “quite stressed” and “extremely stressed” but at lower rates overall and as compared to the whole cohort.

The last set of data we had was the open-text comments for feedback submitted by students as they submitted their assignments. Apart from one student who didn’t like having to submit their assignment answers prior to getting the results (‘makes me feel like I’m back at school’), they were overwhelmingly positive about the learning experience, and valued different aspects as the following selection shows:

*“Videos, emails from fellow students great. Maths is a kind of meditation, being engrossed in it for hours enables me to enjoy other activities more. Your program is fabulous.”*

*“The video clips are wonderful. Enjoying using my mind mathematically. Can’t think of any improvements, it’s a superb service.”*

*“I like the video tutorials, Also, I like the fact that the working out is broken down into a very simplistic form. I am finding it easier to understand now and I can’t wait until the next module.”*

*“So far so good. Enjoying the challenge. The presentation videos are very helpful and the layout user friendly.”*

*“I have been pleasantly surprised. The videos are good and I find it better than face to face when I can pause and think it through for myself before continuing.”*

*“….learning materials enable any set pace through the module…”.*

## Discussion and plans for the future

From the beginning of the marketing period this MOOC proved more popular than we expected. “Who would want to spend the last month of their summer holiday doing maths?” we wondered. We were really hoping to get the kind of numbers to make the effort worthwhile, i.e. more than the 10 or 20 that might attend a face-to-face series of maths skills workshops on campus. To have over 200 enrol far exceeded our expectations, and shows that there is demand for fully online learning opportunities, even within a campus-based university community where face-to-face learning experiences are the norm, particularly for undergraduates.

Getting almost an extra 52 students joining the group, making a total of 252 enrolled participants, after the course started, could only be put down to positive word of mouth feedback from students already in the MOOC. This and the lack of requests for support were also very good news for the design of the course and the website. That plus the significant number of hits on the website pages give us some indication that the course was easy to engage with and to use.

Participants submitted assignments at an overall rate of 18.3% (averaged across the four modules), significantly higher than the 10% completion rates expected of current MOOCs. The raw numbers are also impressive – a total of 98 assignments were submitted, indicating that that the program worked for a higher number of students than we would expect to be able to support in a face-to-face workshop. We can see from the comments made at time of submission that it worked to allow students to develop skills in the four maths topics for a diverse range of students, but particularly mature-age learners having a 10–20 year gap since studying maths, and those who were a bit stressed about learning maths.

The fact that just under half of our learners were “a bit” stressed about learning maths was in itself an achievement. It tells us that there is a demand for fully online learning, even for those who are not confident in learning the topic. The open-text comments noted that the fully online program allowed students to move through at whatever pace suited them, and that the video resources which allowed for stop-points were “better than face to face” in that you could really think things through in your own time before answering. These are promising findings not just for MOOCs but potentially for all online learning designs.

However we are not clear as to whether the more stressed students found these aspects as helpful. The data set is too small and the issue needs additional research.

We have a small budget to further develop this MOOC and the challenge is to decide what direction to take. Based on the feedback from the numerous staff who contacted us when we began to promote the MOOC we feel that there is potential to expand the number of topics, and customise for different disciplines adding more contextualised information about why the selected maths skills to be refreshed and practiced are important for that particular profession. Our next steps will be to hold some further discussions with these members of staff representing the various professions that depend on maths, to find out where the priorities lie in extending the MOOC.

## Conclusion

Our evaluation showed that task and resource-based MOOCs can be appropriate designs for learning outcomes in foundational skills based courses, and that fully online learning opportunities can work to engage the non-confident and non-expert learner and help them work towards skills mastery.

The Reluctant Mathematician MOOC took as its starting point on the one hand, some issues and challenges that were particular to maths learners and on the other, many years of experience in developing audio-visual resources and bridging programs to support those struggling with maths at university. We designed for the non-expert and/or stressed maths learner/reviser and provided a highly scaffolded and structured learning space that was well received. However, the design and resources were also flexible enough to allow experts to dip in and choose the more challenging material.

This provided an engaging learning experience for a wide variety of university and non-university learners from recent school leavers through to those who had studied maths 10–20 years ago. The fully online format including the new video-quiz resources allowed students to move through the program at their own pace and in their own time, and for some this can be more effective than face-to-face study where you can stop and really think and try out a skill before the lesson proceeds.

Those who submitted the assignment generally did well and those who submitted their work were typically ‘a little stressed’ about learning maths indicating that fully online self-paced learning can work well for these students. The data is inconclusive about what works or does not work with the MOOC for those who are highly stressed about learning maths, and this is an area for future study.

Lastly, we feel it is timely to move away from binary definitions of MOOCs to a more nuanced conversation as to whether a MOOC is fit for purpose, whether the learning objectives can be met by the intended learners, and whether they can assist with outstanding issues particular to the academic discipline.

## Competing Interests

The author declares that they have no competing interests.