Blog Education Research Open Education

Ethical dimensions of MOOCs

Just completing a research marathon that started last summer looking into open online courses and both the social and ethical considerations for learners.

The work has included:
A systematic review of literature up to 2013 and a subsequent update to June 2014.
Narrative synthesis of opinion articles in the area of study.
Deeper exploration of opinions through a series of interviews.

Some of this work has been presented at conference, including OpenEd13, Utah and OER14, Newcastle with co-author David Kernohan.

I’ve just produced a diagram to summarise the findings to date. I’ve used some of the dimensions of Khan’s 2003 eLearning Framework which preceded open courses in the form we know then today. Many of the original dimensions are still relevant, some need extending, and other new areas for ethical consideration emerge from the results of my work. I hope this gives communities, institutions and those involved with open education on the web some ideas for the types of discussions that are worthy of having, if we are truly going to contribute to open education globally in an equitable and fair way.

Ethical considerations in MOOCs
Ethical considerations in MOOCs

Creative Commons License
Ethical Dimensions of MOOCs by Vivien Rolfe is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at

Blog Education Research Open Education

MOOC literature. Where are we?

Second Life Literature













Alan just tweeted a tot up of the Second Life education publications profile that came from a paper just out. He made me wonder where we are with MOOCs?

There have been two systematic reviews of MOOC literature – the first by Liyanagunawardena et al (2013) “MOOCS a systematic study” that retrieved literature up to 2012, and the second that I presented at OpenEd “MOOC research on student experience” retrieving literature up to the autumn of 2013.

CC BY Tot up of MOOC literature from two systematic reviews.
CC BY Tot up of MOOC literature from two systematic reviews.


Search strategy ROLFE – keywords targeting “MOOCS +student experience, social responsibility”.

Search strategy LIYANAGUNAWARDENA – keywords broad “MOOC”

Total publications – for both reviews this included peer-reviewed journals, conference, authoritative reports, comment / opinion articles and case studies. For L’s review it also included magazine articles.

Journals / conf. – total numbers of articles in peer-reviewed journals or conference proceedings. At this point these numbers will include literature reviews and case studies.

You can draw you own conclusions. For me having done systematic reviews on education subjects before the low numbers of empirical studies is never much of a surprise, and certainly within the MOOCature in 2014, more studies have subsequently been published.

Of the empirical studies that I found, all were cross-sectional analyses of student opinions though either questionnaires or interview. Only one was a comparative study looking at the experiences of two groups of participants.

Bone of Contention


LIYANAGUNAWARDENA, Tharindu Rekha; ADAMS, Andrew Alexandar; WILLIAMS, Shirley Ann. MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distance Learning, [S.l.], v. 14, n. 3, p. 202-227, jun. 2013. ISSN 1492-3831. Available at: <>. Date accessed: 19 Jun. 2014.

Blog Open Education

One for my baby, and one more for the MOOC

Final weeks of Jazz MOOC
(Copy of my blog post to

This is the third blog sharing my experiences of a MOOC.

Read part 1.
Read part 2.
Read part 3.

The final 3 weeks of jazz-MOOC continued to be really interesting, providing insight into ways of improving improvisation techniques, namely using MODES, following melody and chord SEQUENCES, going for THEMES and VARIATIONS.

How did the whole thing seem to be going overall? Well alongside from my favourite quote:

“capture the moos of the piece, and develop a theme”

by now there was a nice group of people (maybe 15 or so) regularly uploading and commenting on SoundCloud. People were very nice and I wouldn’t say the comments were at all constructive or of any real benefit, but they were rather nice.

The forum continued to be comments and threads about the course rather than questioning and sharing ideas about the content, or getting help on the assignments. The forum seemed to have many more contributors that were present on SoundCloud, so I hope they felt part of a community more than I did. I still felt quite isolated, and unlike going to school or University for the first time, you don’t seem to make “buddies” online.

Peer-assignment. FAIL!

To me this is the one aspect where online assessment of the masses WILL SIMPLY NEVER WORK. The peer-assignment framework was subjective, and placed quite a high onus on “presentation” (and someone would be penalised for not having the greatest recording equipment in the world, or access to a scanner to upload music scores). You absolutely cannot review someone else’s assignment if you lack the knowledge. There is a huge assumption that the content will give us all the understanding to then go and grade each other’s work, but this really wasn’t the case.

I peer reviewed many assignments and admit that I didn’t have the experience to know if a LOCRIAN MODE always did work over a minor 7 flat 5 scale, and I certainly couldn’t evaluate whether my classmates were playing these correctly in their recordings. Maybe peer-review might work online for other more factual subjects where the answers are clear-cut, and easily obtained from the videos. But it didn’t quite cross the boundaries into the arts for me.

Also you could tell people were getting BORED. Initial assignments were getting 5 or 6 reviews, and were only getting 1 or 2 at the end. The overall grades were always marvellously generous. These grades went toward your overall assessment mark.

As with face-to-face learning, it was interesting that students got completely hung-up on the grade as being the far most important thing. Some commented they were really unhappy with their grades – that some students didn’t know what they were doing: so the peer-review wasn’t perceived as being an accurate process.

So did I pass my MOOC?

I received a final email saying goodbye and a link to a survey (the third for the entire course). I didn’t get an overall grade which suggests I failed the thing because the MOOC said:

“Those of you that score 70% or higher will receive a Statement of Accomplishment. These certificates are stored with your profile on the Coursera site and will not be automatically e-mailed out to you”.

So maybe I didn’t score 70% and I didn’t warrant a statement of accomplishment.

But I DID accomplish stuff!

And surely I’m the judge of that? Learning is not about assessment, credentials, badges, grades. Learning and the role of the traditional skilled lecturer is to inspire and motivate the individual to find out more, and that is exactly what I did. The course tutor Gary Burton was great, and I didn’t understand much of the theory, but I did get the overall idea and have tried to play differently at gigs and I think this is working.

Gary Burton, Tutor on the Jazz Improvisation MOOC.

So do MOOCs have a future?

This question is being asked almost on a daily basis at the moment. Presumably the answer is yes because of the high level of investment and marketing behind them. But ARE MOOCs IMPORTANT TO THE FUTURE OF EDUCATION?

Problem 1 – technical glitches
These were minor but clearly there were aspects that needed to be ironed out. The interface wasn’t entirely easy to navigate – I didn’t realize the quizzes were part of the assessment. So, some good old usability studies would make the MOOC more accessible to everyone. (A big factor for drop outs I recon).

Problem 2 – assessment
Collecting grades, badges, certificates from peer-review processes would be exceedingly dubious without any expert intervention. Peer-review does not work in this format. Self-marking multiple choices would be an option and this would be suitable for introductory courses and learning some basic factual information, but again, a weak option for assessing someone’s in-depth understanding or for art subjects.

Problem 3 – video content
Having a poor-broadband connection out in the countryside, there were times when I couldn’t access the content. A truly “open” resource would be downloadable and available in a range of formats. The videos should be extended to address accessibility and to reach a wider audience in terms of learning styles. These aspects are essential and a requirement in face-to-face classrooms, so why aren’t they a consideration for on-line learning?

Why not make the video content OPEN, or is their commercial potential being protected for a rainy day?

So what do I really think?

Important in the future of education? In the current state I hope not. After careful research and improvement, then maybe as introductory sessions or refreshers for some factual subjects, or some practical subjects that lend themselves to video. These aren’t proven to be pedagogically effective systems. I wouldn’t want my child’s education or any part of it replaced by a MOOC yet.

MOOCs are great for a hobby or to pursue an existing interest, but it would be a sad day when full-time learners were confined to the internet and not able to gain from the life changing opportunities and lifelong friendships that attending college and university has to offer.

MOOC providers now own databases full of millions of potential learners – customers – so are a bit of a loss leader in advance of other educational products that will surely follow. Someone somewhere will want a return on this hefty investment surely?

Read part 1 of my experience

Read part 2

Blog Open Education

Still MOOCing strong

MOOC Week 2
(Copy of my blog post on


Read part 1.
Read part 2.
Read part 3.


I suppose my inquisitive nature got the better of me, so I delved back into the Jazz MOOC for week 2. I was frustrated by the forum, peed-off by the peer-review and naffed-off by the navigation in week 1. But this is jazz and it is cool, and I’m so desperate to try and be better at playing, that I decided to persevere.

Week 1 shame!

Oh the shame of not completing any peer reviews on time and being docked 20% from my assignment. To make matters worse there was no opportunity to go forth with my best excuses. The truth is I was so demoralised by the quality of the first assignment I reviewed belonging to someone else. It was so utterly brilliant I momentarily decided to give up. And when I went back it was too late and the peer-review deadline was passed.

How about a knees-up MOOC?

The peer review is difficult and the whole assessment and personal developmental aspect of the MOOC relies strongly on it. The problem is that I don’t have the knowledge of jazz theory, so how can I review the work of others? And I think human nature as it is, even the humble comments on my own work were stupidly positive. I couldn’t have really scored 7 out of 8 for a critique that for half of it admitted I didn’t know what I was doing.

Modal mayhem!

This week was all about modes. Apparently there are 11 scales that it would be great to learn and then to pop them in on top of solos. Blimey! I don’t think my brain has that level of higher intellectual capacity to actually THINK what I’m doing when I’m playing a solo, let alone compute what would be the best mode to play over a chord progression that I wouldn’t even know what it was. But modes are beautiful and haunting and have some lovely qualities, and this week has stretched my playing and ear in some way. I might therefore be able to go from playing “all the wrong notes but not necessarily in the right order” to “some of the right notes…..” and maybe who knows one day “all of the right notes”.

Bargain of the week!

Well I got to it and when peer assessment came round got stuck into reviewing the mode recordings of 5 other MOOCees. Then for good measure I reviewed a few more. And then the bargain of the week was an email from Coursera congratulating me – “You’ve evaluated 35 submissions! Marvellous!”

Bargain! Back in the MOOC’s good books. #chuffed.

Surprise of the week!

Well again, the dust-balls blow across the forum, and although I did eventually find out how to start a thread (although I had to ask someone), I see that most of the good chat and feedback is taking place on Soundcloud where we are posting our work.

Even more surprisingly it transpires that people are looking at my other stuff on there, and clearly the MOOC users are a cultured and refined bunch since there have been 29 downloads to a recent Chas and Dave recording (Ain’t No Pleasing You).

It makes me think with such a keen global audience, that a Knees-UP MOOC must be the next thing surely? I can see lessons on spoon playing and knees-up steps already.

Blog Open Education

Losing my MOOC virginity with Jazz MOOC

Introducing Jazz MOOC!
(Copy of my blog post on

First of 3 parts.
Read part 2.
Read part 3.

Last week I joined my first Massive Online Open Course from Coursera. This is one of several mass-education platforms established in the US, with  increasing numbers of free courses on a wide range of subjects. So, after having to feed it my email address and then waiting to receive an email, I was finally invited “in”. I then had to complete a questionnaire so they could know all about me. I lied. I’m Evadne Hatstand, 23 years old and from Baltimore.

So I joined my first MOC last week. A massive online course. My first emotion was excitement – it was a course on jazz improvisation from Berklee College of Music led by vibes man Gary Burton. Sorry, MOOCs on “organic chemistry” and “circuits and electronics” just don’t do it for me. Jazz MOOC had attracted people from all around the world – mainly US, Spain, and few from the UK. Most people seemed to be pretty competent musicians (all readily providing their links to YouTube and Soundcloud back-catalogues), and most seemed to be professionals doing something interesting with their lives already.

My initial excitement of doing something so awesome then turned to bewilderment. OK this thing is laid out as a website with links to the left and forum items to the right. I know I’m not stupid, but, I just didn’t have a clue what to “do” next. I watched some introductory videos from the originator, (oh yes, the content is not “open” either and none of the stuff if open licensed or accessible in any form other than a video). They gave an example critique was music featuring Gary, Pat Metheny, Antonio Sanchez and others – the top guys in the world. I saw them at the Newport Jazz Festival last year and they are immense, fluid, complicated.

Antonio Sanchez (left) and Pat Metheny (right), Newport Jazz Festival 2012. Photo by Viv. Oh, that’s me!

The example piece was so er, fluid, complicated, that you didn’t know what section was what? How can I evaluate the “bridge” section? I wouldn’t know the “bridge” section if it came along and bit me on the butt. So I thought I’d go and hang out in the forum. There was a burgeoning thread on “what instrument do you play” which was nice. I put some comments on but had no conversations with folk. It felt very isolating. I KNOW what it felt like! When you are standing at the bar being ignored and everyone around you is getting served. YES! I was in the forum saying stuff but there was no chat. Perhaps it was the time difference. Or perhaps nobody likes me. #Paranoia.


I loved the thought of doing homework again. I was already practicing all my best student excuses. I had my 6 grandmothers’ funerals lined up and everything. There were two assignments in week one, – to critically evaluate a jazz standard plus a recording of three rounds of our own improvisation (which I recorded on Garageband and uploaded to Soundcloud). Interesting as an “open” experience there was no mention of whether I should be sharing copyrighted music publically, or whether I should choose the Creative Commons license by which to declare the Cole Porter’s “What is This Thing Called Love” to the world? I would have liked some clarification.

But I did learn my first thing. After half an hour I learnt how to right-mouse-click to download something to a Mac, and how to drag the file into Garageband. It was all easy once you knew how.

My critical evaluation was brief with my limited knowledge and vocabulary. I didn’t know what musical style it was? I didn’t know what the feel was? I’ve just improvised jazz for over 20 years, I have never labelled things before. Jazz to me is a conversation in a pub or club with some guys over a few beers. There were no books or resources recommended to help me understand the principles and theory.

I enjoyed recording my own improvisation and my experience told me you should always play the tune first and take the listener back there, so I did that.

Peer-review marking

The next round seemed really fun, and that was reviewing the work of others, and a nice peer-review framework was provided. I’m not sure I can grade them on accuracy because I haven’t been told, or don’t know, what the correct approach is yet. And already I’m entirely intimidated by my first reviewee who tells me:

“In Bar 9 there is chromatic movement of the G minor 7b5 both moving away first from the G and then towards the G, with the utilzation of triplets”.

Oh shit. My evaluation said:

“Half way through there are some scale patterns”.

(Note to self. Write “utlitisation of triplets” next time).

And educationally?

I am really enjoying jazz MOC but  pedagogically there is room for it to be so much better.

There are some basic things here that I couldn’t possibly get away with as a university lecturer. There is a huge assumption about people’s prior knowledge. I’ve been playing music for over 40 years and have Grade 8 Piano with distinction and I felt totally out of it. I could no more recognise a mixolyidan mode than I could fly to the moon. There is no stuff supporting our “transition” into the jazzness.

I feel very uneasy grading people’s work when their knowledge is so superior to my own. The peer-review process has to work because jazz MOOC is on a vast scale (there must be hundreds responding to the forum threads). But it isn’t really going to work because I don’t know what I’m reading so how can I give constructive feedback?

The instructional videos were exciting, but I needed something a bit more informative.  I’d like to have seen the music with “bridge” clearly stamped on it.

I love the technology!!

Overall the great thing for me so far is the novelty to have learning technology that actually works. The thing hasn’t crashed. There has not been a single technical glitch. I am not faced with the white screen of death that I get going onto my learning environment at work. I submitted my homework and you could nicely go back and tweak it too (when I suddenly realised how crap it was).

Yes, the navigation and journey through the course is unfamiliar, and it does assume a certain level of computer literacy from the users. When my new students unfamiliar with their learning tools and environments will have an abundance of tutorials and on-line screen-capture resources to help them. Jazz MOOC has a good information-release system behind the scenes that is nice that drip-feeds the content in a timely fashion that I could see a dozen uses for.

What next?

We’ll I’m about to start week two and I’m also laughing very loudly to myself at the current UK media-hype about MOOCs threatening to take over our higher education system. Not a sausage. Maybe motivated adults who want to learn for fun in the evenings would complete a course, but there is no way this would wash with my students for one minute!

As I say in my song….”a million online users with a tutor I’ll never see”.

This is NOT the future of a good education. Actually I’m not convinced MOOCs “are” educational yet.

First of 3 parts.
Read part 2.
Read part 3.