Andy’s Research Group B

Welcome Group B!

If you appointed yourself in GROUP B, please read these instructional materials once and try not to make additional notes. We are simply wanting to test your knowledge gained. You can download the materials to your PC or mobile device via these links.

Andy_GroupB_StaticAsthmaResource (Word document form of learning materials)

Andy_GroupB_StaticAsthmaResource (PDF form of learning materials)


In our final step, we need to test your amazing and newly acquired knowledge by this final quiz. Please try and participate as honestly as possible, otherwise this will affect the results of the research.

3) Please complete this post-test quiz

Andy’s Research Group A

Welcome Group A!

Please go to YouTube and watch this 10 minute video on asthma. It does not matter if you have ever done immunology before or not. Please do not look at the materials more than once – and try not to make notes. We are simply wanting to test your recall of the knowledge that you gain through this resource.

In our final step, we need to test your amazing and newly acquired knowledge by this final quiz. Please try and participate as honestly as possible, otherwise this will affect the results of the research.

3) Please complete this post-test quiz


Thank you so much for participating in this research. Do come back to this blog to read about the results of the research.


Andy’s Research Project

Biomedical Science Final Year Project

I’m really lucky this year to be working with Andy Nguyen who is a final year biomedical scientist and also international student from Vietnam. Andy was interested in looking at student use of multimedia resources in science education, as he’d observed how many of his fellow student use YouTube to assist with their learning. Well – here is Andy’s project and he is looking for volunteers!


Please follow these steps carefully. You will be invited to complete a short ‘pre-test’, study for 10 minutes using a resource provided, and then complete a ‘post-test’. Please complete the tests as fairly as you can, without referring to notes (i.e. an ‘unseen’ test). Do spend as much time as you wish looking at the learning materials, but for consistency, do not take notes so that we are just testing your recall.

Please complete the post-test, without referring back to the materials!


1) Please complete this pre-test quiz
(you may need to copy and paste it into a new browser window if it doesn’t open from here 🙂 )


2) Choose your group!

What we need to do now is randomise you to one of two groups, and we are relying on your honesty! We need you to look at some learning material on asthma, both of which will take you ten minutes to either watch or read through.

Go and find a coin! Toss the coin!

If you have thrown HEADS then please go to GROUP A. For GROUP A CLICK HERE.

If you have thrown TAILS then please go to GROUP B. For GROUP B CLICK HERE.


3) Please complete this post-test quiz (you will also find the link on the Group A or B pages).
(You may need to copy and paste it into a new browser window if it doesn’t open from here 🙂 )

Thank you so much for participating in this research. Do come back to this blog to read about the results of the research.

Oh shit.

Oh Shit


Have you ever had that stomach-sinking feeling and you think “oh shit”? I had it the other night when I realise Spike had found one of David’s slippers, and I failed utterly to convince David that sling-backs might be quite the thing for summer.

Well here is the feeling again; I have a paper accepted by a journal and read the first few sentences so horrified I can’t even reach the end of the abstract. A very familiar conversation with self starts all over. “Here we go again! Why did I end up being an academic. Why did my parents encourage me into science where I have to write things, and not music which I’m far better at and can just create things? There aren’t enough spoon impresarios in the world”.

Of course I was straight onto Google and found this super article “10 famous writers who hated writing” and hallelujah James Joyce:

“Writing in English is the most ingenious torture ever devised for sins committed in previous lives.” James Joyce (quoted in a letter to Fanny Guillermet, 5 September 1918)


Of course the difference with all the people in this article is that they were rather spectacular at writing, despite how they felt. They probably hated the concept of the process of writing on a large scale and then having their work under the public gaze, whereas I think I’m just really crap at it. No, I don’t think, I know it.

The point of putting pen to paper, or finger to keyboard, just fills me with pain. Words swim around on the page and I resort to reading things aloud, which given my estuarine-Essex-English is not a great solution. I went through schooling struggling with Latin, French and German grammar, and was never taught English which I always think quite staggering. The two people I admire (and envy) most in my life, one is a friend from Germany, and the other my sister-in-law who is Goan and educated in Kenya. They have the most perfect and beautiful spoken English. So perhaps our British schooling has failed more of us than I realise, apart from the notable exceptions I hang out with every day on Twitter who seem to reel off Booker Prize-quality articles in the time it takes to hide the remnants of a savaged slipper. (You all know all you hare and you are totally amazing).

Mariana and some of the DS106 people were having the conversation on Twitter last year about us sometimes hating our own work, and I think agreed that was a natural part of the artistic process that we challenge and critique ourselves to become better. I think for some this can be combined with some human emotions of insecurity and low confidence I would assume?

I’m not sure really. I’m facing a day of work writing audit documents for professional bodies. It is a Saturday and I already want to cry, and the thought of completing 12 of these hilariously named “light touch” forms, I know will end badly, although by the end will have cleaned the house, done the washing and walked the dog. Twice.

How on earth did I end up in a job were papers are the currency for success. “Are you REFable”? “What high impact journals have you published in”? CVs. Fellowships. Professional Accreditations. PUBLICATIONS. AAAARRRGGGGHHHH. I just feel so thick.

Why can’t animated GIFs count?

Guinea-pigs in a maelstrom!

The Society for Research into Higher Education (SRHE) Conference 2014

Here's Fluffy

Here’s Fluffy. DS106 Daily Create, Viv Rolfe

I was very excited to be presenting at the SRHE conference last week, if a little intrepid. As a scientist who dips her toe into the world of social science methodologies, I was wondering if my work would stand the test? Judging by the questions and the Tweets I think they enjoyed it and there were no disapproving comments on my methodologies. Phew!


But me aside, the SRHE was an inspirational experience and packed with a fantastic bunch of people. There was a real international flavour and many people were attending for the first time. Are more people now interested in the big questions surrounding Higher Education? I do hope so, because there are some bloody big questions to ask these days, and judging by the action-packed programme, these were the people certainly doing it.

What I learnt?
What concerns me?
What gives me hope?
Let’s go kick ass!

Here is what I learnt in terms of methodology

  • Much of my research may employ questionnaires or interviews, or a combination. I noted how may of these researchers gather data from a broad range of sources, – questionnaires, interviews, case studies, student data, policies. The skill comes in identifying the themes and coding the results for analysis, and to triangulate the results. The volume of work is not to be underestimated.
  • Software such as NVIVO and WEFT QDA are used for analysis and data management. I need to do something other than use Excel!
  • Multidisciplinary approaches are really the thing. Subject specialists are teaming up with social scientists to share the joy. It made me think about the demise of the HEA and subject centres that support discipline-facing research. Teaming up is one approach, but I think there is a massive gap being created in the methodology skills and support that people in HE will require, and particularly those new and starting out.
  • It is quite acceptable to talk through ‘work in progress’, and with the richness of data gathered in some of the studies I am not surprised. I am struggling myself to summarise 7 interviews in 20 minutes of presentation. But everyone is cool here.
  • Someone made the amazing point – well I thought it was amazing – that often we don’t have the language and structures to ask the right questions. In an emerging field where the experiences are not fully articulated – like in my research, student and teacher perceptions of the ethical dilemmas associated with open online learning, this can cause inertia and a lack of knowledge and progress. Therefore research is needed from a very fundamental level not just to define the problem, but to determine the language with which to even explore it.

Here is what concerns me!

  • The demise of the Higher Education Academy subject centres and loss of research method-support for the different disciplines. Yes, teaming up is fine, but how do we maintain the subject contexts, how do we translate the approaches and theories to the discipline expert, and how to support new people starting out?
  • Linked to this demise, how do we ensure that there is a foundation of scholarly research expertise across departments and institutions that can support fellowship and other professional awards where practice evaluation is a key element? Again, linked to the loss of the subject centre networks, people’s professional development is at risk from the gap in methods expertise arising.
  • Where are the small pots of funding for this research? Yes, the SRHE does offer fellowships, but the changing strategies from the likes of the HEA and Jisc mean that the vital exploratory pieces of research simply can no longer be funded. Couple that with getting yourself full economically-costed onto a bid, and in my situation, applying for £10K can result in around two thirds in FEC and university overheads. Innovation will die!!!

Here is hope!

Everyone at the conference was acknowledging the very turbulent times that we are presently working under, with serious difficulties being faced by both university staff and students alike. What was hugely encouraging was the focused and high quality research being conducted and asking some very fundamental questions, for example:

University students’ attitudes to debt: a cross-national study (paper 0084)
Neil Harrison and Steve Agnew

They explored student attitude to debt in England, New Zealand and the US. Whilst debt did make students anxious naturally, this didn’t demotivate them from achieving debt, especially when the perceived value of gaining an education was strong. Debt was also linked to life style choice which was perhaps a little more worrying. The analysis of the data was in the early stages, and the outcomes of this work will certainly be worth waiting for.

The social implications of high participation higher education systems (paper 0119)
Simon Marginson

There was much focus at the conference on the social implications and impact of higher education. This excellent paper reviewed the last three decades of expansion and how this had affected a range of dimensions of HE and society, with in and around half school leavers engaging in HE. Along with others, Simon observed the stratification of the system with the ‘elite’ institutions pulling away from the field, although in Nordic systems, there were lesser differences between institutions.

“The overarching question is: ‘What are the implications of a society with universal participation in higher education?” Again, an early analysis with papers ready soon!

But what about the conference keynotes?

I think they gave me the most hope of all in that they were senior colleagues and big thinkers in the world of HE, and the first time in a long time I’ve heard such clarity of thought and common sense around some of the major issues concerning us.

SUE CLEGG – guinea pigs in a maelstrom. Weeeeeeep.

Sue talked about our moral obligation to inspire future generations – supporting a positive experience and not allowing new students to be “guinea pigs in a maelstrom”.

She reflected on the shift from an elite HE offering to mass participation, and how in the UK the system is now differentiating once more with an ‘elite’ league harnessing the funding and many of the opportunities, with clearly the big shift in students paying for access. She questioned the notion of the ‘graduate’ and the benefits that were really on offer, with many no achieving graduate level jobs, and whether this was the result of the “massification and lowering of quality”?

Many of her comments paralleled my socio-ethical observations of MOOCS:

  • Socially advantaged groups part of a system that maintains their advantage
  • Gender inequalities exist
  • Curriculum imbalances with ‘Western’ ideas excluding the cultural knowledge of indigenous people in some areas
  • As with MOOCs, mass does not equate with fair.


Bob gave a most thought provoking talk, and as with Jurgen, brought more perspectives from senior executive and thought-leaders. Bob talked about the change in HE finances, not just in terms of student fees, but government withdrawal of capital grants and the need for universities to fundraise, something that I’ve not really come across before. Interesting – we view students as the consumers and the key stakeholders, but what about those donating? Do they have a vested interest? How do we work to maintain their involvement?

In good news, Bob recounted the increased recognition by institutional decision makers of teaching quality, and the support given to fellowships and promotional schemes to raise the profile of ‘learning and teaching’ in HE. He admitted we aren’t there yet, with a fair amount of variability in promotional routes and levels of support available in universities.

Jurgen talked about HE overload and hyper-complexity! Institutions are held together by fear, and many staff are fearful, and we don’t have to delve to far into the news to find many case examples of that. Institutions and departments them are competing for resources, and then there are national and international reputations to maintain. There is rising inequality in the system.

Jurgen pointed out that we all talk about diversity and yet we standardise everything? I thought about this long and hard last week when a dyslexic student came to me in fear of the 80 question multiple choice test that he was due to face after Christmas. This is not just a response to ‘massification’ but the present day clamour to measure, monitor and manipulate absolutely everything that can possibly be classified: student experience, teaching excellence, research capability…..the list is endless, as is the vast amount of time and resource spent analysing, maintaining and responding to this data.

As Jurgen asked:

Aren’t there bigger problems to solve?

That league tables were:

Like an academic arms race.

Bob very much echoed this sentiment:

Add the shoe sizes of VC’s into league tables! Would be just as accurate.

Let’s go kick ass!

Jurgen ended by asking how we can take action? We are all very good at researching and talking about these things, but what to do? How to start some creative disorder? How to work against the current zeitgeist? So indeed, creative disorder, but how? I suppose we can continue the research, we can blog for more immediate opinion and reactions, but we are talking about change on an organisational and national scale here.

There will be similar debates this week with the results of the REF and the increasingly publicised ‘corruption’ within the system, how chasing targets and rankings causes “pressure cooker cultures”, bullying and even worse. Yet we go on cheerfully smiling through the next visitors day!


So, research? YES.

Blog and discuss publically? YES.

Influence colleagues and institutional leaders? YES.

Become a member of the SRHE? YES!


Systematic approach to desk-top research and university projects

How to conduct effective desk-top research?

This article is for any university student about to embark on writing essays or completing dissertations and projects for the first time. I have also run workshops introducing these methods and they do seem to be overwhelmingly useful even to more experienced researchers. This article is also intended to help dissertation supervisors who may want to produce a ‘mini-systematic review’ for an undergraduate or postgraduate research project. This provides a robust methodology for the students to follow and is a much more rewarding and exacting project than a mere literature review. It will also satisfy requirements of those professional bodies who look for an element of ‘data analysis’ within the project.

So, let us embark on an interesting and hopefully informative journey about how to carry out effective desk-top research.

Airport departures

CC BY SA Viv Rolfe







Introducing the systematic review

The word “systematic” in relation to a review involves the use of precise methods to gather and assess the results of research publications that (most importantly) minimises bias within the process. The result should be a robust and reliable assimilation of evidence in order to reach a reliable conclusion. Medical systematic reviews are conducted and published through the Cochrane Library named after Archie Cochrane a Scottish doctor who established the idea of evidence-based medicine. Why do I mention systematic reviews in relation to desk-top research? Well – if you understand the premise and approaches of a systematic review and apply them to your essays, coursework and dissertations, then you will be undertaking a high quality piece of work (or suggesting a high quality assignment if you are setting the work). The steps highlighted below would also provide you with a methodology and the basis of a methods section for a dissertation.

Figure 1 illustrates the systematic approach. The details on the left hand side are the minimum approach that could be undertaken in an essay or piece of desk-top research. For more in-depth undergraduate projects, and certainly for full systematic reviews, the details on the right hand side would need to be fully understood and reported.

Systematic Approach

FIGURE 1 Overview of the systematic approach CC BY SA Viv Rolfe

Full systematic reviews can be conducted on any subject, not just medical ones. I have written  ones on education subjects – and here too, they are useful to  pool knowledge about best practice, or to evaluate new innovations in teaching for example. In education, often the methods are more relaxed as generally education papers do not meet the high quality standards of medical papers and their research designs. This is often due to not being able to randomise groups of students / learners due to the constraints of timetabling and classrooms. This isn’t the entire story though, as generally there is a feeling that much medical research and education research is simply not conducted as well as it could be.

Systematic principles – we should all use them!

A full systematic review is a serious piece of research and I like to teach the principles to my university students wherever possible because it provides them with a basis for doing high quality literature reviews, essays and dissertations. In fact I believe that anyone conducting research should know these principles. How many times do we hear that people are just using a Google Search or even Scholar, and they think it is research? The mainstay for any professional research must be the use of peer-reviewed and edited articles, and Scholar will not provide a robust enough search of these, and will also retrieve non-peer-reviewed reports and documents. Interesting as background reading certainly, but not for citation within a professional piece of work.

1) Setting the research question

The formulation of a precise research question is the starting point for any research and can be quite tricky. In medicine the PICO framework is used to define the various elements – population, intervention, comparison and outcome measure. For example I might be interested whether probiotics help people with diarrhoea.

Population – patients with diarrhoea
Intervention – probiotics
Comparison – no treatment
Outcome – alleviation of diarrhoeal symptoms

So a question might be,

In patients with diarrhoea, do probiotics compared to no treatment, alleviate symptoms?

A PICO based question is the starting point of any dissertation student of mine, although not all the categories may apply. Once the question is set, the search strategy evolves and we can start generating keywords around the question categories.

But let’s take an education example. I’m interested in free online learning in the form of massive online open courses – MOOCs and the student experience.

P = learners
C = face to face/ traditional learning
O = student experience

The question might be,

Do MOOCs enhance the experience of learners compared to traditional methods?


2) Deciding where to search?

A systematic review will aim to find ALL the articles in the world! This means not just using electronic databases, but hand searching books and journals, and contacting experts for unpublished or ongoing research. This can be quite a time intensive process. Today, the process is greatly helped by being able to save your searches within electronic databases, so once established (e.g. you might run a search at the start of your student project), you can simply run it again at the end to check for recent articles. Be pragmatic with the time you have – you might not be able to search everywhere, and the school of thought is that actually a good search of electronic databases will retrieve you the majority of articles these days, although do take care if you are particularly interested in more historic ones that may not be digitised.

So, where you decide to search will depend on what your organisation or local library has access to. Web of Knowledge and Medline are the mainstays of my research – which is both medical and educational. For my review on MOOCs I also used SCOPUS, IEEE and others. These cover both conference proceedings and workshop proceedings alongside published articles (original research, literature reviews, comments, opinions, letters etc).


3) Building up keyword lists for searching

From our PICO categories, we can start building up lists of keywords on similar themes.

P = learners, students, users
I = MOOCs, xMOOC, cMOOC, massive online open course, free course
C = face to face teaching, traditional teaching
O = student experience, learning gain, knowledge gain

The next step is building up these words further. This is where I recommend using Wikipedia. It is a great keyword generator. I will also run some searches at this point to find relevant studies and look at their keywords to add to the list. If you were carrying out a full systematic review to publish, you would spend some time building up your keywords and then testing the results to ensure you were retrieving relevant articles. This iterative process might go on for some time, although for shorter-time scale projects such as undergraduate work, this may not be desirable.

4) Getting the keywords organised using Boolean logic

In some research I recently conducted looking at massive online open courses – MOOCs – I used six online databases to search, and used Boolean notation for searching with my keyword lists. There is a nice explanation of the use of Boolean logic on Ithaca College Library website. This in its simplest form uses the words (inputed in capitals – AND, OR, NOT) to combine keywords in order to expand and cross-reference your search accordingly. The Figure 2 summarises this approach.

Boolean logic

FIGURE 2 Principles of Boolean logic to expand (OR) and cross-reference (AND) searches. CC BY SA Viv Rolfe

You can also truncate words to search for all the variants of word endings using an asterisk *

e.g. MOOC MOOCs we can search for MOOC*
e.g. Massive or massively we can search for massiv*

If searching phrases these need to be in quotations otherwise the individual words will be searched for separately and return thousands of results.

e.g. “massiv* online open cours*”


I’ve referred to the use of Boolean notation in another blog article – “Seek and ye shall find” complete with webcasts and instructions. This is following very simple principles and those expert in searching and forming Boolean instruction will be more complex than this. Here are some of the more commonly used ‘operators’ or instructions within the notation.

# means search
OR – this will link together keywords and is used to broaden a search
AND – this will cross-reference two searches (and not expand the search as you might suspect)
NOT – this will exclude terms from the search


Going back to our question whether probiotics are effective for patients with diarrhoea, we could just haphazardly search for the keywords as shown below in Figure 3. However, as shown by the numbers, you will retrieve vast numbers of records and your search will not be specifically addressing your question.

Keyword searching

FIGURE 3 How NOT to search using keywords! CC BY SA Viv Rolfe







The use of Boolean notation can be illustrated by the formation of a Venn diagram which shows the principles of combining the three separate searches using the word ‘AND’. The ‘OR’ term will enable you to expand out your searches such as for probiotics and lactobacillus. You may also search for humans and adults as a focus, and also the disease of interest. By using the ‘AND’ term you are cross-referencing the three searches to find those papers in the centre of the Venn diagram (Figure 4) – you can see a more manageable number of 1312 papers. These of course can be further limited perhaps by searching just for clinical trials.

Boolean search

FIGURE 4 Results of Boolean search









5) Running the search and being organised!

Organisation is key and many online databases can set up accounts to save your searches (Medline is great for this) or export your outputs to a reference manager. I prefer to sometimes run the search, save the results as a ‘txt’ file and input into Microsoft Excel for analysis. The analysis steps might be important in a systematic review where you have pre-set what your research question is and your criteria for including studies. You can therefore use a new Excel sheet for each step in the analysis to maintain a good record of your process.

If you are completing a full systematic review, you will wish to refine your search in an iterative manner. That is, you will look at your search results to see if they are retrieving relevant articles, and refine the keywords and Boolean strategy if necessary to produce a more precise result. This step can in my experience take far more time than you might realise. The benefit is, once the search is right, you can save it and use it to update your coursework / project or review in the future.

6) What are study inclusion and exclusion criteria? (Could be optional depending on type of project or research)

For an undergraduate project you may not wish to be so stringent to think about what types of studies you wish to include or exclude. If you are completing a literature review, you may want just to provide an overall evaluation of everything that you have found. If you are being more systematic and wish to generate data for your project, you can follow the steps undertaken by a full systematic review, and record the numbers of studies you include and exclude at each phase. The beauty of this within a project or piece of research is that you are generating legitimate research data that can be displayed in a number of established figures and formats as illustrated below in Figure 5. Here, the results of a literature search and numbers of studies that are excluded during the process are shown.

Literature search flow diagram

FIGURE 5 Literature search flow diagram CC BY SA Viv Rolfe









Some excellent details on how to report systematic review results can be found not he following website, describing the PRISMA statements – preferred reporting items for systematic reviews and meta analyses.

In my studies of ‘education’, I set the entry gate quite wide so not to restrict the numbers of studies based on their design and quality. You will need to decide your inclusion and exclusion criteria at the start when you are writing your research proposal, or planning your essay. If you did wish to consider excluding types of articles, you might for example be doing a medical review and may well wish to only include randomised controlled trials. You might be researching an area of biomedical science and wish only to include animal investigations. If you are interested in systematic reviews in education specifically, this is a subject of development and debate the present time (e.g. Bennett 2005).

When you are analysing the results of your searches you will often soon spot ‘duplicate studies’. You will almost certainly find the same study on a number of databases, so you can use the ‘sort’ function to scan your lists of authors and remove duplicates. Studies can be duplicated in more subtle ways, for example an author might publish an abstract of data in a national journal, and then present the data at international conference. These are strictly duplicate studies because they contain the same data. The duplicate will need to be removed as shown in Figure 5.

7) Data collection and analysis

If you are intending to follow a systematic approach you will need to construct a series of spreadsheets to gather and organise your results. If you are completing a full systematic review you will establish the layout of a data extraction table prior to starting the review. This would include items such as author name, date of publication, methodology, outcome measures, and a host of other details. Again, the Cochrane organisation has further details on data extraction.

Sorting your search results and applying the inclusion and exclusion criteria does take time, but ultimately it will give you the good results you are looking for. In a full-scale systematic review a number of authors would do these steps independently to ensure the process is accurate and to avoid bias introduced by personal choices and preferences. A third author can help discuss any areas of conflict or indecision. Filtering of the papers generally occurs in two phases:

Phase 1 of filtering. You can quickly filter your results often by just looking at the titles and author names to identify duplicates. You may need to review the abstracts at this point to ensure they match your inclusion criteria. Anything that is unclear will need to be checked by reviewing the full paper.

Phase 2 of selecting and filtering. If your inclusion criteria is looking for a specific methodology – e.g. randomised controlled trial, or specific subset of articles – e.g. animal studies, if you cannot glean this information from the abstract you will need the full paper to review.

So you might go through a phase of ordering full papers, and again use a reference manager of file system on your computer to organise yourself. I generally obtain the full paper for every article as I go along.

8) Qualitative versus quantitive analysis

For a full systematic review, if you have identified enough studies you can then extract data for pooling in a meta-analysis to provide quantitative data. As part of a review it is also good practice to provide a brief ‘narrative’ of the papers identified, and also to summarise your results in table form. The extent to which you do all of this will depend on the numbers of papers retrieved, and for the purposes of ‘containing’ an undergraduate project within 5000 words which is often the limit, you may need to restrict the textual explanations of the papers.

Providing the ‘narrative’ is often the part that students struggle to do within project result sections, therefore it is worth gaining a deeper understanding of the approaches and styles that can be undertaken. Popay et al in 2006 wrote a report on narrative synthesis that may be a starting point.

9) Finishing off and identifying themes and conclusions

If you have adhered to your question, keywords and inclusion / exclusion criteria, you should end up with a corpus (body of literature) directly relevant to your question. Depending on the volume of papers retrieved you may be able to look for sub-themes and organise your discussion around these. For example, searching for probiotics and diarrhoeal disease may reveal areas of research focusing on children as opposed to adults for example. The research may focus on different types of bacteria or blends of bacteria. A systematic approach is a great way of organising your research from start to finish!



Bennett, J., Fred Lubben , Sylvia Hogarth & Bob Campbell (2005). Systematic reviews of research in science education: rigour or rigidity?, International Journal of Science Education, 27:4, 387-406.

Cochrane Library (2014). About Cochrane Systematic Reviews and Protocols. Available:

Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., … & Duffy, S. (2006). Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme. Version, 1.



The Only Way is Ethics

The Only Way is Ethics


Thank you Facebook for doing what many of us have been trying to do for some time, and that is raising awareness of the important and largely un-held ethical debate regarding the use of personal information and data from the internet in research. The Facebook story bought together three important things:

  • A commercial organisation claiming that their terms and conditions cover them to do anything they wish when researching human subjects.
  • Authors who didn’t make it transparent whether their research had been reviewed at a university ethical committee, what issues emerged and how they were addressed in the study.
  • Journal publishers who seem to take leniency in the ethical nature of their papers rather than upholding high standards of research and review.

The Facebook experiment has surfaced a range of opinions with claims that it was socially irresponsible, and that it was ethically dodgy,

“participation wasn’t voluntary, it was enforced, and as a result people’s state of mind may have been manipulated, in my view, without consent” (The Reed Diaries).

Peter Reed rightly suggests that advertising companies have been manipulating us since the dawn of time, and Stephen Downes agrees that commercial companies deal and trade in consumer data to bring us the products and services we would like. Companies constantly experiment with consumer data and “ethics regulations are routinely ignored”.

All of the opinions this week miss other important points regarding the clearly unethical nature of the research conducted. And also, having worked in consumer understanding within a multinational blue-chip company, the statement that we routinely ignored ethics, as did many of my collaborators, for me, is entirely garbage.

What is research?
First, let us separate out what is ‘research’ from what is merely the handling of data. What we are then distinguishing between is research ethics and the ethical and moral values that constitute a commercial organisations degree of social and corporate responsibility. This definition is from the US National Commission for the Protection of Human Subjects of Biomedical and Behavioural Research:

 “The term ‘research’ designates an activity designed to test an hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalizable knowledge (expressed, for example, in theories, principles, and statements of relationships)”.

Researchers have an obligation to ensure:

“Persons are treated in an ethical manner not only by respecting their decisions and protecting them from harm, but also by making efforts to secure their well-being”.

How are research ethics governed?
When conducting research it is the responsibility of the investigators and ethics boards of universities or organisations (commercial or otherwise) to evaluate the risks and benefits of their work. This is an important step on many levels, to protect the researcher, to ensure the work is of the best quality, and to also provide the required documentation to ensure the participant is fully aware of the risks, hence ‘informed’.

How should these systems work? Regardless of the field of research, whether it is medical, psychological, biomedical, sociological, the responsibility of the researcher is to identify and comply with relevant ethical protocols, and in reality this usually means complying with internal protocols alongside codes of conduct from often more than one professional bodies and organisations.

When I worked in market research, we complied with the codes of practice from the UK Market Research Society. Those companies we worked with, such as those processing store card data,  also followed research codes of practice. It has amused me the sweeping statements this week about ALL commercial companies being unethical. I would even go so far to say that for commercial companies, there is much more at stake than for an academic institution. Conducting good ethical research would be directly linked to reputation, and if a company is investing half its R&D budget on a product (the rest on packaging and even more on marketing), the research underpinning it needs to be robust. I suggest that with both universities and companies conducting human research, there is a wide range of ethical standards employed, but we cannot tar everyone with the same brush.

How does all this apply to online research?

The Market Research Society defines online research as:

“Collecting information from a social networking service” and “any other collection of personal data in the online environment for the purpose of research”.

And therefore it follows:

“In accordance with the principle of voluntary informed participation, information identifying respondents (personal data) must not be collected from without their consent”, and “researchers conducting social media monitoring, text analytics or sentiment analysis should take steps to avoid collecting personal data. Researcher must ensure that any personal data is not further processed without consent”.

And ethical codes particularly govern working with children and vulnerable individuals:

“In accordance with the principle of voluntary informed participation, information identifying children must not be collected from forums, social networking sites, blogs, etc without their consent and without obtaining the consent of a parent or responsible adult”.

The flaw in the research published this week, and many other studies like it, is that the risks and benefits of the study were not fully weighed up. The methods of ‘informing’ the participants about such risks and the nature of the work was not in place, and there were no means of ensuring each participant was “truly informed?” Many people think that any information on the internet is up for grabs, and the ‘public private’ argument is a bit of a smoke screen. All internet research requires ethical consideration. Fact.

With research on Facebook, Twitter, MOOC platforms and other social networking services, the answer appears far to simple. The Terms and Conditions increasingly seem to out trump the need for any ethical discussion. This is a horrific situation, and if it persists, it gives anyone researching carte blanche to experiment with human subjects, including possibly minors or vulnerable people.

So what were the risks?
So we have established that the study was a deliberate experiment with a hypothesis that had considered the recruitment of individuals into a study. In January 2012, “participants were randomly selected based on their User ID. The experiment manipulated the extent to which people (N = 689,003) were exposed to emotional expressions in their News Feed”. (Kramer et al 2014).

The outcome was they had manipulated people’s emotional thoughts through a social network, not a new finding, but a study done on a large scale none the less. So what might the risks have been to individuals?

The UK Office of National Statistics in their ‘Social Trends 41 for Health’ analysis in 2009/2010 shows that 1 in 10 adults (11%) of England were diagnosed with depression. That is 11% of adults within that year, not the prevalence over a lifetime which is much higher. Over the preceding decade, prescriptions of antidepressants increased by 334% in England. You will find similar figures on the internet elsewhere, so assuming therefore that a fair proportion of study participants might have been depressed, what might the impact on them have been, and what might the impact on non-depressed individuals have been over the research period? Well both negative thoughts, and levels of rumination and dwelling on negative thoughts are measured outcomes in studies of depression; research suggests negativity may or may not contribute to it, and work in this field is generally in its infancy (Pasyugina 2014, Takano 2014).

Within this population there would undoubtedly have been risks. Individuals were not fully informed of the implications and that they were participating in an experiment, with none of the affordances of being free to withdraw at any time. But it doesn’t matter because they gave their consent right?

What do the Facebook terms cover?
The authors claim that signing up to a Facebook account constitutes informed consent for this research. At least they made passing mention of informed consent. In another interesting Facebook clinical study looking at using the service as an enhancement to therapy there was no mention of ethical approval within the publication that had conducted research on vulnerable participants.

So, searching through the “Full Data Use Policy” that they refer to, I couldn’t find any reference to the statement made by the authors in their paper, that creating an account constitutes informed consent. Maybe I just missed it, but this hardly makes for users of Facebook being fully or truly informed that research might be taking place?

In all of the terms I could only find ONE mention of the word ‘research’ and how Facebook uses your information:

“for internal operations, including troubleshooting, data analysis, testing, research and service improvement”.

It also states to what extent they share their data with others, and I’m assuming here that they shared their data with the researchers in order for them to analyse it and write up the study.

“While you are allowing us to use the information we receive about you, you always own all of your information. Your trust is important to us, which is why we don’t share information we receive about you with others unless we have:

received your permission; 

given you notice, such as by telling you about it in this policy; or 

removed your name and any other personally identifying information from it.”

I suspect the 600,000 odd participants were not invited to provide their permission.

Where does this leave us?
Unfortunately we cannot stop social networks and corporations setting their terms and conditions to suit their commercial needs. I’m not sure whether they can legitimately ‘out trump’ the need for any ethical approval governing research it might undertake because these are legal terms, not philosophical arguments. Facebook can simply amend their terms now to cover research, and make their mechanisms of informed consent more explicit, but academics and publishers can still commit to good quality research? Surely, those researchers who published this week had some moral and ethical conscience, or were they gilded by the prospect of funding and reputation-boosting work?

Academic publishers should surely want to uphold the standards of the articles that they produce, and to uphold the rigours of the peer-review process? Some journals do send papers back that haven’t been ethically considered, and these stances are far more rigours in science and medicine than in other subject areas.

What do the public think?
In all this, nobody knows what the public thinks. We know what opinion makers on social networks think, and we know what some researchers like myself might think. A few conversations with my friends reveals that it is just generally accepted that everyone gathers your personal data anyway. What they didn’t understand were the implications of the research that might have led to risks to the participants.

The situation is that some less ethical corporations allow their socio-ethical boundaries to gradually creep forwards, and it is only when we look back at how far they have been stretched that we get a perspective of the distance of travel. My personal view is if academic researchers and academic publishers do not start contributing to the debate and to govern the quality of research, then who will?

One friend drew parallels with the ‘frog in the pot’ experiment. Don’t worry, no frogs were harmed in the making of the video.






Ethical dimensions of MOOCs

Just completing a research marathon that started last summer looking into open online courses and both the social and ethical considerations for learners.

The work has included:
A systematic review of literature up to 2013 and a subsequent update to June 2014.
Narrative synthesis of opinion articles in the area of study.
Deeper exploration of opinions through a series of interviews.

Some of this work has been presented at conference, including OpenEd13, Utah and OER14, Newcastle with co-author David Kernohan.

I’ve just produced a diagram to summarise the findings to date. I’ve used some of the dimensions of Khan’s 2003 eLearning Framework which preceded open courses in the form we know then today. Many of the original dimensions are still relevant, some need extending, and other new areas for ethical consideration emerge from the results of my work. I hope this gives communities, institutions and those involved with open education on the web some ideas for the types of discussions that are worthy of having, if we are truly going to contribute to open education globally in an equitable and fair way.

Ethical considerations in MOOCs

Ethical considerations in MOOCs

Creative Commons License
Ethical Dimensions of MOOCs by Vivien Rolfe is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at

MOOC literature. Where are we?

Second Life Literature













Alan just tweeted a tot up of the Second Life education publications profile that came from a paper just out. He made me wonder where we are with MOOCs?

There have been two systematic reviews of MOOC literature – the first by Liyanagunawardena et al (2013) “MOOCS a systematic study” that retrieved literature up to 2012, and the second that I presented at OpenEd “MOOC research on student experience” retrieving literature up to the autumn of 2013.

CC BY Tot up of MOOC literature from two systematic reviews.

CC BY Tot up of MOOC literature from two systematic reviews.


Search strategy ROLFE – keywords targeting “MOOCS +student experience, social responsibility”.

Search strategy LIYANAGUNAWARDENA – keywords broad “MOOC”

Total publications – for both reviews this included peer-reviewed journals, conference, authoritative reports, comment / opinion articles and case studies. For L’s review it also included magazine articles.

Journals / conf. – total numbers of articles in peer-reviewed journals or conference proceedings. At this point these numbers will include literature reviews and case studies.

You can draw you own conclusions. For me having done systematic reviews on education subjects before the low numbers of empirical studies is never much of a surprise, and certainly within the MOOCature in 2014, more studies have subsequently been published.

Of the empirical studies that I found, all were cross-sectional analyses of student opinions though either questionnaires or interview. Only one was a comparative study looking at the experiences of two groups of participants.

Bone of Contention



LIYANAGUNAWARDENA, Tharindu Rekha; ADAMS, Andrew Alexandar; WILLIAMS, Shirley Ann. MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distance Learning, [S.l.], v. 14, n. 3, p. 202-227, jun. 2013. ISSN 1492-3831. Available at: <>. Date accessed: 19 Jun. 2014.