Revising the Research Companion

Where Cersei Lannister helps us through the process of peer review.

Backplot
The Research Companion went on sale a decade ago. It’s a text I wrote for anyone doing research (at any level) in the social and health sciences.

Over years of undergraduate and postgraduate study, plus research and teaching jobs within psychology and healthcare, I’d certainly read enough methods books. And been on plenty of courses.

Most of which I’d felt let down by as they would tell me what to do, but never how to do it. Methods teaching seemed full of people who didn’t really want to be there but had ended up covering the punishment course nobody else wanted to teach. And in the case of some academics in psychology, were taught by people whose research had never gone beyond studying their students. Or within healthcare by senior clinicians who had always had juniors do the dirty work of actual data collection and analysis. In short the teaching and resources were often obscure, outdated, unimaginative, and impractical. Delivered unenthusiastically by people with little or no hands-on experience of running real-world research.

Making it better
Part of my academic research has been on self-help and media advice giving and I apply that by my main job as an agony aunt (advice columnist). I figured I could use the practical aspects of advice giving to fill in the gaps missed by existing research methods texts and training.

I wrote a book designed to be easy to follow and practical. That acknowledged what didn’t work as well as what could, and most of all highlighted how research could be both challenging and enjoyable. I wanted to share people’s experiences, their ‘sticky knowledge’, tips and hacks to give pragmatic advice that also empowered people to approach research in a confident, agile and flexible way.

In the past 10 years the book has held up pretty well as most of the suggestions around effective, ethical and empowering research remain. However much has changed within that time around new technologies (particularly mobile phones), social media, and emerging new methods plus wider dialogues around access, rights and responsibilities within research.

Moving forward
The time has come to spruce up the existing text, revising it for a Second Edition.

However, for that to happen it requires something that makes me stressy. Peer review.

I’ve never got the hang of the process. When work goes out for review I’m initially proud of it, but then the nasty doubts creep in. What if the reviewers spot it’s not good enough, or needs completely revising? Or binning? What will happen to if others are involved in the work – will I let them down? What if the reviewers want me do things beyond my skills and abilities? What if reviewers are overly personal and mean? Will I have to make good on my fantasies of burning down their office? What if they LIKE my ideas and I actually have to do the work? On and on it goes.

And when the comments come back you have to pretend it’s all good for you even if you don’t feel good at all. Which is when I usually end up making my I’m-pretending-I’m-okay-but-really-I’m-not face.

Part of the point of the Research Companion is making transparent things we keep hidden. We may stress the importance of using/teaching research methodologies or peer review, but we don’t reflect on how they make us feel. Maybe you don’t relate to my experiences about methods or review, but if you do here’s to recognizing you’re not alone.

Another thing we tend to keep hidden is feedback. When we receive reviewer comments we may incorporate them into our work or acknowledge the help in subsequent publications. But we don’t tend to openly share with others what we’ve been told is right – or wrong – with our work.

Again, to make this process transparent and so those who’re unfamiliar with it can see what it’s like, I am sharing the comments I’ve had back from reviewers of the book. These may also be useful if you teach or use research methods in the health or social sciences or development where you may spot areas that need expanding or updating in your own practice.

Reviewer feedback
The good news is, for the most part, they liked the book. And made clear, justifiable criticisms. They haven’t just told me what needs to change, they’ve explained why. And more crucially for me as a writer, how I might want to go about it.

Before I hand over to the reviewers I’d like to thank the friends and colleagues who’ve read the book and informally told me what they think works and needs to be added or improved. And to the four reviewers (anonymised in this post) who’ve taken the time to go through the book and recommend changes.

Here are the positive things they shared…
Reviewer A “Often, when we teach students about the research process we give them a false sense of how it works. We don’t let them into the muddy, difficult and demotivating aspects of conducting research with people, because perhaps if we did they might never take up careers in research!” (They went on to add how this text gives a more realistic view of the research process).

Reviewer B “These (skills covered in the book) aren’t usually taught in classrooms but are only usually gained through experience or discussion with peers/colleagues. This book can help to prevent ‘silly’ mistakes in research. It provides useful tips to plan and run a successful research, while keeping one’s sanity in tact!”

Reviewer C “This book is a classic in that it covers the basics of research that are not often talked, or written about in one source. Books on methodologies abound, but for beginning or more experienced researchers this book is invaluable. It takes you through the essence of what is involved in the research process and helps open eyes and facilitate a more effective and enjoyable research process. It deserves to be read cover to cover by those new to research, as well as those well ensconced in the field”

Reviewer D “I think a key value of this book is that it helps researchers feel they are not alone, and that messiness is inevitable – this is really encouraging!! I appreciate the honesty of both Petra and her contributors – I’ve felt betrayed by academic writing that glosses over the challenges and makes me feel stupid for struggling! The emotional side of research – especially anxiety, and especially for young researchers working on their own projects – is seldom addressed”.

They also liked…
The practical focus, the use of case studies, stories and real life examples of things that go well (and badly) in research, and role-play exercises. More of these will be added to the book and expanded on this blog too.

Checklists, tables and reflection activities were popular, although it may be we think about how to make them look a little more exciting in the second edition. Plus Reviewer D did caution: “Rather than a checklist approach, consider offering principles (e.g. plan transport carefully, and cover eventualities such as weather, delays or getting lost), and including stories that give examples of mistakes people have made. The book would be more useful if it encouraged researchers to think for themselves, giving tools and topics to consider rather than answers”

There were, of course, things that needed improving, so I steeled myself to work through those

All reviewers rightly suggested all of the references within the text need checking and updating, as do the links to web resources. Reviewer C noted the book may lose authority if more contemporary ideas, sources and materials were not thoroughly reviewed and cited.

Far more of a focus was required on solitary/lone workers or those not inside or affiliated with a university or academic institution or an organization who’s able to support them professionally and personally in research.

The recommended resources and tools and overall tone of the text was too UK-centric. Even though the first edition was primarily aimed at a Western audience, reviewers felt it didn’t always match their situation outside of the UK and those working in the global south believed it barely catered for their needs. As Reviewer D noted: “(it’s) based on assumptions that can’t be made anywhere outside the UK, and very particular fields of research (e.g. that people have a general idea about research, that research is an acceptable occupation in context, that support structures exist to deal with any problems you flag, that cultural and socio-economic and educational norms apply, that governance and rights protection are in place…..). In a new edition, at the very least make explicit the assumptions that underpin the advice…
… many UK and European researchers (especially students of anthropology, global health and development, not to mention “African studies”) go abroad to do research, and if a book written for their home context were used as a guide, disaster would be certain… (A handbook for this groups of researchers would potentially have a good market as well – including for students trained in those countries in Western methods!!)”

More information is needed on planning and registering trials, and information on how that may differ globally, plus links to tools and guidance for the specifics of doing this across different settings and disciplines.

Reviewer B highlighted that new technologies and social media have altered aspects of doing and presenting research – so the sections describing methods and linking to resources to help understand/do them need to include citizen based research, visual and creative methods, storytelling and narrative based research, and opportunities for mapping and data sharing/networking. Reviewer C recommended more detail on using the Internet to undertake research (both practical advice and ethical implications), plus the use of apps to collect data. While Reviewer D recommended additional information on doing online searches for research papers, books, networks and other materials. Both Reviewers B and D noted the need for a far wider focus on participatory methods, particularly within a global context, and expanding the book to reflect on sensitivities that may arise within research on a local/national/international level around politics, social trends etc (see above and below for more on this).

All reviewers noted issues in engaging with, accessing, supporting and representing participants and requested more details on both how to do this and a critical reflection on the idea of ‘hard to reach’ and ‘hard to engage’ in research.

Impact is an issue within both academic research and work in the third sector. Reviewer B noted having a new section devoted to thinking about impact, engagement and evaluation would be beneficial. Reviewer C picked up on this with a request that more attention needs to be paid to the practicalities of getting research funded.

Reviewer C suggested having a section that contextualized social/health research on an historical timeline. While Reviewer D had a related suggestion of unpacking the histories and meanings of social/health research and recommendations around what was necessary to ensure the second edition had a more critical and global focus: “given the complexities of globalisation, post-colonialism, and all the rest, ethical and moral research increasingly requires a lot more thought and awareness and sensitivity and savvy than previously. Knowing the history of how research has gone wrong (the Nazis are a good example, but also think about drug trials on Africans etc), as well as the story one might inadvertently become a part of (e.g. colonialism, governmental control), is necessary to avoid naivete that leads to further damage”.

Reviewer D went on to add more insights which I’m including below (with their permission) as I believe it’s a crucial analysis of what’s missing not only in the first edition of my book, but in other research methods books (and teaching), currently.
“Research partnerships – trends e.g. north-south, partnering with consumers etc. Managing roles, expectations, assumptions, contributions
Research consultancies – e.g. M&E (monitoring and evaluation) work for NGO’s (Non government Organisations), bidding for research put out to tender e.g. by NGOs and charities, government departments etc
Dealing with funders/getting funding – politics, donor agendas, research trends (e.g. transdisciplinarity)
Communicating across disciplines and perspectives – e.g. quantitative health research vs social sciences/medical anthropology, clinicians vs researchers/academics etc
Carrying out someone else’s research – e.g. being a research assistant/junior researcher, doing a PhD/Masters within a larger project etc
Working with difference e.g. culture, age, socio-economic. Even if specific to the UK, researchers may enter households where English is not spoken, where there are specifics about dress, etc (e.g. skirts for women, covering knees and shoulders, taking off shoes when entering). There are many other differences to account for, including things like family roles (e.g. is it acceptable for a wife to be interviewed without permission from her husband). Not easy to provide a checklist.
Handling oneself professionally: negotiating different roles, especially where a clinician or manager is also doing research – consider conflicts of interest, appropriate boundaries, dangers of attaching “voluntary participation” in research which is conducted by someone who may also be your GP, psychologist or PHC nurse.
Knowing your limitations and acting within them – including personal characteristics such as young skinny woman wanting to interview girls with anorexia, white man from Holland wanting to talk to Africans about black empowerment – you might just not be the right person! Also capacity to deal with difficult issues, sensitive or vulnerable groups, etc. Don’t confuse research with achieving change, and be careful about trying to “make a difference” outside your own skill area (e.g. anthropologists in public health wanting to run awareness programs, skills training etc – this may not be your area of expertise, so either learn/get support or find some other way to “give back” – especially outside your home country and culture).
Developing skills and accessing the support you need – thinking beyond training and managers – these may not be available, and/or you may need additional support – perhaps a section on mentorship (what it is, how to negotiate it, what to look for, how to set up a supervision contract [formal/informal]), identifying and learning from role models etc. Reflective practice!!
Informed consent procedures and ethics: addressing the difference between what an ethics committee may require, and what might be appropriate and genuinely ethical in context. How to think and behave ethically and morally beyond professional guidelines, e.g. when confronted by extreme poverty, lack of understanding of what research will really entail, and acting in good faith when disseminating, even when your participants have no way of knowing what is said.
Building a research career: thinking strategically about conference attendance and participation, dissemination, publications etc. Managing the balance between needing to satisfy institutional and personal requirements (e.g. peer-reviewed publications), and “making a difference”. Understanding and negotiating mixed motives for research (own and others), and the politics of academia. If not a career researcher, how to incorporate research successfully and meaningfully into your work. Reflective practice again…
Diverse working environments: time management, working from home, doing research in one’s own time, work-life balance (as if that were actually possible…)
Approaching a new topic/problem: find out what is already being done, approach experts ESPECIALLY non-academics! E.g. if researching homelessness, talk to organisations that work with homeless people – practicalities, actual problems/challenges vs what the literature might say. How to choose a topic ethically and morally (“Interesting” is not a good reason to ask for people’s time and information).
Implementation & recommendations: If you want your research implemented, you need to build relationships with your target implementers from early on, make sure your question matches their priorities, and if possible discuss the data and interpretation with them along the way. If you make recommendations be careful about assumptions, e.g. don’t recommend “training” to address staff attitudes unless you are really sure that “awareness” is the problem!! (can be incredibly annoying). Maybe principle is about keeping research close to practice, relevant etc. Consider including how to write a policy brief. I like the emphasis on dissemination as an ethical/moral requirement.
Participatory research: I am very upset with the people who write the theory that makes this look easy!!!! It is incredible messy and complicated. Running focus groups, negotiating expectations, gaining participation at all, rather than just cooperation (I’m not sure you can do truly participatory research on a question you have chosen yourself). Lots of practicalities about managing this, even if just “involving consumers”. “The community” is an especially problematic concept…

Research is unpredictable, and the skills you often need are self-awareness, thinking on your feet, identifying the support you need, and making a plan. The book still assumes there is an answer readily available, whereas this is almost never the case outside a very specific context”.

What I do like about the feedback process when it works well is even with quite strong criticisms a good reviewer will have you nodding along and agreeing with them. In the case of Reviewer D I had thought of some, but not all, of their points but they phrased them far more succinctly than I’d managed. Which in turn is going to help me focus and save me time as I work through updating each chapter.

What next?
I’ve had this feedback for the past few weeks so I’ve taken time to reflect on it. To talk about it with some of the reviewers and to map their feedback to the specific sections of the book to see what will be included in a revised text, and what may be better explored on this blog – where topics can be more easily unpacked, discussed and updated.

The biggest challenge in both revising the book and incorporating feedback is remembering this isn’t writing from scratch nor a complete overhaul. Challenges lie ahead around planning, timekeeping and staying focused. Difficult when much of the feedback (see above) invites so many additional avenues worth exploring in more depth than the aims and scope of the book will allow.

I will be sharing the draft chapters with anyone who wants to give feedback, unpacking some of the revisions here on the blog (alongside more practical posts on ‘how to do’ research) plus showing back to Reviewers A-D what I’ve done to ensure I’ve addressed their criticisms.

All the reviewers were asked to recommend books they felt I might benefit from reading before I start my revisions. Here are their suggestions that might interest you, too.

Bell, J. 2005. Doing Your Research Project: A Guide for First-time Researchers in Education, Health and Social Science. McGraw-Hill International.
Dawson C. 2009. A Practical Guide to Research Methods: A user-friendly manual for mastering research techniques and projects. How To Books.
Dunn D. The practical researcher: a student guide to conducting psychological research. Wiley-Blackwell.
Kapitan L. 2010. Introduction to Art Therapy Research. New York: Brunner-Routledge.
Lunenberg FC and Irby J. 2008. Writing a successful thesis or dissertation: tips and strategies for students in the social and behavioural sciences. Thousand Oaks, California: Corwin Press
Moore N. 2006. How to Do Research: The Practical Guide to Designing and Managing Research Projects. Facet Publications.
Piantanida M and Garman N. 2009. The Qualitative Dissertation. Corwin.
Walker D. 2014. An introduction to health services research. Sage Publications

So, here I go. No excuses. Time to start revising – and writing.

If you’ve read the Research Companion and have thoughts on how it can be bettered, please do add them below. And if you’ve not read it but have ideas about what practical advice you’d like on doing research in the social or health sciences or development please also feel free to ask away in the comments.