Accidental Evaluator

If you were to ask a young person what they’d like to be when they grow up, chances are they wouldn’t respond with ‘evaluator’. This maxim rings true with many in the evaluation profession, but it is precisely because of this that many evaluators experience an interesting and circuitous route into the profession itself. The Accidental Evaluator will be a blog series from IOD PARC exploring the ways in which our colleagues, longstanding and newly acquainted, experienced or recently initiated, ended up in the evaluation profession.

For the first instalment in our series, we put the kettle on and sat down in the Edinburgh office with Sheelagh O’Reilly, one of IOD PARC’s longest standing associates.

Background

A self-described ‘generalist’, Sheelagh is full of stories of navigating evaluations, questionable methodological applications, organisational idiosyncrasies, frustrations with development ineffectiveness, and ideas on how evaluations can bring value to organisations. These stories are informed by her vast experience within the development and evaluation sector. With a background in Rural Resource Management by way of ten years in a brewery and a wealth of other interesting roles, Sheelagh is now an experienced Monitoring and Evaluation Consultant focused on resilience and climate change.

She has a strong focus on gender and social inclusion and has worked with bilateral and multilateral organisations, academic institutions, private businesses, and NGOs. Her practical experience includes organisation and programme monitoring, evaluation management, quality assurance, and other aspects of assessing and improving development effectiveness. These experiences have enabled her to develop a clear set of principles that guide her work in development and evaluative practice.

Sheelagh first started working with IOD PARC after having met Julian Gayfer, IOD PARC’s Managing Director, when they were both living in Vietnam. After both moving back to the UK, Julian reached out to Sheelagh to offer her a role as a Principal Consultant, which Sheelagh duly accepted. Though she had no previous experience in delivering evaluations, Sheelagh’s broad technical background and experiences of working in Vietnam had exposed her to numerous projects that were either evaluated themselves, or taught evaluation as part of project management, which Sheelagh notes as a “seriously underrated skill in development”:

“You can’t assume that someone who is an evaluator is also a manager – you generally see large-scale projects managed by people with no experience in project management, and while management expertise is really useful, for some people it just doesn’t materialise”.

This background not only allowed her to pick up new skills in impact evaluation, but also to use her existing skillset in reviewing large datasets and pulling out the relevant information. She discovered over time that extracting these “nuggets” of information was a key element in good evaluative practice.

 

Sheelagh’s Principles of Evaluation

Upon further discussion of key elements of good evaluative practice and the principles that make an effective evaluator; Sheelagh highlights the importance of providing people with the opportunity to learn on the job, as is an evaluator’s responsibility for their own personal development as a professional:

“I think it’s important for people themselves to find the area they’re interested in and organise their own personal development on that. I think that’s true in every profession because new things do come up and things do change.”

Sheelagh additionally highlights producing deliverables of an evaluation on-time as a core principle when approaching  an evaluation. This is corroborated by her mindset that when undertaking large evaluations for powerful clients, delivering one’s work on time to the expected standard encourages confidence in one’s credibility: that a significant portion of managing an evaluation is concerned with the timely and financially considerate finalisation of deliverables, alongside the relationship with the client.

An awareness of one’s personal limitations is also key to successfully operating in the field of evaluation according to Sheelagh: she argues that knowing one’s own strengths and weaknesses is highly beneficial, specifically when forming an evaluation team. This allows an individual to examine the skills required to carry out the evaluation and ensure that the evaluation team covers them all. In Sheelagh’s opinion, ensuring that one identifies the appropriate public-facing resources internally is equally as important:

“You can still wiggle that around internally, but if you need project management skills and a more junior person is doing it, make sure it’s clear. Otherwise, people are doing lots of work and building their skills but it’s not ending up on their CV. You must be prepared to put people on the bid sometimes and be prepared to argue why.”

However, she also stresses the importance of team composition and balancing a broad knowledge base with an understanding of each team members’ limitations. The evaluation process is strengthened when managed by a diverse and therefore effective evaluation team, the creation of which Sheelagh describes as a “work of art”.

 

Looking to the Future

Sheelagh has now moved to working part-time as an independent consultant. Reflecting on her method for selecting the evaluations on which she works, Sheelagh maintains that she is “not prepared to engage in a tick-box exercise”, preferring to take on projects that she views as being of high value. Delivering evaluations for evaluation’s sake is certainly not in Sheelagh’s best interests, nor ultimately the client’s, thus Sheelagh brings her interview to a close with a solid piece of advice for the aspiring evaluator:

There must be some value that you think the work might actually be useful either at strategic or institutional level or can facilitate learning on something innovative or novel. Identifying such work will become easier with experience, but also by learning how to read between the lines of the Terms of Reference”.