Data, data and more data: Ode to evaluation

Organizations and businesses are constantly producing data of all sorts: financial reports, staff attendance reports, marketing reports, client feedback. Sometimes the amount of data being collected can be both onerous and expensive in both time and resources. I have been surprised at the number of organizations I have assessed where copious amounts of data were rigorously collected but then went on to gather dust, never to see the light of day again. On the opposite end I have noted many non-profit organizations who thought that collecting data was a waste of time and took them away from their core work. Paper work was at a minimum.

Evaluation, in its broadest sense, is a key tool in bringing about improvements to the organization and to the services or products provided. While there is a whole academic expertise called "Evaluation" with specific research techniques, I also believe very much in down to earth, basic evaluation that can be implemented by any organization. This blog will provide a rudimentary overview on what "evaluation" might look like.

Ask the right questions, get the right data

Asking the right questions is an essential step in getting usable data to improve your organization/business. What information do we need to seek to monitor our key functions, as well as to provide us with information to launch new directions and new improvements?

Evaluation should grow out of asking these "right" questions. Because of the demands of donors, non-profits can sometimes focus nearly all of their attention on questions about outputs or impact. However, there are other types of questions that need to be asked if the organization is to be healthy and remain capable of taking care of the identified needs. It is about gathering the best information possible to inform our decisions.

Questions can be as broad as "what is the mission of our organization?" or as specific as "are we getting best value for our HR dollars in this particular project?" Questions will also change over time as organizations develop, expand or adjust to changing environments.

Keeping data gathering simple, with minimal cost

Once we have identified the answers we need, organizations can identify the methodology for gathering data. Evaluation does not need to be expensive or undertaken only by external experts. Sometimes simple data gathering exercises can produce the most effective analyses. In addition, when governance and management take responsibility themselves for analyzing evaluation data, they are more likely to be motivated to apply the learnings from that evaluation.

Certain analyses should be part of the regular fabric of any organization such as financial reports and HR appraisals. There needs to be an ongoing mechanism to access the views of the target population and the wider donor population. With social media tools, there are a wide range of possibilities to gather useful data without significant expenditure.

However, a potential difficulty faced by management is when too many questions are being addressed at once. Evaluation becomes this onerous exercise where mountains of information are being gathered but is too overwhelming to be appropriately used. Management needs to identify which pieces of information are most vital for them to move forward.


Gathering the Data: 

Mechanisms for gathering data can be expensive as well as time consuming, particularly if individualized software is developed. Our targeted questions to be answered as well as our marketing budget should inform the methods we use. Do we undertake significant one-off marketing surveys or do we gather data continuously to keep abreast of the views of our "audience"? Are there simple ways to collect data on an ongoing basis that are cost-effective. For example, analyzing the number of likes on a particular posting in facebook, or the number of times a tweet is shared or viewed may have minimal cost. This will nonetheless require someone within the organization to have responsibility for monitoring the flow of information in a timely way. Getting input from staff on evaluation tools will encourage buy in and provide some innovative ideas.

Applying learnings from the Data:
An evaluation process and the data it produces is only good if it is evaluated.  Often we build these wonderful evaluation tools with the very latest technology. However, in the end, human eyes need to understand the data and make consequent recommendations. When we make major decisions within our organization such as structural changes or changes in our service delivery, we should be referring to our data to inform our decision making. One of the obstacles faced by organizations is where they need some longer term data to inform their decision making but have started the data gathering phase too late. Deciding on what information is needed at an early stage will ensure that when analysis is made, there is adequate information rather than scrambling at the last minute to find applicable data.

If we are not using our evaluation information then we should take a hard look at it and decide a) whether this is information we actually need to be gathering or b). how can we more effectively analyze and use what we are receiving.

Evaluations are an essential part of any good non-profit or for-profit organization. Those that undertake internal evaluations as a part of the fabric of the organization will be more stable at the same time as more willing and capable of undertaking change. Staff that are included in the ongoing internal evaluations will be more responsive to implementing recommendations identified. If you are unsure how to identify the key questions, how to plan out the methodology for gathering data or how to analyze the information in meaningful ways, seek an external expert to assist with the setting up of the evaluation but do not lose sight of the ownership of the evaluation processes. They belong squarely in the hands of the governance and management.