8 effective phases of impact data management

Are you a mission-driven organization looking to build an impact data strategy? Where do you start? This article gives a practical and step-by-step process that you can follow based on your organization's stage. The purpose of this article is not to show you the finish line but to demonstrate how to take a step-by-step journey. The truth is that most organizations remain at an early stage; rather, mission-driven organizations should build a long-term iterative strategy to improve and come closer to their mission and vision continuously. 

The key benefits of taking this approach are: 

•  Improved understanding of your impact roadmap to meet the purpose of your organization
• Real knowledge about the effectiveness of your programs, products, or services
• Access to impact capital based on the most effective use of impact data
• Making your role even more meaningful 
  
  

1. Impact management starts with impact strategy

Let's say your organization is just starting with the first impact report; where do you start? Starting from the data is not the right way. The foundation of high-quality impact reporting always begins with an impact strategy. So the next question is, how do I define my strategy? Depending on your impact perspective, you can start with the theory of change, logic model, logical framework, or the Impact Management Project dimensions. And based on that, design the indicators to measure performance. 

• Create an impact strategy per project, program, country, or portfolio
• Improve the understanding of the context by defining risks and assumptions
• Build on feedback loops by relating outcomes to the activities for a more efficient impact management
• Develop a due diligence and application process aligned with the impact

2. You have some program and operational data; now what?

Let's say you are an executive director of a CDFI, healthcare, or education organization. Chances are you are already collecting some form of demographic, program, or service data. Are they useful in the impact report? The probability is high that this data is limited to activity or output data. While you can undoubtedly build impact reports out of them (and that's what an overwhelming number of organizations do), unfortunately, it will not be sufficient for an impact or outcome-oriented grant or impact evidence proposal. While there are many steps in improving impact data maturity, it is essential to streamline the process so that data aggregation is an ongoing activity and not a year-end scrambling exercise.

Start with clear documentation of all AS-IS data.  

The chances are that your first inventory will reveal that you already have some of the following data:

• Financial Results:  Grants, revenue, expenses, and other accounting results
• Operational or Activity Results: Number of students, training, etc
• Stakeholder or Beneficiary Demographics:  race, income, age, etc
• Fundraising reporting:  Donor profile, donations results

The first step here is to build a clear inventory and examine data. As soon as you have created the inventory, you are likely to see incompleteness or duplication of data. You must allocate sufficient time to cleanse data once and for all. While this data can serve as a baseline, you must prepare a strategy to avoid frequent data aggregation pain in every impact reporting cycle.   Next, do this - 

• Remove any duplicated results.
• Do not report anything that can not be verified.
• Unique count: Commonly, beneficiaries/stakeholders attend multiple activities such as training classes, events, etc. In such a situation, it is better to report unique and total counts both.

Automate your impact data aggregation for continuous feedback 

Data collection and analysis are not a one-time affair. Mission-driven organizations must build a robust data aggregation strategy to summarize and analyze the latest results after every activity automatically. Every time your field manager goes to the field and syncs your data, your system should automatically build reports, scorecards, and dashboards.

You are here.

While 95% of social sector organizations use Excel or Google Docs as data management tools along with other third-party reporting dashboards such as Google Data Studio or Tableau,  this strategy is likely to create a broken process. First, you must design a data aggregation strategy. Tools like Excel or Google docs are easier to use, but they create many data management challenges, such as:

• Data loss when the file manager leaves
• Formula update every time you bring new data 
• Tracking sources of data
• Continuously changing impact metrics and data strategy
  

To avoid year-end data scrambling, you want to be here.

Average social sector organizations have anywhere from 1 to 50+ data sources internally. Each of them may have different types of stakeholders with varying requirements of reporting, such as:

• Report to funders on a regular frequency like monthly, quarterly, semi-annual, or annual period
• Narrative-oriented report with emphasis on robust storytelling
• Individual and personalized reports to large donors
• Grantee or investment effectiveness reports
• Dashboard for internal operations and other KPIs

Management should strive to build a robust data warehousing-like application to improve high-quality impact data evidence and reporting. Most Excel or Google Docs-like systems will lose data integrity over time. Instead, a system that can provide high-quality reporting is needed.

So how do we do that?

• Build a simple data warehouse-like system that simplifies data aggregation from the most common data sources important for impact reporting.
• Ideally, that system should allow you to build formulas at the database level so that the user does not have to update them every time
• Provide simple data analytics to find answers to the most frequent output, outcome, and unique data reporting with the ability to visualize results based on reporting type.

Define an impact data roadmap.

There are many objectives of your social impact journey. You must define priorities depending on our internal management goals and funder requirements. Based on each priority, you will continue adding metrics to your impact strategy (theory of change, logic model, or Impact Management Project dimensions). While it is not necessary to do all, depending on your internal goals, you will need to develop a systematic plan to scale your program. Your plan should include the following:

• Stakeholder voice - immediate feedback
• Stakeholder voice - qualitative feedback
• Performance tracking
• Outcome tracking
• Stakeholder voice - impact management approach

3. Start with the stakeholder's voice

A stakeholder is perhaps the central focal point for assessing impact. Many programs can get much better insights by analyzing qualitative feedback from clients (beneficiaries or participants). Measuring the beneficiary's voice must go well beyond simple data collection. The essential requirement is, " Can we find something that you are not looking for ?". If you are looking for something specific, you can always ask a close-ended question. 

It starts with a well-defined survey and a better engagement process

A well-designed stakeholder survey is a foundation for building an impact management journey. The key to successful stakeholder listening requires the following:

• Well-designed surveys with more precise questions and a mix of open-ended and closed-ended questions
• Various dimensions of feedback specific to products or services provided by the organization
• Cost-effective surveys with a clear collection method (door-to-door, email, SMS, call center), etc.
• A decent size of the sample to survey (typically around 5% to 15% depending on the size of the beneficiary or stakeholder pool)

The following customer impact report helps you understand the methodology, questions, and results for a publicly traded company & community development institution.

4. Integrate qualitative and quantitative analysis 

• While the first generation of the survey can be quantitative, a well-designed approach may include qualitative data—an excellent example of a qualitative approach in the interview. Specific well-designed open-ended question will offer a real insight into what stakeholder thinks. A closed-ended question often fails to capture the stakeholder sentiment as they focus on what the organization wants to learn and not what the stakeholder wants to say.
• There are many different ways to learn what the stakeholder says. A few examples are:
• A poll question followed by opened-ended and well-defined follow-up questions can be captured through social media, interviews, or survey questions.
• Analyzing emotions and tones in what people write online, like social media feeds, impact reports, or reviews. Processing qualitative text analysis through Artificial Intelligence can give you tremendous insights into what your stakeholders are saying. 
• A well-designed approach can capture real individual, stakeholder, and group sentiments. The following example gives the overall summary of critical emotions:
• Predict whether they are happy, sad, confident, and more
• Key challenges facing product and services
• What are the other issues they would like your organization to address in the future?
• You can quickly understand the causality or correlation between crucial indicators by linking qualitative responses with quantitative or closed-ended question results, as seen in the scorecard below.

Simple emotional analysis & Integrated Scorecard

While qualitative sentiment analysis is valuable in understanding beneficiary voices, the real value comes after analyzing and correlating qualitative and quantitative analysis.  The following example impact scorecard gives an in-depth understanding of how you can analyze and communicate results effectively:

5. Start with performance management

Mission-driven organizations should clearly define critical indicators measuring the program's success. These indicators may be activities and/or output-oriented but must refer to the overall progress to evaluate internal performance.  

Compare cohort results over a period of time.

Analysis can often be derived from existing results tracking systems such as the educational scores of students. The key to such analysis is to conduct the study under similar conditions, with a non-biased approach.

In this example, we can gather learnings like:

• The average score of participants grew from 4 to 6
• 35% of the cohort had a score greater than six at the end of the program
• 96% of participants have seen a positive  change in the positive average score

Compare individual stakeholder results over a period of time

Target audience: Impact Analyst, Evaluation Team

In this case, it may be necessary to look at the improvement patterns between multiple stakeholders to see if a particular approach works better for individual students. 

Compare program results with key constraints.

Target Audience: Impact Analyst, Evaluation Team

Funders may be looking to learn more about social, economic, and financial results. Before starting, funders must define a clear hypothesis of the improvement that they are seeking, such as: 

• Total monthly salary improvement over a period of time
• Overall monthly salary improvement over a period by Gender

This requires comparing multiple metrics and filtering/grouping by specific criteria such as gender, district, or education level. Funders can initiate an action or strategic plan using the impact scorecard if they do not see necessary improvements. 

6. Build a good outcome tracking strategy

 
Outcome tracking should be designed based on subjective analysis and an evaluative approach. The fundamental criteria are to review the holistic improvement or observe a specific limitation over time. This analysis not only confirms the overall breadth of improvement but also points out the deficiencies in the program approach.

7. Impact management project stakeholder

The Impact Management Project has made significant progress by bringing different impact practitioners together to agree on a standard definition of impact and the dimensions of performance that matter for impact measurement, management, and reporting across the value chain. While this approach is likely to be well-adapted in the impact investment industry, we believe it could and should be adopted by philanthropic organizations. On the other hand, businesses and ESG investments are already paying attention to this approach.

How to design an IMP-based stakeholder survey

Enterprise or mission-driven organizations might want to start by understanding their stakeholder by collecting data on their demographics. This information is often available in their internal systems, such as student enrollment, loan management systems, etc. Generally, demographic information should be collected based on appropriate data collection for each stakeholder (WHO) and not based on a selected survey.  

Enterprises need further data on which impacts are most important to the stakeholder. They also need to assess the degree of change (How Much) that has occurred for each stakeholder.

  Listen to this  in-depth webinar about the history of stakeholder-oriented impact management and the evolution of impact management: 

8. Social return on investments

Social Return on Investment (SROI), defined by Social Value International, is a pioneering approach to engage in stakeholder-based social value accountability. Social return on investment (SROI) is a principles-based method for calculating a social return valuation. This method allows evaluation of the impact on stakeholders and creates a better performance measurement for investments.

SROI accounts for the stakeholder's view of impact, similar to the Impact Management Project (IMP). However, unlike IMP, it requires financial 'proxy' values on all those impacts identified by stakeholders, which do not typically have market values. The challenge, however, is that the 'proxy' is often context-sensitive, and finding standard proxies may not be easy and may require additional costs. On the other hand, SROI gives an investor a great amount of guidance if they can define high-quality proxies considering the stakeholder's views.  

It is important to understand that SROI is one mechanism to demonstrate impact, but it should be compared and valued against other alternatives described in this article.

Conclusion

There are many ways to demonstrate the social impact of your mission.  Depending on the maturity of your organization, you can start your journey and build a step-by-step strategy to learn and demonstrate impact.  We demonstrated the following commonly used techniques that will surely put you on the path to raising impact capital from your mission.

1. Define Impact Strategy
2. Programmatic Reporting: Financial, Demographics, Operation, Donors
3. Stakeholder's voice
4. Performance Improvement
5. Outcome Improvement
6. Impact Management Project-based Stakeholder Analysis
7. Social Return On Investment (SROI)
8. Aggregating results from partners

Learn More:

Impact Measurement