Skip to main content

Methodology & Tools

Where do you begin to choose the right assessment tool for a project? Let’s start with some questions:

What are you trying to measure? Why?

There are different tools that are better suited for different tasks. If you want to understand impact, for example, a survey may not be the way to go. You might need to interview participants or invite them to a group activity where they talk together about their experience. If you need to understand satisfaction, though, a survey may provide you with all of the information you need.

Equally important to what you are measuring is why you are measuring it. Are you looking for new funding for a project? Do you want to understand how well students/stakeholders are learning a new concept so that you can revise your teaching to improve learning? Do you need to know the scope of work of an office to develop a new staffing plan? Each of these reasons will be best served by using different tools.

What resources do you have available to use?

If you have skills in interviewing and focus group facilitation, then using a survey could be challenging and frustrating to you. Selecting a tool that is available to you and that you know how to use or can learn quickly will set you up for success. Being practical about both your limitations as well as your assets will help you decide on a tool that will provide you the best information.

How much time do you have?

Assessment can be time-consuming, so it is important to be clear from the beginning about the amount of time you can dedicate to it. For most of us, it is not our primary work, but should be a critical component of your decision-making process. For this reason, choosing a methodology that will set you up for success from the beginning is critical.

For many, a survey has been the go-to way to collect information from a large group of people. Survey tools make it easy to put one together and send it out, and many tools also can do simple analysis for you. Just because something appears easy, and you understand the technology of it, does not make it the best or quickest tool, however. To create a strong survey that asks questions to provide you with the information you need and includes stakeholders in the design can take weeks, if not months to do. And sending out a “quick and dirty” survey can do more harm than good. The analysis can also take a long time if it is a complicated survey.

How will you be inclusive in your assessment?

You want to be certain that you are designing an assessment process that includes voices from all of your students/stakeholders and shifts focus away from the majority as the “norm.” Qualitative tools often provide more flexibility to incorporate the voices of all participants. When using quantitative tools, it is important to disaggregate (or separate) responses by relevant demographic groups to understand the experiences of all.

What information do your stakeholders need?

Once you are clear about why you are assessing a program, you will have a better understanding of which stakeholders need information. If the primary group you will be sharing information with is the Board of Trustees, they may prefer data charts and graphs to show change over time. If your primary audience is donors, they may want to see quotes from participants about how the experience changed them. Those are two very different types of information that may need to be collected using different tools. Having clarity before you begin about the end result will help you decide on a tool.

There are two categories of methodology:

  • Qualitative: analysis used to tell a story or demonstrate key themes. Detailed descriptions of people, events, situations, interaction, and observed behaviors.
  • Quantitativedata collection that assigns numbers to objects, events, or observations according to some rule. Generally analyzed using descriptive and inferential statistics.