Methodology & Tools
Where do you begin to choose the right assessment tool for a project? Let’s start with some questions:
What are you trying to measure? Why?
There are different tools that are better suited for different tasks. If you want to understand impact, for example, a survey may not be the way to go. You might need to interview participants or invite them to a group activity where they talk together about their experience. If you need to understand satisfaction, though, a survey may provide you with all of the information you need.
Equally important to what you are measuring is why you are measuring it. Are you looking for new funding for a project? Do you want to understand how well students/stakeholders are learning a new concept so that you can revise your teaching to improve learning? Do you need to know the scope of work of an office to develop a new staffing plan? Each of these reasons will be best served by using different tools.
What resources do you have available to use?
How much time do you have?
Assessment can be time-consuming, so it is important to be clear from the beginning about the amount of time you can dedicate to it. For most of us, it is not our primary work, but should be a critical component of your decision-making process. For this reason, choosing a methodology that will set you up for success from the beginning is critical.
For many, a survey has been the go-to way to collect information from a large group of people. Survey tools make it easy to put one together and send it out, and many tools also can do simple analysis for you. Just because something appears easy, and you understand the technology of it, does not make it the best or quickest tool, however. To create a strong survey that asks questions to provide you with the information you need and includes stakeholders in the design can take weeks, if not months to do. And sending out a “quick and dirty” survey can do more harm than good. The analysis can also take a long time if it is a complicated survey.
How will you be inclusive in your assessment?
What information do your stakeholders need?
There are two categories of methodology:
- Qualitative: analysis used to tell a story or demonstrate key themes. Detailed descriptions of people, events, situations, interaction, and observed behaviors.
- Quantitative: data collection that assigns numbers to objects, events, or observations according to some rule. Generally analyzed using descriptive and inferential statistics.