4 Best Practice Tips for Working with Survey Data

Photo by Christin Hume on Unsplash
  1. The Analysis
  2. The Report

How survey formatting can make or break the analysis

Before you create a survey, make sure it is the best way to get the answers you are looking for. Some quick questions to consider:

  • Do you have a large enough group to survey so as to have a large enough sample size for aggregation?
  • Do you have enough time and scope to create, manage, and then analyze a survey?

Creating both structure and flexibility within your analysis

Once the survey closes, it’s time for the hard work to begin. How you choose to ingest the survey results depends on the situation. Some cases might have you writing the analysis based directly off of the output table, which is enabled by the csv download option available in most survey tools. Other times, you might be loading the data into an existing database. The latter scenario can be a good way to keep track of the output data and connect the survey responses to other data you have on your clients (company or member attributes being a common example), but it isn’t always an option. Either way, how you name the columns and clean the dataset can define the rest of the analysis. Choosing the right name for each column will affect how easily you can scale the analysis for later iterations. While running the survey again might not currently be in the scope of your project, it generally serves to be prepared for the scenario in case it comes along as it can save you substantial amounts of time. This is also the reason for being careful with how you clean the data. In a csv situation, it can be tempting to use a tool like Excel or Google Sheets to make edits directly to the table. While it may be easy the first time, if you need to pull the raw output again or are running the survey another time, you will need to first remember all the formatting and cleaning steps that were previously executed and then work through those steps again. Instead, writing a script in R or Python will allow you to work through all those issues once and then be able to replicate it as many times as needed (and it will provide you with a form of documentation for the cleaning phase).

Crafting the best representation of your findings

The final report and its visualizations are arguably the most important part of the entire survey analysis since they are the face of the project. This means that the choices you make about how you represent the data will have a meaningful impact on how well the findings are received. As with any design project, each detail is important to the overall package. One pertinent example of this is the wording used throughout the report. A large piece of this is to accurately represent the contents of the survey. After seeing the results, it can be tempting to reword the description of the question to better fit the talking point of that specific datapoint, but that doesn’t necessarily mean that it’s the right call. Given that the questions were chosen for a reason, changing the wording in the report can actually be quite misleading as the smallest edits can lead to a whole new spectrum of interpretations. Going back to the commuting example, if we compare the original question “How many days a week do you take a personal vehicle to work?” to the more specific question “How many days a week do you drive a car to work?” the takeaway from the question is transformed. While the audience is still seeing the results as a version of the number of days, the first question allows them to consider all the ways the respondents may have interpreted it, while changing it after the fact is actually assuming the interpretation and not providing the necessary context for your audience. A common reason for these types of edits is simply that the question wasn’t well written when it was included in the survey and so to improve the appearance of the report, someone has decided to improve the question itself. While the intentions are good, the outcome isn’t as beneficial as expected. That’s part of why being intentional about your choices throughout the whole process is so important.

To Sum It Up

Working with survey data is rife with difficulties and potential roadblocks, but the unique insights that the results can provide are worth the effort. Planning ahead and being prepared for those roadblocks will make the whole process more enjoyable and can even increase the impact of your final report. The fact is, that survey data and analysis isn’t going anywhere, so being intentional about how you handle it is likely the best way to alleviate the associated headaches. Making the right decisions about designing and how to run the survey will start you off with a strong foundation. This will set you up to build a versatile data cleaning and analysis pipeline which will allow for scaling and replication. These factors can combine to elevate both the overall content and visuals of your final report, meaning that you can create more impact simply by making the right decisions throughout the process.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Data Culture

Data Culture

We build data-driven organizations and create lasting data culture.