The City of Bellevue (WA) is a data informed city, and wants to hear regularly from its residents. Bellevue wants residents to know it spends wisely and focuses on results that matter to the community. Being publicly accountable, spending limited resources effectively and efficiently while providing value to stakeholders is very important to Bellevue’s elected and appointed officials and the City regularly uses research to keep in touch with residents and examine its performance.
Northwest Research Group has conducted Bellevue’s Annual Performance Measures and Biennial Budget Surveys since 2000. The surveys combine core tracking measures but are updated each year to address the changing research environment and to address current issues and objectives.
Everyone doing research today recognizes that the increasing prevalence of cell phone-only or cell phone-primary households has significantly impacted how we do population-based studies. In addition, response rates have declined. A key challenge for this research is to continue to ensure that we obtain insights from a representative sample of Bellevue’s increasingly diverse residents.
To address this challenge, NWRG transitioned Bellevue’s research from a telephone survey that relied on RDD landline sample to an Address-Based Sample and offered respondents options for responding—online, inbound calls to a toll-free number, and outbound calls to non-responders. While mail surveys have been around for decades, access to a residential address frame is relatively new. And since, virtually everyone has a residential address, we now have access to a sample frame that covers every household and thus every resident in the community. This takes care of Issue #1: COVERAGE.
To address Issue #2: REACH, we use a multi-modal approach to data collection. We initially mail an invitation to sampled households; residents are asked to complete the survey online or by calling into a toll-free number. We also attempted to match a phone number to the address. Follow-up calls were then made in an effort to reach non-responders. Standard dialing protocols were used to increase response rates. Out of a total of more than 500 completed surveys, nearly two out of five were completed online.
The results are pretty dramatic. Those responding online are significantly different than those reached by phone. In the most recent Performance Measures Survey (2015), three out of five respondents were potentially reachable only by a cell phone. While there are certainly costs for mailing, these costs are significantly lower than the costs of cell phone data collection.
|Cell Phone Only / Primary||45%||82%||60%|
|Landline Only / Primary||19%||6%||13%|
Reaching younger people has always been hard and is becoming increasingly difficult. They are often less engaged with the community they live in. While younger residents continue to be under-represented in the total sample, more than three times as many web respondents were between the ages of 18 and 34 compared to those reached by phone.
With growth, Bellevue is no longer the suburban bedroom community it once was. With this approach to sampling and data collection we represented both those living in single-family and multi-family homes.
Bellevue has also become more diverse. While the true diversity of Bellevue is not fully represented, twice as many of those responding on the web were non-white when compared to those reached by phone.
In conclusion, the outcomes of using this new approach to sampling and data collection are positive. Additional strategies such as targeted outreach in neighborhoods where response rates are lower than average and to key demographic segments are currently being used to further ensure that the sample is representative and that Bellevue is talking with and listening to all of its residents.