Case Study - City of Bellevue

The City of Bellevue (WA) is a data informed city, and wants to hear regularly from its residents. Bellevue wants residents to know it spends wisely and focuses on results that matter to the community. Being publicly accountable, spending limited resources effectively and efficiently while providing value to stakeholders is very important to Bellevue’s elected and appointed officials and the City regularly uses research to keep in touch with residents and examine its performance.

Northwest Research Group has conducted Bellevue’s Annual Performance Measures and Biennial Budget Surveys since 2000. The surveys combine core tracking measures but are updated each year to address the changing research environment and to address current issues and objectives.

Challenge

Everyone doing research today recognizes that the increasing prevalence of cell phone-only or cell phone-primary households has significantly impacted how we do population-based studies. In addition, response rates have declined. A key challenge for this research is to continue to ensure that we obtain insights from a representative sample of Bellevue’s increasingly diverse residents.

Approach

To address this challenge, NWRG transitioned Bellevue’s research from a telephone survey that relied on RDD landline sample to an Address-Based Sample and offered respondents options for responding—online, inbound calls to a toll-free number, and outbound calls to non-responders. While mail surveys have been around for decades, access to a residential address frame is relatively new. And since, virtually everyone has a residential address, we now have access to a sample frame that covers every household and thus every resident in the community. This takes care of Issue #1: COVERAGE.

To address Issue #2: REACH, we use a multi-modal approach to data collection. We initially mail an invitation to sampled households; residents are asked to complete the survey online or by calling into a toll-free number. We also attempted to match a phone number to the address. Follow-up calls were then made in an effort to reach non-responders. Standard dialing protocols were used to increase response rates. Out of a total of more than 500 completed surveys, nearly two out of five were completed online.

Outcome

The results are pretty dramatic. Those responding online are significantly different than those reached by phone. In the most recent Performance Measures Survey (2015), three out of five respondents were potentially reachable only by a cell phone. While there are certainly costs for mailing, these costs are significantly lower than the costs of cell phone data collection.

Phone Web Total
Cell Phone Only / Primary 45% 82% 60%
Both 36% 12% 27%
Landline Only / Primary 19% 6% 13%

Reaching younger people has always been hard and is becoming increasingly difficult. They are often less engaged with the community they live in. While younger residents continue to be under-represented in the total sample, more than three times as many web respondents were between the ages of 18 and 34 compared to those reached by phone.

Phone Web Total Population
18-34 9% 32% 18% 29%
35-54 33% 46% 38% 37%
55 Plus 58% 22% 44% 34%

With growth, Bellevue is no longer the suburban bedroom community it once was. With this approach to sampling and data collection we represented both those living in single-family and multi-family homes.

Phone Web Total Population
18-34 70% 32% 54% 53%
35-54 30% 68% 45% 47%

Bellevue has also become more diverse. While the true diversity of Bellevue is not fully represented, twice as many of those responding on the web were non-white when compared to those reached by phone.

Phone Web Total Population
White 84% 67% 78% 64%
Other Race 16% 33% 22% 36%

In conclusion, the outcomes of using this new approach to sampling and data collection are positive. Additional strategies such as targeted outreach in neighborhoods where response rates are lower than average and to key demographic segments are currently being used to further ensure that the sample is representative and that Bellevue is talking with and listening to all of its residents.

Customer Comment

Northwest Research Group is a cutting edge research firm. Their work is timely, accurate, and helps our city communicate resident opinion to our leadership in an easy to understand and statistically sound way.
Richard Siegel
City of Bellevue Performance & Outreach Coordinator