Bellingham Residential Survey Makes My Teeth Itch

By On
• In Bellingham,

Congratulations! Bellingham has just produced another survey of homeowners and seniors. What is wrong with this picture?

Two years ago I wrote an article here entitled, “The Bellingham Residential Survey Makes My Head Hurt.” I was just getting over that headache when I received notice that yet another residential survey has emerged (Residential Survey Report 2018) and now my teeth itch. The city is attempting, yet again, to pass off on the public a survey conducted with questionable scientific rigor and, therefore, of limited or no value. Why? 79% of the self-selected respondents were homeowners and most of them were 55 years old or older. The survey was utterly devoid of randomness necessary to provide results in which one might have a mote of confidence. If you want to know what old-fart homeowners think, dig in, because this is YOUR kind of survey! If you are a renter, a young adult, homeless, or just did not happen to hear about the survey, your opinion effectively was not sought.

Renters accounted for approximately 20% of the self-selected respondents while renters make up over half the population. Only 7% of those who took the survey were between the ages of 19 and 30. Overall, there were only 1,295 total respondents (Did I mention they self-selected?) from a population of about 88,500. According to the last census about 16% of the population was under 18 so the base number of all possible adult respondents is about 75,000. That indicates, charitably, that the response rate was <2% (self-selected - not random).

Here are the basics of the bases of the survey. Judge for yourself.

City of Bellingham Residential Survey Report, 2018 Executive Summary

The results from the 2018 City of Bellingham Residential Survey provide important insights about residents’ opinions regarding City services and community priorities. While the quality of life in Bellingham is very high, there are notable trends regarding challenges facing the community and the City. We note the following key points and overall trends in reviewing the research.

Respondent Demographics

79 percent of households responding to the survey classify themselves as homeowners, and approximately 20 percent classify themselves as renters. The US Census Bureau estimates that within Bellingham 45% of housing units are occupied by the owner, indicating that the response rate for renters has remained excessively low as in previous studies. The 2018 study took efforts to increase rental responses and was able to raise the response rate for renters by 5% from the 2016 study.

The majority (51 percent) of respondents identify as female, 43 percent identify as male, and 1 percent identify as neither male nor female. 5 percent of respondents prefer not to report their gender.

There are respondents from every neighborhood in the city of Bellingham. Some neighborhoods account for very large rates of response (Columbia, Samish, and South Hill), while others only accounted for a handful (Meridian and Irongate). However, these response rates tend to correspond with the population density in those areas. For the most part, the survey respondents are seasoned residents of the Bellingham almost 70 percent of respondents have lived here for more than 10 years.

25 percent of respondents report having children in their household under the age of 18. This could include not only parents, but also guardians of minors and other household arrangements. More than half (54 percent) of survey respondents are at least 55 years old. 7 percent of responses came from people between 18 and 30 years old. This is disproportionate to Bellingham’s overall demographics as of 2010, only 24 percent of the total population is 55 or older, and more than 26 percent fall into the 20-29 age range (US Census, 2010). Historically, the respondents of this year’s survey have higher incomes than those in previous surveys. 19 percent of respondents report annual household incomes less than $35,000, 34 percent report household incomes between $35,000and $75,000, and 47 percent report household incomes over $75,000.

Total sample: n = 1295 Homeowners: n = 1026 Renters: n = 265

Two years ago I challenged the authors at Western Washington University about the 2016 survey and received this reply from Dr. Hart Hodges, in part (the full reply can be found in my previous article):

“A relatively high percentage of respondents are homeowners and have lived in Bellingham for a number of years. I suspect James [McCafferty] highlighted those findings to make sure the reader knows the results of the survey are not necessarily accurate (meaning they do not necessarily reflect the views of the population as a whole).” [NB: bolding and link are mine]

One of the comments I received on my 2016 article panned that year’s survey in this manner:

“I saw your posting in NW Citizen in regards to the City’s Residential Survey. This is an area of specialization for my firm and I agree with many of your comments. Regarding the margin of error they cite. I can’t even figure out where that number would come from: standard number for that sample size would be + or - 2.8%. Their response rate is actually quite low and as you point out not representative of the resident population.”

So why do we keep doing this to ourselves? Predictably, as in the past, city mangers and City Council members will go to the survey and select charts and statistics to bolster their arguments for the goat-rope du jour. There will be no critical thought of the questionable bases and biases of the data, or that it is a survey stacked with respondents who are homeowners and old.

About Dick Conoboy

Citizen Journalist and Editor • Member since Jan 26, 2008

Dick Conoboy is a recovering civilian federal worker and military officer who was offered and accepted an all-expense paid, one year trip to Vietnam in 1968. He is a former Army [...]

Comments by Readers

Michael Chiavario

Mar 03, 2019

Nice work Dick.



Michael Lilliquist

Mar 20, 2019

Dick shared his blog entry directly with the city concil, and I replied to him by email. I share that reply with you as well:

“Thanks for your constructive feedback.  As you know, I agree with much of what you say about the methodological shortcomings of the survey.  However, the survey was conducted with “scientific rigor,” and that is why it is possible to adjust for the un-representativeness of the sample. As was indicated during the presentation to the city council, this can be done in a number of ways.  James McCafferty offered to perform some “cross tabulations” to look into whether or not the skewed sample affected the main results.  I have taken him up on that offer, and I am awaiting the results.
“There are other problems with the survey, as well. Most glaringly, we do not have any yardstick or basis for comparison for most of the results. Again, James McCafferty has offered to re-dress those shortcomings, in a proposal to the administration. That would require extra work and funding, however, and I have not yet had a chance to discuss the matter with the mayor.”

Michael Lilliquist

Mar 22, 2019

I received a brief email reply from Rowan Innes on the cross-tabs. He said, “Ratings of Quality of life, housing affordability, and livability of neighborhoods all skewed slightly more positive among both homeowners and the high income earners.” He indicated the results showed “very little significant differences.” On a separate question, for city budget and taxes, renters and lower income responders tended to answeer “I don’t know” more often, which is not too surprising when you think about it.


Dick Conoboy

Mar 23, 2019

But what all this points to is doing the job correctly the first time and avoiding this folderol around cross tabulating and whatnot.  Doing it right cannot be too difficult but if it is, we may just consider dropping the survey altogether. 

Is then this survey supposed to be an accurate measure by which the council makes its decisions on housing?  If so, spend the money on doing a proper, randomized survey that doesn’t need fixing after the fact.  Or will I have to write a similar article on this survey in 2020? 

This is on the city’s Survey page:

“Opinion surveying provides city officials useful data about Bellingham resident’s priorities and satisfaction with city services. Some surveys are conducted to provide City officials with scientifically valid results that can be accurately applied to whole populations. Others provide a rich source of anecdotal feedback to accompany other public involvement methods.”

Why do we need anecdotal evidence, which is essentially what this survey gives us?  What other public involvement methods does this speak to?  My allotted three minutes at the podium during a hearing?   And heaven forbid if I go over three minutes! I have a rich source of anecdotal evidence on UFOs and Sasquatch that proves absolutely nothing.  Why would anyone even place this sentence on the official website of the city?