Weekend Reads: Examining the Seattle Chamber of Commerce Poll

by Kevin Schofield

This weekend's read is the second edition of the Seattle Metropolitan Chamber of Commerce's twice-yearly "The Index" poll on voter sentiment in Seattle. It made headlines earlier this week, with the Chamber declaring that respondents had become more pessimistic about our city since the previous poll in August.

Before we dive into the results, however, it's worth considering how much we trust the Chamber to accurately portray the audience's sentiments. Polls like this are statistical samples: A small percentage of people are interviewed, and their answers (after some processing) are presented as representing the broader population. Poll results are never exactly correct; rather, they are presented as being within a "margin of error" of the truth. Generally speaking, the higher the number of people who are polled, the more accurate the results, because individuals with extreme "outlier" views have less impact on the whole result. The converse is also important: The fewer people polled, the higher the margin of error, because "outliers" have more impact.

The Chamber polled 700 registered voters for its survey and calculated that it has a margin of error of 3.9%, meaning that if 50% of survey respondents said "yes" to a given question, then somewhere between 46.1% and 53.9% of Seattle registered voters would also say "yes." As we are reading through the survey results, if we see that the difference between two answers is less than 3.9%, then there is no measurable difference between them. Also, if the change from the previous survey is less than 3.9% (in either direction), it is essentially unchanged.

But there are some important caveats to this. The first, and most important, is that the 3.9% margin of error applies only to questions asked of all 700 respondents; the margin for any subgroup will be higher — and potentially much higher if the subgroup is very small. We know from other sources that about 61% of Seattle registered voters are Democrats: almost 450,000 people. Assuming they are proportionally represented in the Chamber's poll, that would mean the Chamber surveyed about 427 registered Democrats. But Native Americans and Alaska Natives make up only 0.5% of Seattle's voters — about 3,700 people — and a proportional representation of them would only be about 3 or 4 people out of the 700 polled. One extreme outlier among the Democrats polled will get fairly diluted, but one outlier among the Native Americans will have an outsize impact on the results for that subgroup. The Chamber is smart enough not to break out results for an ethnic group that small, but it does provide breakouts for Socialists (7% of voters) and Republicans (12%), even though they would respectively only account for 49 and 84 of the survey responses.

Pollsters deal with this issue by intentionally over-polling small subgroups, and then re-weighting the results to match the true demographics of the population. The Chamber does not say whether it did indeed over-sample the smaller groups, but it does make clear that it weighted the responses to match the city's demographic profile, and to match the proportion of registered voters in each City Council district. The latter was important, because it intentionally polled 100 individuals from each of the seven Council districts, even though registered voters (or other demographics) are not equally distributed across them. Nearly all polls suffer from "response bias," meaning that people with strong feelings and opinions are more likely to take the time to respond, so a poll this small with significant weighting could be heavily influenced by response bias. Moreover, this poll was limited to registered voters, which is more representative of the population than the classic "likely voters" used in election polls, but that still leaves out a large fraction of Seattle adults who are not registered to vote — including those who are not U.S. citizens.

The takeaway: This is a relatively small poll that has been heavily rebalanced to try to match voter demographics. The combination makes it more prone to bias in the results. That doesn't mean it is biased, but we should not give much credence to small differences, and even less to measurements for small demographic groups. On the other hand, where we see large differences and consistent patterns across subgroups, that can give us more confidence in the results. But we also need to keep in mind that this poll is a political tool: It only surveyed voters, not the general population, implying that its intent is to deliver a message to elected officials on what the voters want.

The Chamber's report on its poll contains 62 pages of detailed charts; I'm going to leave most of it for you to read through on your own (with the above caveats), but it's worth mentioning a few of the top results.

Overall, 76% of those surveyed felt that Seattle is on the wrong track, an increase of 8 points since the last poll; that's double the margin of error, so it's a measurable shift. Similarly, 81% felt that the overall quality of life in Seattle is worse than four years ago, an increase of 5 points (just barely greater than the margin of error). Respondents were split 50-50 as to whether they are optimistic about the future of the region. Not surprisingly, and in a pattern we see in much of the poll's results, the responses are highly politicized. Democrats — the party in power in Seattle — are much more positive, and independents and Republicans are far more negative. That is a common outcome in political polls: The party in power thinks things are better, and the parties out of power think they are worse — and their views will flip overnight when the party in power changes.

Interestingly, a supermajority of every demographic believed that the quality of life is worse in Seattle now compared with four years ago. Demographic differences are consistently within the margin of error for nearly every category except for political affiliation, and for the age 50—64 demographic. Of note: There are several questions for which the age 50—64 group is an extreme outlier, hinting (but not proving) that the results for that subgroup (20% of the population) may not be accurate.

Two-thirds of survey respondents said they've actively considered leaving Seattle. Again, it's lower for Democrats and higher for independents and Republicans, but it was still a majority of Democrats and even Socialists. This is consistent with a national trend that has been called "The Great Sorting," in which people across the country are moving to communities that share their political beliefs.

On homelessness, the top issue for survey respondents, there was a 73-point margin between those who wanted the City to provide outreach, offer shelter, and remove encampments (86%), and those who want to stop all encampment "sweeps" entirely (13%). This is a bit misleading, because those were the only two choices, allowing no room for those with more nuanced views, yet it is a devastating blow to the "Stop the Sweeps" movement. Among Democrats, 89% opted for encampment removals; even 55% of Socialists held that view. At the same time, there was strong support for expanding regional and state partnerships on homelessness response, and for investing more in behavioral health programs.

On public safety: A majority of every demographic — and a supermajority of most — do not feel safe visiting Downtown Seattle at night. Over three-quarters of survey respondents also preferred reforming the Seattle Police Department (SPD) while hiring back officers over a "defund and decriminalize" approach; though, as with the homeless encampment question, there was no room for nuanced answers to this complicated question. In a similar vein, no group — not even Socialists or Democrats — trusts the Seattle City Council to reform SPD.

There are additional sections in the survey report on housing, taxes, the City budget, the path to prosperity, and child care.

The obvious takeaway from the survey report, and perhaps the one that the Chamber was looking for given its political bent, was a lack of support for the most progressive policies that the City Council and a coalition of activist organizations have been pushing for. It's hard to miss that the results of many of the questions are highly politicized, with independents and Republicans on a very different page than Democrats and Socialists; that is probably to be expected, given our current national politics. On the other hand, the places where there is an overwhelming result and little difference across political parties, including police reform and removing homeless encampments, send a clear message to newly seated Mayor Harrell and to the City Council.

The Index (Seattle Metropolitan Chamber of Commerce)

Featured image by JamesWM.foto/Shutterstock.com.