Re^2: Test Survey Results

[amarquis]

in reply to Re: Test Survey Results
in thread Test Survey Results

The spread from sample size is likely dwarfed by selection bias in this case.*

"For example, fully 66% of respondents claimed that their primary code base has a test suite but it's been my experience that the number is far smaller." is where the selection bias shows. I agree that the real number is much smaller, and what we are seeing is that people who like to talk about test suites are likely to have test suites.

* - The simple way to estimate your standard deviation from a sample is the bootstrap method: you assume the population is like your sample and just use the deviation of the sample. The deviation of a rate (means, percentages, etc.) goes as the deviation of one draw divided by root the number of draws. So, in this case, about 3.8%. Even if you had n=1000, your spread would be ~1.5% or so. As a poll taker, your greatest challenge is much more often how your select your sample than it is the sample size.