Self selection & biased samples
Today’s cartoon on dilbert highlights an issue with any user research, it all depends who you ask. When preparing to conduct any sort of user research you should spend time considering how you will recruit users, how your choice of selection methods may bias the sample and what criteria you should use. It is important that the sample should reflect your target audience since opinions and how users interact can differ greatly between groups. The location, time and method of sampling can greatly affect the type of users you end up with.
Some sampling bias is likely with any user research since they will only include people willing to take part in research (self selection). Incentives can be used to encourage participation, however it is important to select the correct incentive since it will make it more likely that you recruit those that find the incentive of value. A degree of sampling bias is not necessarily a problem as long steps are taken to ensure the sample is representative of the target audience and the sampling method is considered at the time of analysis.
Before implementing the findings of research it is important to evaluate them against the needs of the wider target audience and the business, whether changes will impact other parts of the site and whether the respondents are your key audience. No research should be taken at face value blindly implementing users opinions. User research is an important tool informing us of our audiences needs, opinions and behaviour but we need to look at it critically and in the context in which it was conducted.