You could also go the other way. People who do get vaccinated might not be smarter, they might just be more weak and sickly and figure that they better get the vaccine because they are more likely to die if they get the flu. That being the case, if the vaccine did cause any problems, it would cause a problem in the people least likely to be able to deal with it. So, you might get natural selection by killing off the weak by giving them the vaccine.