This is not ignorance, just difference in usage. Many people define a vegetarian to be someone who doesn't eat meat. Milk and eggs are okay.

Many people also use the word "vegan" to mean a person who doesn't eat any animal products at all, including milk, eggs, honey, etc.

English doesn't really have a central authority when it comes to the meaning of words, so we have to rely on common usage. I'm pretty sure that the population has come down on the side of "Vegetarians can drink milk." Your particular sub-community may define the word differently.

In Chinese, the word for "vegetarian" can even mean that onions and garlic are prohibited!