I agree with Lauren that consistency is important, but I also think it's important for an author to consider some possible negative effects of switching POVs.
The big one, for me as a reader, is that I LIKE being in a POV character's head, and switching too often keeps me from settling in and getting comfortable. I can't care about too many characters at once. I like to see the scene from one perspective, and REALLY understand it from that POV.
Another possible draw back, one that I've become more aware of since I've started writing, is that switching POVs can detract from tension and mystery. Character A doesn't know everything. There are times when it's exciting, as a reader, for me to know something that Character A doesn't know, and this could certainly be achieved by switching POVs. But there are other times when it's actually better for me to be asking the same question that Character A is asking, and having the same frustration with getting answers. If the author has set up a pattern of switching POVs, and then doesn't switch at the exact time that I most want him/her to, I'm going to be pretty frustrated. Not in a good way.
I also think that too-frequent switches can be disorienting, unless the author spends a lot of words clarifying who's POV we're in, and those extra words can break up the flow of the story.
There may be other problems - anybody?
But I wouldn't say that it's wrong to switch POV, under all circumstances. I just think that it's a technique that should be used judiciously, and consciously.
I think there may be a link to the old 'show, don't tell' rule. There are obviously times when the author should 'tell', and there are times when the author should decide to establish shifting POVs. But often, with a little craft and effort, the author can find ways to show what s/he wants known without having to switch over to the other character and telling the readers what the character's thinking. And spending the time to figure this out often, in my opinion, leads to tighter, more satisfying writing.