It is true, to a degree, that it's difficult. The central "problem", if you can call it that, is that when you become published, and start building an audience, both your audience and your publisher begin to expect you to do things in a similar vein as what you've done before because it's easier to market work to an established audience. If your first novel was a science fiction story, and suddenly you write a historical Victorian romance novel, you can see how the marketability of your book would suddenly be difficult, since it would be almost as if you're getting published again for the first time. You also wouldn't want to alienate your existing audience who see your name, and read your book, expecting the same.
That's not to say it can't be done. Once you become "established", it is possible to use your status to explore other avenues. For example, Iain Banks wrote three or four novels before he published his science fiction work Consider Phlebas under the name Iain M. Banks. Providing a clear delineation between your different genres in this fashion can be useful, because it provides clear markers to readers that "this is this", and "this is that".