Some writers are outliners. Some writing advisors are adamant that outlining is necessary.
Other writers are "discovery writers," because one of the reasons they're writing is to discover something (the story, the characters, what the author really thinks about some theme, etc.). Also known as "pantsers," because they write "by the seat of their pants." Also known as "organic" writers. In my experience, discovery writers tend not to be so adamant that it's the only way to write. But they do say, "If you aren't surprising yourself, how are you going to surprise the reader?"
A few years ago, Richard North Patterson was reading and answering questions at my local Borders. I loved the string of books he wrote in the middle of his career, so I went to see him. In response to a question from the audience, he said, "Writing without knowing where you are going and how you are going to get there is authorial malpractice." Alas, he has published two novels since then, and I guessed the "surprise" ending to each about 20 pages in.
On the other hand, I've started three novels without knowing the ending. I've written 50,000 words on each (can you guess in which month I did my writing? ;-), and I still don't have an ending for any of them. I did outline the beginnings, and found that there were plenty of ways to surprise myself as I wrote the scenes.
Stephen King is famous for being an organic writer. Some readers say that's obvious from some of King's endings. But you gotta admit that whatever he's doing is working for him.
What will work for you? Dunno. It's a good idea to try outlining at least once, to see how it works for you.
I generally outline at the scene level: One "card" per scene. (I use Scrivener for this, but you can use physical cards or any other way to capture and organize scene ideas.)
Whether you outline or not, try sketching out each scene before you write it (perhaps immediately before you write it). Before I write a scene, I usually know:
- the viewpoint character
- what the viewpoint character wants, and why they want that—i.e. how it relates to their larger story goal
- who opposes the viewpoint character
- what that opponent wants, and why they want that
- the outcome of the conflict—usually the viewpoint character doesn't get what they want; often they end up even worse off than at the start of the scene
That's enough guidance to keep me focused. Related to your question, this sketch makes the characters' motivations clear. At the same time, I usually don't know the details of how each character will go about trying to get what they want. So the sketch leaves plenty of room for me to surprise myself as I write.