I know what you mean, and Steven King talks incessantly about how his characters come to life during the writing process, and end up forging their own paths through the story based on the persona that's been established. You can simply repeat the process again after this though, and combine the benefits of multiple methodologies. You would just allow the characters to move dynamically within the story as you describe, and then do a rewrite after that. Everyone writes differently, personally, I enjoy laying the foreshadowing on a bit thick, and this method is especially helpful for that kind of thing. I've already been doing it here for some time, posting videos and quotes that seem random, but 10 years from now, when the series wraps, people can watch the finale, look back here, and be shocked by the fact that it's obvious in retrospect that I knew exactly how everything would end from day one. It's just a magic trick, designed to entertain the audience.