The Cognitive Hierarchy
Dreaming As Imagining
The same thing is happening when you are awake and are imagining something. Take a moment and imagine you are at the post office mailing a letter. In order to imagine that, you selected things, images, and events that were likely to happen. You created a going-to-the-post-office stimulus at a midway point in the cognitive hierarchy. That stimulus then got processed down until your entire imaginary experience was specified.
Imagining Versus Dreaming
To continue then, when you imagine something, you consciously create the plot of what happens at the middle level. That stimulus then propogates down until it is specified in the low levels. On the other hand, when you are dreaming, the plot originates from the high levels. It then propogates through the middle levels and on down to the low levels. But you only consciously experience the stimuli that go through the middle levels. That is why unexpected things happen when you dream.
It's much easier to explain why dreams feel real. When you are awake, and imagining, your imagined plot has to compete with the plot of what is actually going on around you. But when you're asleep, the outside world is cut off (by complex physical mechanisms). This lets you can concentrate on the dreamed plot.
The reason you can't usually remember your dreams is that your long-term memory systems are shut down while you're dreaming. That's why, if you're woken while you're dreaming, you can only remember the last 15 minutes or so of your dream.