When it comes to media, our culture is obsessed with surprise.

I get it. The moment when Darth Vader announced he was Luke Skywalker’s father was a watershed moment in cinema for an entire generation. Very few people saw it coming. The surprise was part of what made it stick in the mind so strongly. But it’s not like “I am your father” is a weak moment on repeated viewing. Even if The Empire Strikes Back is my least favorite of the three original Star Wars movies, I recognize that it’s a very strong film start to back and works well even on repeated viewings. There’s nothing wrong with the twist at the end, I just don’t think it had to be a surprise to have its impact.

But our culture hates knowing things ahead of time. “No spoilers” wasn’t even a meaningful phrase when I was younger but now most eight year olds could tell you what it means and provide examples of things they don’t want spoiled. Perhaps most interesting, a great deal of psychological research suggest that surprise isn’t even that important to a person’s enjoyment of a story. Spoilers change a person’s enjoyment very little to none at all in surveys done on the topic.

Some of our fixation on surprise undoubtedly comes from the rise of social media and the exponential explosion in the ways we can encounter spoilers. Some of it is probably rooted in the desire to be first to do a thing, or at least feel like you’re the first. The new and novel is a necessary part of the human experience and today, when so much of our world is mapped, settled and tamed by the hand of humanity media is one of our primary was to find new things. New people, places we’ve never been and ideas we’ve never considered. So surprise in story is a valuable thing, to be sure.

But surprise alone is not enough.

There’s a movement among media critics to simply praise anything that is surprising, especially if that surprise comes through subversion of expectations. In our increasingly media savvy world, achieving surprise in stories is harder and harder. To combat this, some creators chose to deliberately play in to tropes for a time, then suddenly replace the expected conclusion of those tropes with something different – they subvert expectations. The Darth Vader scene I cited at the beginning is a good example of this.

Vader was presented as an irredeemable villain for the entirety of the first Star Wars film and most of The Empire Strikes Back. But the revelation that Vader was Luke Skywalker’s father cast him – and everything we had learned about Luke’s father – in a new light, and forced us to reevaluate what we thought about the story so far. Our expectations for the climax of the story and what would happen afterwards were completely avoided and new outcomes were now possible. That’s the subversion of expectations.

What’s important to note about this particular subversion is that it worked so effectively because it didn’t directly contradict most of what we knew – the only real point of contradiction was Obi-Wan’s statement in the first film that Vader killed Luke’s father, an understandable lie to tell the son of the Galaxy’s most brutal villain. Add in the way it fit with Vader’s behavior in the rest of The Empire Strikes Back and the revelation made a horrifying kind of sense.

The problem is, subversion for the sake of subversion rarely takes the time to set up this important ground work. Take another moment in the Star Wars franchise, in The Last Jedi when Luke Skywalker takes his father’s old lightsaber from Rey and tosses it over one shoulder in an act of casual disregard that in no way matches the attitude of Luke or any other Jedi towards lightsabers at any other point in the franchise. This is a visually funny moment and we’re not expecting it, in fact I laughed on first viewing. But the dissonance this creates is off-putting and the moment probably doesn’t hold up to repeated viewing (I’ve only watched the film once) as its entire value is in surprise. We can’t appreciate it for what it says about the characters or their parts in the saga because it doesn’t fit with anything we know about those characters up to that point or, really, that we learn about them afterwards.

Audiences love novelty but, at the same time, you can’t take away what they’ve come to know just for the sake of novelty or your story runs the very real risk of losing its audience. Media cannot be strictly formulaic but one way the craft of storytelling is much like mathematics is that both require one to show your work. Subversion is fine, but without careful thought and patient crafting to make that subversion consistent with everything else you’ll get a failing grade. Don’t just go for surprise – make sure your characters and plot hold up when the novelty is gone and you’re well on your way to a classic.