When Controllers Speak: The Narrative Benefit to Controller Features like Built-In Speakers

There’s been a lot of talk lately about how AI and VR will “disrupt” game development and enhance player “immersion.” Both of these terms are ones I’ve personally grown sick of hearing, as they often feel like the tech sector’s cliché key words for chasing after shiny objects.

Often the best game narrative design tools are the ones closest at hand for a player. Namely, the controller, and in particular the use of built-in features like speakers or, in the case of the Playstation controller, the touchpad. Despite being simple and often gimmicky, we underestimate how such tools subtly work on our perception of a game narrative. We’ve become so used to having controller functions, especially among aging players and designers who grew up with Rumble Packs and the like, that we forget that they can be more than just ambience generators as with action adventure titles like Horizon Zero Dawn or horror classics like Until Dawn. They can break down the distance between player and game character with more elegance and impact than virtual reality or AI.

Take The Last of Us series, for example. Throughout the games there are running themes of intimacy and what deeds can be wrought by human hands both on a macro level and an interpersonal one. In the first entry, you can collect various tactile media from the world before the spore epidemic, including ragged comic issues, hastily written letters, cannibal manifests and more. Late in the game, however, you also come across recordings on dictation devices which (at least for the remasters) are recited not just on-screen but through your DualShock 4 or DualSense controller.

Most Popular

These recordings are a simple yet elegant way to transport the player into Joel’s perspective. The world of The Last of Us (assumed to be based off of the year of its original release, 2013) is at a remove from us, even today after having experienced a pandemic of our own. It’s a speculative post-apocalyptic scenario based off unnervingly believable fungal science (though of course, there’s no way to predict if our bodies would mutate in such a dramatic fashion). Despite all this, the player can have a brief moment of insight into what being in Joel’s position would be like.

By using the controller’s speaker, the object in your hand is transformed in a sense into the recorder Joel holds. This is emphasized by the fact that there are other key moments where the controller acts as Joel’s flashlight, giving a satisfying click that is also nerve-wracking during high-stakes stealth gameplay. Your flashlight’s battery is, of course, faulty at inconvenient moments and will need a rattling shake (both in and outside of the game world) to get it working once more.

I want to clarify that when such a moment happens, I’m not claiming that this controller-cum-dictation-device is a vessel for empathy. The player either has empathy for these moments or they find the functions a novelty instead. But what it offers is a powerful moment of potential empathy or catharsis in game narrative design, as in Part II when the controller is a stand-in for Joel’s acoustic guitar. Stoic Joel opens up in one of his rare instances of vulnerability for his adoptive daughter Ellie and as he awkwardly mentions it’s been years since he’s sung or played guitar for anyone, the player fumbles with the tutorial for using the touchpad to strum. What’s more, without giving anything away, the acoustic guitar over the course of both entries becomes a powerful symbol for Joel as someone reconciling with the past and for the arc of his relationship with Ellie.

What’s strange about this moment is that despite having played through this scene recently, I cannot remember if the controller also sounded the notes, or if I believe the controller speaker was working during this moment as well. I technically didn’t even require that extra function then, as the story beat was poignant enough that I personally felt effectively like I was the one holding the guitar. And I think that’s the beauty of opting for simplicity in game narrative design. It may sound self-evident, but the bells and whistles are only going to embellish a well-crafted vignette.

In that regard, what works well for The Last of Us isn’t something universally applicable to story-driven games. What’s more, using such features can sometimes date a game in ways similar to using era-specific slang in prose can age a novel. But this is not necessarily a negative judgment.

Kenneth Shepherd commented in 2021 on how No More Heroes III was subject to this sort of retro representation, despite being a game that previously was distinctive for the way it creatively utilized the Wiimote’s speaker. In the original game from 2007, before every boss fight the player’s Wiimote would be analogous to Travis Touchdown’s flip phone. Players like me were surprised when the call from Travis’ eccentric boss Sylvia Christel came from inside our houses instead of being mediated by our TV screens. Not to mention how strange it felt when you held your Wiimote up to your ear and received a call for the first time.

Players today have lost this feature with series remasters, which is a shame. As along with the fun breaking-the-fourth wall moment it also made the player feel more involved in the madcap and chaotic action of a world full of assassins. The Wiimote speaker and motion sensors also made the player feel like they were operating the geek assassin’s light-saber-esque sword Blood Berry.

Like The Last of Us, No More Heroes understood the assignment with regard to effectively using the features of its era’s controllers in narratively significant ways. Both of these games are from a decade ago or more, but both are unapologetic about portraying a specific era even if those portrayals are very speculative or surreal. Transistor was adept at this as well and its art nouveau cyberpunk world is perhaps as divorced from our world as you can get. Any time a controller’s features have successfully been used as a tool in games, it forces us to think about the relationship between bodies, technology, and the meanings made from interacting with the audiovisuals on screen.

In recent years many have commented on how games are more akin to theater than film. The way some games continue to make use of the controller’s features is significant to this discussion because it’s about making the most of a player’s performance during gameplay. We are often playing a specific role in games where the controller is integrated into the narrative. We are also under no illusions that we are immersed in the game’s world. Yet, in my opinion, we gain a stronger sense of what being in that world might be like. By forcing the player to mime some of the action on-screen and hear some of the key sounds, music, or dialogue as if it’s in the room, we can improve our sense of speculation. While AI and VR do have potential, I wish we’d focus more on genuine connection in game narrative design and less on verisimilitude.

Phoenix Simms is an Atlantic Canadian writer and indie game narrative designer. You can find her work at Unwinnable, Videodame, Third Person, and her portfolio. Her stream-of-consciousness can be found at @phoenixsimms.