The Language of Faces

How Culture, Psychology, and Social Context Shape Our Expressions

Nov 02, 2024

Facial expressions and accents are like universal cues that connect us, allowing emotions to cross cultural boundaries. But it turns out even these instinctual ways of communicating carry subtle accents influenced by culture, social background, and the context around us. Just like spoken language varies across regions, our facial expressions reflect layers of identity, cultural norms, and social expectations. As a stranger in a strange land (an American living in Australia), I am both confronted and fascinated by these identities, norms, and expectations every day. Here’s what the research suggests about how we share emotions and why that sharing is both universal and uniquely shaped by where we come from.

Universality in Emotion

In the 1970s, psychologist Paul Ekman’s work found that basic emotions—like happiness, sadness, anger, fear, surprise, and disgust—are recognized across cultures.1 One hypothesis is that this shared set of expressions evolved to help us communicate quickly and clearly, laying a foundation for empathy and shared understanding. A smile generally means happiness, and a frown signals sadness—no matter where you are. Ekman’s work suggests that some expressions are hardwired, but this universality is only part of the story. Later research suggested that cultural influences can create unique layers in how we express and interpret these emotions.

Nonverbal Dialects

While Ekman showed us the basics, researcher Hillary Elfenbein introduced nonverbal dialects.2 These subtle differences mean that we tend to interpret expressions better within our own culture—a phenomenon called the in-group advantage.3 Like regional accents in speech, these nonverbal dialects create variations in how we show our emotions. For example, while a smile might mean happiness everywhere, its intensity or the way it’s combined with other gestures might vary. These details help us identify people from our own cultural group, adding depth to expressions and giving them a local flavor.

For instance, shortly after I moved to Australia, I quickly began to realize, while out walking, that I could often tell at a distance if someone was Australian or not, just by the way they were talking. The pauses, gestures, and rhythm of their conversation had a distinct flow—a cultural accent I’d never noticed before. These nonverbal accents, much like spoken dialects, add culturally specific details that resonate within familiar cultural contexts.

How Our Brains Flex Across Cultures

Cognitive psychology suggests that our brains can adapt to recognize emotions across cultures. With enough exposure or training, people can learn to interpret expressions from other backgrounds, bridging cultural divides. This adaptability is due to brain plasticity—our brain’s ability to learn and adjust. And studies suggest that regular interaction with other cultures, or specific training in recognizing emotions, can improve our accuracy, helping us connect better in global settings.

Dynamic Facial Movements

Researchers Chen and Jack approached the idea in a new way by studying how our facial muscles move when we express emotions.4 Using data-driven modeling, they found that while core expressions are universal (as Ekman suggested), there’s more going on. Intensity, arousal, and dominance all add nuance (i.e., expressions are a mix of shared signals and subtle cultural variations). Their insights are especially valuable for fields like AI, where understanding these nuances could help build technology that better respects cultural differences in expression.

If you're curious about how AI could use this kind of nuanced understanding of facial expressions, check out "Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models." The work explores how animatronic faces can learn to mimic human expressions through self-supervised learning, capturing subtle variations in intensity and emotion—an approach that not only makes robots more relatable but also highlights the potential for culturally aware AI interactions, adapting expressions to resonate with diverse social cues and contexts. If you’re interested in the intersection of AI and human expression, it’s a fascinating read.

Social Signaling

In yet another layer, Carlos Crivelli’s work5 shows that expressions are not just emotional cues—they’re social signals that adapt to context. For instance, in Western cultures, a “fear” expression might indicate vulnerability. But in some Melanesian communities, the same expression can signal aggression or dominance. These findings remind us that expressions are adaptive behaviors shaped by culture, serving different purposes based on social norms.

Social Class and Facial Cues

Recent studies suggest that even neutral expressions can hint at social class.6 Researchers Bjornsdottir and Rule found that subtle cues—like a slight smile or relaxed expression—can give away socio-economic status. People might unconsciously interpret slight smiles or more relaxed expressions as markers of higher status, while more neutral or closed-off expressions are linked with lower social backgrounds. These impressions, much like accents in speech, influence how we perceive others in various settings, from job interviews to social gatherings.

But before you get too fired up about how we might be socializing each other into these elitist biases, check out the latest research suggesting that toddlers as young as 14 months pick up on, and preferentially engage with, wealthier individuals.

Emotional Expressivity in Diverse Societies

Studies by Niedenthal and colleagues add yet another layer by showing that countries with a history of migration and cultural diversity often have more expressive emotional cultures.7 In places like the U.S., where various languages and norms intersect, clear, recognizable expressions help bridge communication gaps. This tendency toward expressivity supports social bonding in diverse settings, helping people from different backgrounds feel more connected.

Why It Matters

Understanding these accents in facial expressions can reduce misunderstandings and foster better communication. Training in emotional intelligence and cultural awareness can help people navigate these subtleties, making it easier to connect across cultures. By recognizing nonverbal dialects, we can build more trust, break down biases, and communicate more effectively in diverse workplaces and social circles.

Technology’s Role in Decoding Expressions

As digital communication grows, technology also has a part to play in bridging these cultural differences. Video calls, social media, and AI often expose us to a range of cultural expressions. But without the right context, it’s easy to misinterpret them. For example, AI systems that recognize an East Asian expression of happiness as genuine, rather than neutral or disengaged, could support more accurate interactions. Developing culturally aware technology will be key to fostering inclusivity in an increasingly connected world.

A Blend of Universal and Unique

So, while our facial expressions are a (mostly) universal way to communicate, they’re also influenced by culture and context. Ekman’s work on core expressions gives us a biological base, but nonverbal dialects and insights from cognitive psychology remind us that culture and learning play huge roles. Human expression is complex, blending biology with layers of social and cultural meaning.

Recognizing our emotions as both universal and culturally accented might help us better navigate the social world. And by appreciating both shared cues and unique accents, we potentially deepen our connections, communicate with more empathy, and navigate diverse interactions with greater ease.