Everyone has an opinion – but my gut is telling me differently. Should I trust it?

A friend wants to quit her job. It’s rewarding work, and she gets regular salary increases. But she wants a change.

My advice has been to hold off until she has an idea of what she wants to do next. Instead she’s been turning to tarot cards, looking for clues in cups, swords, wands and coins.

Related: I’m an adult. Why do I regress under my parents’ roof?

I have a soft spot for tarot as a prompt for introspection and creativity. But its growing popularity makes me wonder if we’re looking outside ourselves for permission to do what we think or feel to be right for us.

My friend disregards any card that can’t be interpreted as “quit”. Why might a card be more convincing than advice from loved ones, or our own intuition?

We’ve probably all had experience of a hunch we strongly suspect to be true, even if we can’t put our finger on why. Scientists have defined intuition as “knowledge gained without rational thought”, often a reflection of implicit, unconscious learning.

Those occasional “gut feelings” can reflect knowledge or understanding that is well-founded but just below the level of our awareness. So why do we struggle to trust them?

I’ve often felt my intuition kick in at work, which I don’t second-guess, because it seems to stem from experience. I’ve also called off relationships after realising, sometimes suddenly, that something’s not right.’

Yet it’s all too easy to doubt ourselves, especially now. The internet is flooded with opinions on what to do and how to be, invariably expressed with the utmost assurance: dump him! Work through it! Quit your job! Don’t!

Of course, big decisions benefit from thought and research. But crowdsourcing advice online from strangers, or even from friends, may only add to our internal confusion, preventing us from knowing our own minds. Ultimately, we have to reach a decision that’s right for us.

My friend Paloma once moved countries on a near whim. “Something in me understood that that was what I needed to do, and I couldn’t rationalise it to anyone – including my parents, who pushed back on it many times,” she tells me.

It was “absolutely” the right decision, Paloma says now. “I don’t know why it made sense – I just did it, and it worked out.”

But she routinely finds that inner wisdom is drowned out by “chatter”, like friends’ advice and her anxiety.

“I can’t tell if I’m just worried about something, if I’m gauging risk or if I’m being guided to the right decision,” she says. Other times, it’s yearning: “You think you have intuition towards a person, even just a purchase, but it’s desire, or consumerism.”

In particular, Paloma struggles against the western world’s emphasis on rational decision-making and economic sense. “That framework isn’t necessarily the correct way to go about doing things – it’s just one possible way. That’s a whole thing I do battle with all the time.”

Her dad tried to get her to justify her move overseas with a budget and spreadsheets. The numbers didn’t add up; she did it anyway. “I’ve found there are things that are ultimately good for me that will never make sense ‘on paper’.”

***

The most helpful tool I’ve discovered for tuning into my latent beliefs and desires is “morning pages”: three pages of stream-of-consciousness writing, written freehand soon after waking. These are a cornerstone of The Artist’s Way, the best-selling creativity guide by Julia Cameron.

Before developing this habit, I’d journalled only in times of emotional upheaval. The morning pages are a different approach; I set aside my conscious, narrativising mind to see what shows up. Often I’m surprised by the patterns and paths forward that reveal themselves.

Next month Cameron publishes Living the Artist’s Way: An Intuitive Path to Creativity, about her decades of experience channelling “the wisdom inside” for support not just with writing, but in every area of life.

Like many artists, Cameron sees her intuition as guidance from outside herself, though, she tells me, she does not attempt to define it precisely: “It’s something that I feel, that I sense. I think it’s something that comes to all of us, if we listen closely enough.”

Cameron describes it as a heightened awareness or state of grace, bringing “inspiration, guidance, clarity and hope” and allowing us to access a “higher self” – whether you think of that as the wisest version of you, or something more spiritual.

“I think we’re afraid to say that we believe in intuition, because we’re afraid of being labelled crazy,” Cameron says. But her approach is straightforward: she asks a specific question, then listens. “The response is often gentle, direct and optimistic.”

It’s a skill she’s developed over years of practice, Cameron says. Writing the dialogue down, as in the morning pages, helped her to learn to trust what she hears, because she has a record of all the times it steered her well.

Related: Why are so many of us addicted to drama?

I tell Cameron about my friend Mandy, who broke up with her partner. Mandy’s growing doubt about her relationship had manifested, literally, in her gut: her digestion played up, and her skin broke out.

Once Mandy made the decision, she felt physically lighter and revived – “sparkly” was the word she used with me: “There was that physical feeling of ‘You’ve done the right thing.’”

Cameron suggests that such a “sense of brightness” stems from trusting ourselves, meeting our needs and surrendering to forces outside our control. “I think we have a choice: we can either have faith, or we can have fear,” she says. “When we surrender our control to a higher force, we’re in a sense going along with the joke.”

Once or twice, in times of personal angst, I’ve had a sudden, unexpected understanding of exactly what I needed to do, like a wise and kindly voice cutting through my prevarication and taking charge.

But, Cameron agrees, such realisations are not easily talked about. “We live in a society that tells us not to trust our intuition – it’s difficult for us to fly in the face of all the messages of our culture.”

That said, she adds, people have become “far more open-minded” since the pandemic and the “spiritual hunger” it provoked: “We found ourselves saying: ‘There must be something more.’”

It’s true that mainstream interest in spirituality increased with Covid-19: astrologers and tarot readers reported a boom in bookings; other people found or reinforced their faith. Both responses, traditional and “new age”, were linked to protecting mental health through the global upheaval.

The pandemic tested our innate desire for certainty and control; it also forced us to spend time alone with our thoughts, leading many to question our values, desires and daily lives. Solitude: the Science and Power of Being Alone, a forthcoming book by the psychologists Netta Weinstein, Thuy-vy T Nguyen and science writer Heather Hansen, shows that time alone helps people to connect and act in line with their values.

We might see being true to ourselves in the face of others’ opinions and social pressures – what the authors call “autonomous functioning” – as a product of being in touch with our intuition. But what does the science say?

***

“Intuition is real,” says Joel Pearson, a professor of cognitive neuroscience at the University of New South Wales in Sydney, and author of a forthcoming book on the subject. “We can understand it, and measure it.”

Pearson is among a handful of scientists studying the murky science behind gut feeling. “People are fascinated by it, but there’s a bit of confusion about what it is,” he tells me.

“To be clear, I’m not talking about a spiritual, magical thing that connects everybody in the ether … The way I see intuition can be explained with the science that we already have.”

His definition is the “learnt, productive use of unconscious information for better choices or actions”, best trusted only in contexts where we already have considerable experience.

Pearson gives the example of walking into an unfamiliar cafe, and disliking it for some reason you can’t specify. “Your brain is processing hundreds, if not thousands of different things: the temperature, the music, the tables, the boss, the style of the staff.”

With time and experience, we come to associate these clues “with good or bad outcomes”, Pearson says, which trigger an emotional response in the body.

This sensation is often subtle, felt in the stomach – hence “gut feeling” – or sometimes the fingertips, Pearson says. “Your heart rate will go up, you’ll start sweating a little bit more – your body will change, and you’ll feel these things as they’re happening, even if you have no idea why.”

His research has already established “big individual differences” in people’s connection with their intuition, though the reason why isn’t yet clear. Pearson suspects that some people are just more sensitive to their internal bodily state: “They notice when these things are happening and feel that gut response, while other people don’t so much – they don’t learn to rely on it.”

He likens intuition to AI and machine learning: our brains are hard-wired to see patterns and reach conclusions that reflect their inputs. If you’ve never been to a cafe before, for example, your intuitive response is not going to be meaningful or trustworthy, says Pearson. “You have to build these associations.”

Blindly trusting intuition can embed unconscious bias, such as age-, gender- or race-based prejudice – so it’s important to use it judiciously. “There are situations when you can use it – but there are situations when you absolutely shouldn’t,” says Pearson.

He warns against trusting your intuition when you’re depressed, anxious, angry or otherwise in the grips of strong emotion; or when it’s driving you towards a substance or behaviour that’s addictive.

“Food, alcohol, drugs, cigarettes, social media, gambling: the pull towards those things can feel natural, a lot like intuition – but it’s not, and we shouldn’t confuse the two,” says Pearson.

Even positive emotions can lead us astray. “If you’ve just fallen in love, or won the lottery, you’re going to misattribute that feeling to whatever you’re making the decision about … and wipe out any of the subtleties of intuition.”

Gut feelings are also not a dependable guide in unfamiliar or unpredictable contexts, or as a predictor of low-probability events like a plane crash or shark attack.

Like me, Pearson trusts his hunches most when at work. His research has found that people in many different fields, from sports to the military, do the same – though they may not say so publicly.

“A lot of CEOs and C-suite managers are really into intuition, but they won’t admit that to the board of directors,” Pearson says: they fear being written off as spiritual. But if we’re to better understand and harness intuition, he continues, “the first step is moving away from this taboo.

“Part of the issue, in psychology, has been people arguing that it’s good, or it’s bad – and it’s both. Once you understand how the brain works, and what the components are … the whole landscape changes.”

At the pub on Saturday night, I explain all of this to my friend who wants to quit her job. She listens attentively. “What I’m hearing,” she says, “is that I should quit my job.”