Nobel laureate Daniel Kahneman is notable for his work on the psychology of judgement and decision making. Photograph: Richard Saker

The idea of consumers making fully reasoned decisions is finally being debunked. Events like the financial crisis and fresh research have successfully challenged the idea that rationality is at the heart of our choices.

The most prominent thinker in this area is psychologist Daniel Kahneman. His book, «Thinking, Fast and Slow», has now sold over a million copies in the UK. He’s demonstrated that our mind has two systems of thought. System 1 is a fast, automatic and intuitive process over which we have little conscious control. System 2, which corresponds with our idea of rational reasoning is slow, deliberative and effortful.

This cleaving of the mind into two spheres isn’t new – the idea stretches back to antiquity. However, what is new is the idea that the vast majority of decisions are made by System 1. Even when we think we’re making reasoned conscious decisions often the conscious mind is merely post-rationalising decisions that have already been made. In the memorable words of Jonathan Haidt, author of The Righteous Mind, the rational minds «thinks it’s the Oval Office when actually it’s the press office».

This is problematic for businesses in a number of ways. Firstly, it means that asking consumers what they want is flawed. If they don’t have full introspective insight into their decision-making process what they claim motivates them is often inaccurate.

Another, broader repercussion springs from the fact that choice is effortful. As Kahneman has said, «Thinking is to humans as swimming is to cats; they can do it but they’d prefer not to.» It means that the ever-burgeoning growth in choice might have unintended consequences.

Classic psychological research by Sheena Iyengar and Mark Lepper of Columbia University has investigated the potential negative effects of an excess of choice. They set up a stall at a posh food market selling a range of gourmet jams. They alternated between offering six and 24 varieties of jam. While the large display attracted more interest than the small one, people were 10 times as likely to buy from the stall with the smaller variety of choice. For brands that try and squeeze out every extra percentage point of sales a tenfold increase is astonishing.

The psychologists concluded that consumers failed to buy at the more varied stall as the choice became too effortful. Additionally, researchers like Barry Schwartz have suggested that happiness with a purchase declines when there is excessive choice. In these circumstances consumers worry that there’s an increased chance that one of those unchosen goods would have been a better use of their money.

Despite these findings the variety of choice we face is ever-expanding. Even Starbucks offers 80,000 potential combinations of drinks. But if these mind-boggling levels of choice dampen sales why would companies, whose aim is to maximise profits, continue to offer such large ranges?

One answer might be in the fact that companies can boost the price that customers are willing to pay by introducing higher-end goods. The agency, Mountainview, ran an ingenious experiment in which they offered people a range of beers. When consumers were offered a mass market beer (Carling) for £1 and a premium beer (Budvar) for £2, two-thirds opted for the pricier beer. However, when a third option was introduced, the super-premium Kronenbourg Blanc at £4, preferences switched. A few people chose Kronenbourg Blanc but the remaining 90% chose Budvar. Consumers who had preferred Carling to Budvar in one context switched to Budvar in the presence of a high-end option. The presence of the higher-end good seems to shift people into the middle choice – for some people it avoids the feelings of being ripped off or of being seen as frugal.

The second reason that choice doesn’t seem to be being reduced is that companies are becoming better at how they present their offerings to consumers. Amazon, for example, has millions of items yet its algorithms carefully select a handful of recommendations based on past purchasing behaviour. Despite the breadth of the potential range, the Amazon experience can be less baffling than a visit to a small bookstore. Other brands offer less technologically driven solutions, guiding the consumer by promoting their most popular items. If everyone else has bought it, the thinking goes, it must be a satisfactory choice.

These examples and experiments suggest that the range of choices we’re offered can have profound effects. Introducing additional choices can change customers’ preferences – even when the new option isn’t purchased. However, companies must be wary – too much choice can be demotivating. Behavioural economics will continue to generate interesting and useful insights for companies which they can then test and learn from in almost real-time on their digital ads and website.

Richard Shotton is head of insight at ZenithOptimedia.