I’m desperate to go on holiday somewhere hot and sunny and I don’t have time to plan my own itinerary. No biggie, I’ll just ask ChatGPT. Sure enough: as if by magic, and almost instantly, everything I need appears on my screen.

Three years ago, I’d have used a search engine like Google or Bing, entering a few keywords – “winter sun holidays” and in return getting a list of tour companies’ websites and reviews-based sites such as TripAdvisor – before clicking on the results (the “search-and-click” model). If I wanted a more personalised itinerary, I’d probably have followed up with a call. It all seems so cumbersome now.

Using a chatbot is more like asking a friend – “Where can I go and sunbathe in the morning, get my cultural fix in the afternoon and eat great food at night? Need your advice.” It’s a conversational experience, and the answer you get is often complete in itself.

Will traditional searches decrease in number now we have GenAI? And will people browse fewer websites as a result? According to new research by Anja Lambrecht, Professor of Marketing at LBS, and Nicolas Padilla, Assistant Professor of Marketing at LBS, the answers are yes and yes, sizeably. These are just two findings in their paper ‘The impact of LLM adoption on online user behavior’, authored with two academics from UCLA, H. Tai Lam, Assistant Professor of Marketing and Brett Hollenbeck, Associate Professor of Marketing.

Are LLMs substituting for traditional search or serving as a complementary tool?

Large Language Models (LLMs) are a kind of generative artificial intelligence (GenAI); “large” because they learn to read and write human by digesting immense datasets, “generative” because they can create original content. The researchers explored how LLMs have in a very short time transformed how we use the internet, and what this means for content providers and the broader web ecosystem: search engines, online publishers and advertisers.

“If users shift queries away from traditional search engines that direct traffic to a wide range of websites toward LLMs that provide synthesised answers and fewer pathways to external content, the underlying revenue model for content creators may be jeopardised,” they explain. “These shifts also raise competitive questions, as LLMs increasingly operate as alternative gateways to information that challenge the central role long held by search engines.”

At the same time, LLMs may serve as a complementary tool, they suggest, fulfilling a distinct function that is complementary to traditional search and online content consumption: “For example, LLMs excel at understanding complex queries and pre-structuring broad sets of information in an accessible way, but at times struggle to reliably providing specific factual details or credible primary sources. This balance of strengths and weaknesses could result in their adoption, facilitating more efficient initial search, which then results in a greater overall search volume and visits to publisher websites.”

Becoming aware of shifts in consumer behaviour and being able to base that awareness on firm evidence is vital for businesses seeking to get their products and services noticed above the competition. Understanding whether LLMs substitute for or complement traditional online search and content consumption is important for evaluating their impact.

LLM adoption: three areas to investigate

The research paper investigates three aspects of the adoption of LLMs:

  1. How does it affect online search? Although LLMs are able to answer questions, which might suggest substitution, concerns around the quality of those answers could mean people still continue to search in other ways as well, potentially even increasing the overall search volume.

  2. When does LLM use substitute for broader online activity, and when doesn’t it – and what is the impact on web publishers? The researchers evaluated the impact on adopters’ online traffic overall, for smaller vs larger websites, and for two distinct content categories: educational websites and user-generated content sites.

  3. Does LLM adoption affect users’ advertising exposure? If so, that might shift web publishers’ ability to monetise content, and retailers’ ability to reach consumers. Till now, brands have typically invested in a combination of search ads, which are triggered by particular keywords in a search, and display ads, which show up when a person is online but not necessarily searching, based on their demographic, browsing habits and interests.

To tackle these questions, the researchers analysed a huge set of clickstream data – almost 1.2 million url-level data from desktop browsers from late 2022 through to mid-December 2023 (OpenAI launched Chat GPT, the first user-friendly chatbot, on 30 November 2022). They narrowed this down to the online usage of 2014 American households that had used LLMs for three consecutive weeks.

Firstly, they looked at search, the entry point to online browsing for most users. As anticipated, the number of traditional searches dropped – slowly at first, as users got acquainted with the new technology – and by 20 weeks in, by more than 20%. The decline affected question-based searches but not navigational searches (i.e., when someone is looking for something specific, such as a brand name).

Then, they examined how this had affected website traffic. Frequently visited websites were not affected, but smaller websites suffered a significant drop in business. The researchers point out that the results suggest large, well-established sites remained relatively insulated, but at least some of the smaller sites, which are most dependent on search-driven referrals, lose out.

Source link