Aiming for the Best Search Experience to Find Items Immediately—What Is Behind Mercari R4D’s AI Search and Recommendation Model Development Work Selected for GENIAC?
R4D’s research has been selected for the Generative AI Accelerator Challenge (GENIAC), a national project to enhance the development capabilities and accelerate the social implementation of generative AI in Japan.
In this article, we’ll explain the R4D research selected for the project in detail.
Making item searches more convenient
In brief, the purpose of this project is to improve the search experience for users of secondhand marketplaces such as Mercari.
To date, over 4 billion items have been listed on Mercari, resulting in the platform’s many transactions. By leveraging data such as item information and transaction logs, we aim to develop the foundation for a Japan-born AI search and recommendation system tailored to the reuse market in order to innovate the search experience for Mercari users.
Existing search technology is ineffective at returning search results for items with no known names or for ambiguously worded searches
Many of the items on marketplace apps are used goods that do not have model numbers, or one-of-a-kind items whose actual name is unknown.
For example, the barcodes or international standard book numbers (ISBN) on books make it possible to get the relevant item information quickly. However, Mercari offers a wide range of items, including antique furniture and vintage ornaments, one-of-a-kind handmade items, toilet paper rolls, and even artisanal mud balls; there are therefore many items without official product information.
In fact, about 80% of the items listed on Mercari cannot be linked to any existing product catalog.
Such niche items are often listed using simple item descriptions or described in unique technical terms, making it rather difficult to return suitable search results using conventional keyword-matching search engines (which will be explained later in this post).
Moreover, the more niche an item is, the smaller the pool of potential buyers there is to search for or purchase that item. In data analysis terms, this results in less user behavior data related to that item. As a result, conventional item recommendation systems based on user behavior data also struggle to function effectively in this environment.
Additionally, it becomes even more challenging to accommodate vague natural language searches, such as “a vintage-style chair I’d like to take with me on a weekend camping trip,” or “an elegant fashion arrangement that I could wear to a wedding.” Even if an item perfectly matching the search is listed on a marketplace, if the item description states something like “formal outfit with a low-key vibe,” the item will not be associated with the search terms “wedding,” “elegant,” or “arrangement,” making it less likely for the item to appear in the list of suggestions.
This leads to users trying to run their searches multiple times and missing out on purchasing opportunities because the search experience is not accurate enough to return certain results.
Generative retrieval is a key technology of this project that can help solve these challenges. An AI trained on vast amounts of item information and that understands the meaning and context of each item can respond to users’ search requests flexibly.
Before explaining the mechanism of this new AI-based search and recommendation system, let’s look at a brief description of conventional search mechanisms for comparison purposes.
Conventional search mechanisms
The basic principle of conventional search engines is keyword matching.
First, the search engine reads the data of all the items and automatically creates something similar to a reverse-lookup dictionary based on the words in the listings.
Let’s use the following four items in our example.
- Item A (ID: 1): Completely wireless earphones, waterproof, noise-canceling
- Item B (ID: 2): Deep bass, waterproof Bluetooth speaker
- Item C (ID: 3): Bone conduction earphones for running
- Item D (ID: 4): Bone conduction Bluetooth earphones, waterproof, for meetings
The search engine breaks the descriptions down into individual words and stores them in a database as follows.
| Word | List of IDs of items where the word appears |
|---|---|
| Earphones | ID: 1, ID: 3, ID: 4 |
| Waterproof | ID: 1, ID: 2, ID: 4 |
| Speakers | ID: 2 |
| Bluetooth | ID: 2, ID: 4 |
| … | … |
Next, let’s pretend the user enters “waterproof earphones” into the search field. The search system refers to the database above and finds IDs 1 and 4, which share the terms “earphones” and “waterproof.”
If the search returns a large number of items, the system will determine the order in which to display the search results. It will rearrange and display the results in descending order based on factors such as the concentration of keywords in each item description (BM25 algorithm) as well as the “sales score” and “click-through rate” on the e-commerce site.
Essentially, the matching of search words and item descriptions is the key to the search process.
The mechanism of searches using generative retrieval technology
So, what exactly is “generative retrieval technology,” which is described as the key of this R4D research project?
In simple terms, it is an approach that uses a massive AI to grasp and memorize all the listed items on a marketplace. The AI can then use this information to interpret the context of a user’s search request and directly suggest appropriate items.
To understand this mechanism better, let’s look at how the AI model is created.
First, an ID (semantic ID) that reflects the “meaning” of each item is created based on the training data. (The key point of this process is to assign meaning to the ID.)
Take for example a pair of sneakers. As an item, the sneakers belong to the sneaker category, which is within the fashion category. They are also described as red and white in color, 26 cm in size, and optimal for running. All of this information will be included in the ID to reflect the meaning of the sneakers as an item. (Of course, the actual ID will not be a string of words, but rather a numeric ID converted from the words in the item description.)
Next, the AI model memorizes all the semantic IDs of the items.
When a user actually performs a search, the AI model generates a semantic ID from the context and nuance of the search based on the search words (e.g., running sneakers, red) or ambiguous queries in natural language (e.g., red sneakers optimal for running to lose weight) entered by the user. The AI model then finds items that have matching or similar IDs from among all the memorized items, and returns those items as search results.
Unlike conventional search engines, systems using generative retrieval technology do not search for items that match the keywords. They match an ID that includes the user’s search intent generated from the search query with the IDs that reflect the meaning of the items the AI model memorized during model training and returns the search results.
Therefore, even if the user tries to search for a niche item, or if the search phrase is written in ambiguous natural language, the AI model can still provide high-precision searches and recommendations, because it has a deep understanding of all the items it was trained on, enabling high-precision searches and recommendations.
A unique AI model trained on data specific to Mercari
This section introduces the three types of data used to develop the AI model for this research project. This data is the very thing that ensures the uniqueness of the AI model R4D is developing. Leveraging data unique to Mercari allows R4D to develop an AI model tailored to the reuse market.
- Mercari’s log of 4 billion items
- This includes item features extracted from listing text information (such as item titles, descriptions, and categories) and image data using a pre-trained multi-modal encoder.
- It also includes user search logs and behavior logs.
- Capital business partner Suruga-ya’s detailed catalog of 30 million items
- This catalog contains accurate and detailed information about items, such as their category, brand, color, and product attributes (such as series name and model name).
- 3 million pieces of nuanced review text
- In place of reviews posted by users, this data consists of artificially synthesized item review data generated using an LLM.
- It includes not only physical catalog information such as the item brand and color, but also text that describes usage scenes and user impressions (nuanced language) such as “suitable for festivals,” “safe on rainy days,” and “outerwear for commuting.”
If this AI search and recommendation system is implemented, what benefits will users see on Mercari?
First, there will be a dramatic improvement in the search experience.
As this article explained earlier, R4D aims to address vague search requests that conventional keyword searches struggle with. This will enable an experience where the user can find what they want with a single search. R4D’s goal is to relieve the stress of conducting searches multiple times and improve item purchase rates.
Our next focus is the acceleration of crossborder transactions (global expansion).
Japan’s hobby and entertainment culture of animation, comics, games, and esports (ACGE) domains is in high demand globally, and the GMV of crossborder transactions on Mercari has rapidly grown by 15 times over the past three years. The company plans to continue expanding the Mercari Global App to more countries going forward.
The AI model R4D is developing will break down the barriers of language and cultural context understanding, allowing overseas users to search intuitively in their native language for the items they want, even if they are not familiar with certain technical terms.
Finally, we come to the acceleration of resource circulation.
If the circulation of long-tail products that were buried within Mercari due to low discoverability improves, it will lead to the avoidance of over 690,000 tons of greenhouse gas emissions annually. This is equivalent to the amount of CO2 absorbed by approximately 79 million cedar trees in a single year.
We hope you’ve enjoyed this summary of R4D’s research on AI search and recommendations, which has been adopted by GENIAC. Thanks for reading to the end.
You can look forward to more updates on this and other R4D research projects and results coming soon.