Foreword
This is a small, deliberately simple experiment — a first-pass look at how well low-cost semantic search performs on a real product catalogue, not a comprehensive benchmark. The catalogue belonged to a Shopify apparel store; we used only publicly available product-page information, with no access keys, passwords, or private data.
We share it because the approach is inexpensive, easy to reproduce, and increasingly relevant for e-commerce.
Note: if you’d like something like this for your store, or you have any questions, please feel free to
ask us.
Introduction
Most online-store search is still built around keywords. Type the exact word that appears in a product title or description and you’ll find the item — but phrase your need the way a real shopper thinks (
“comfortable clothes for home and office”,
“a gift for my husband who loves the outdoors”) and keyword search often returns weak matches, or nothing at all.
Semantic search closes that gap by matching on
meaning rather than exact words. This case study measures how effective, accurate, and fast a simple semantic-search setup can be — and what it costs.
A quick primer: embeddings and vector search
An
embedding model turns a piece of text into a list of numbers (a
vector) that captures its meaning. Texts with similar meaning end up close together in this vector space. To search, you embed the user’s query the same way and look for the product vectors nearest to it — here, using
cosine distance as the measure of closeness. The smaller the distance, the more semantically relevant the product.
This retrieval step is the core of
Retrieval-Augmented Generation (RAG), and on its own it already powers “search by intent”.
What we tested
We used OpenAI’s
text-embedding-3-small model to embed every product description in the catalogue. We chose it deliberately: it is inexpensive — about
2¢ per 1M tokens, or
1¢ per 1M tokens with batch requests — and more than capable for product-scale text. Test scripts were written in
Python. We issued
single requests to emulate a live shopper’s search, and cheaper
batch requests to build the initial database of embeddings.
Where the vectors live
We stored the embeddings in
PostgreSQL with the pgvector extension and ran the search directly in the database. That keeps the stack simple and is a perfectly good starting point. The same setup runs with a local embedding model when data must stay on-premises.
In larger or higher-traffic systems you would often reach for a dedicated
vector database — Qdrant, Pinecone, Milvus, or Weaviate — which add horizontal scaling, metadata filtering, and faster approximate nearest-neighbour search. Which one fits depends on catalogue volume, latency targets, and how much infrastructure you want to run.
Speed
For live, single queries — the path a real shopper hits — responses from the embedding service averaged about
0.8 seconds, with occasional peaks to 1.6 and even 2.8 seconds.
Building the database with batch requests averaged about
3.5 seconds per batch of 100 products, occasionally rising to 5.5–10 seconds. That batch time is a one-off cost: the embeddings database is built once and then simply kept up to date as the catalogue changes. In short — fast enough for interactive search, provided you design for the occasional latency spike.