Marqo

Last reviewed Sep 13, 2026 · 819 words

Marqo's pitch is that you send it raw text or image URLs and get semantic search back, with no separate embedding service, no model-serving container and no pipeline code. That pitch is true, and it is the reason to choose it over Qdrant or Weaviate for a first project. The cost is that the embedding model now lives inside your search engine: our catalogue's 4 GB minimum is the honest floor for the default text model on CPU, and indexing speed on CPU is in the low tens of documents per second. For a few hundred thousand documents that is an afternoon; for tens of millions you want a GPU or a different architecture.

One container replaces three

A conventional vector search stack has an embedding step (a model behind an API, either hosted or self-run), a vector database, and glue code that keeps the two in sync when documents change. Marqo collapses that. You create an index with a named model, add_documents with plain fields, and search with a plain query string; the server embeds on both sides. Deleting a document removes its vectors. Updating re-embeds. There is nothing to drift.

import marqo
mq = marqo.Client(url="http://localhost:8882")
mq.create_index("docs", model="hf/e5-base-v2")
mq.index("docs").add_documents(
    [{"_id": "1", "title": "Restic restore steps", "body": "..."}],
    tensor_fields=["title", "body"],
)
mq.index("docs").search("how do I restore a backup")

That is the complete application-side code for text search. Image search is the same call with an open_clip model and a field holding an image URL, and a text query then finds matching pictures. Multimodal in 6 lines is the part that impresses people in demos.

The model choice is the deployment decision

Because the model runs in-process, picking it decides your RAM, your latency and your relevance in one go. Small sentence-transformer models embed fast on CPU and fit the 4 GB floor. The larger e5 and CLIP variants push memory toward 8 GB and want a GPU; Marqo ships a CUDA image for that case and a single consumer card handles indexing at hundreds of documents per second. You cannot change a live index's model without reindexing, so pick on a sample corpus before loading the real one.

Marqo 2.x sits on Vespa underneath, which is why it scales further than a naive Python service and also why the container is not small. Mount the data directory to a volume the first time you run it: recreate the container without one and every index is gone.

Where a split stack still wins

Qdrant plus your own embedding call beats Marqo in 3 situations I keep meeting. When you already generate embeddings elsewhere (an LLM pipeline that embeds anyway), Marqo's built-in model is dead weight and its "bring your own vectors" mode is less natural than a pure vector store. When you need a hosted embedding API such as OpenAI's for quality reasons, Marqo can be configured to call out for some models but the all-in-one advantage evaporates. And when you need heavy filtering, payload indexing and multi-tenancy at scale, Qdrant's feature set is deeper. Weaviate is the closest direct rival, with its own vectoriser modules, a much larger community, and more RAM appetite. Our vector database guide covers that fork in the road properly.

If what you actually need is keyword search over product listings or documentation with typo tolerance and instant results, Meilisearch on 200 MB of RAM will serve better than any vector engine; semantic search is a specific tool, not an upgrade to all search.

The REST API is stable and the Python client is thin

Everything the client does maps 1:1 to HTTP calls on port 8882, so a Go or Node application uses it without an SDK. Hybrid search (lexical plus tensor, fused) is a query parameter. Filtering uses a small string syntax on document fields. There is no authentication built in; put it behind a reverse proxy with an auth layer or keep it on a private network, because an open Marqo port lets anyone delete your indexes with one DELETE request.

What I'd do

For a team building its first semantic search or a retrieval layer for an internal assistant, I would start with Marqo on a 4-core, 8 GB box with hf/e5-base-v2, mount the volume, and stay with it until the corpus passes a few million documents or GPU costs start to matter. At that point re-evaluate: if embeddings already exist elsewhere in the pipeline, move to Qdrant; if not, add a GPU and stay. The all-in-one design is the right default for the first year of a project, and the simplicity it buys is worth the 4 GB.