[Thank Goodness It's Search] Do you need vectors ?

Welcome to Thank Goodness It's Search series—your Friday fix of OpenSearch learnings, feature drops, and real-world solutions. I will keep it short, sharp, and search-focused—so you can end your week with more search knowledge than you started with.

When, Why and What do you need vectors for?

When should you convert your data to vectors?

These are the top 5 scenarios where converting your data to vectors can significantly enhance search performance, particularly when your users are:

1. Searching using long-tail, natural language queries
2. Looking for similar items based on their preferences or behavior.
3. Seeking conceptual understanding of the data, such as synonyms or related concepts.
4. Searching for information that's embedded within an image, a Cross-Modal Search. For example "red shoe with green lace" and the product is tagged with multiple colors and you want the order to be maintained right (and not return say, green shoes with red lace!).
5. When users see zero or irrelevant results and you are looking to expand recall and surface more related content if the exact match is not found.

Why should you convert your data to vectors?

Converting data to vectors is a fundamental step in enabling semantic search capabilities. By transforming complex data into numerical vector representations, we can leverage machine learning algorithms for advanced processing. This vectorization is essential for multiple use cases including natural language processing, image recognition, and recommendation systems. Vector search plays a crucial role in enhancing recall for RAG (Retrieval Augmented Generation) applications by helping OpenSearch retrieve relevant context for foundation models. However, without proper fine-tuning of semantic search, the retrieved context may be inaccurate, leading to irrelevant or hallucinated responses when integrated with Large Language Models.

Remember, vector search isn't some magic bullet that solves everything! OpenSearch's good old lexical search with BM25 already does a solid job giving you relevant results right out of the box. So, you should start by tuning the baseline lexical results first. Then do a proper A/B test comparing lexical vs semantic vs hybrid search to see what actually works best for your gen AI app. Only then should you go through the effort of vectorizing your whole dataset.

What should be converted to vectors?

When determining what to vectorize, focus on data that benefits from semantic understanding. Product descriptions are a prime example - vectorization captures their semantic meaning, allowing products with similar description to be clustered together in vector space for more accurate similarity search results.

However, be selective about what you vectorize. Structured data such as numbers, dates, and categorical fields typically perform better with traditional search methods. Vectorizing everything can be both computationally expensive and unnecessary. Most general-purpose, pre-trained text embedding models also lack domain-specific understanding. For instance, in life sciences, specialized terms like apoptosis or mitosis may not automatically be associated with broader concepts like biological processes. Likewise, searching for a company name like Cognizant could return irrelevant results such as Conscious, which clearly misses the mark. Looking at yet another example from retail, a domain-specific term like herding ball might not be recognized as a pet toy by a general, pre-trained, text-embedding model. You might end up with completely unrelated results (like elephants) instead.

To mitigate these issues, you would have to tag your data with standardized, domain-specific terms or train a custom embedding model that understands your industry’s vocabulary and concept relationships. This approach does add additional overhead but leads to significantly more accurate and relevant search outcomes. It all depends on your use case and your user's search patterns.

Top 5 myth-busters about vector search

1. Vectors are always the best solution for search! Not necessarily. While vector search can boost recall, it may also introduce noise. The best results come from carefully balancing precision and recall.
2. Vectors can replace traditional search methods! False. Vectors complement and do not replace the traditional search. For use cases involving domain-specific taxonomies or tuned keyword matching, traditional lexical techniques (like synonym lists and custom dictionaries) often outperform.
3. Vectors are always better than traditional search! Not true. Vector search isn't a one-size-fits-all solution. Success depends on data type, query intent, and the need for exact matches versus semantic similarity.
4. Vectors require extensive computational resources! This is old news! Modern techniques such as quantization and model distillation make vector search more efficient and accessible—even for smaller infrastructure setups.
5. Vectors are only for large datasets! Incorrect. Vector search can also benefit small and mid-sized datasets by improving relevance, especially where semantic understanding matters more than sheer volume. OpenSearch also offers exact kNN which has 100% recall and can be especially useful for smaller datasets.

Conclusion

Converting data to vectors is a powerful tool for enhancing search, but should be viewed as one of many tools rather than a complete solution. When used strategically alongside other search techniques, vectors can improve relevance and accuracy of results. However, the quality of results still depends on having clean, well-tuned underlying search. Poor search quality will lead to poor results regardless of the search approach used.

Before implementing vector search, you should:

1. Evaluate specific use case and data requirements
2. Ensure you have a solid foundation with traditional search methods and baseline recall.
3. Consider computational resources and infrastructure needed
4. Test and compare results between different search approaches
5. Plan for ongoing maintenance and optimization

The key is finding the right balance between vector and traditional search methods for your specific needs. Start small, measure results, and scale up vector implementation based on proven value and performance improvements.

Call for action

If you found this article helpful, please share it with your network. If you have any questions or want to discuss how vectors can improve your search experience, feel free to reach out. And if you want to see vectors in action, check out the OpenSearch documentation on vector search.

Looking to build smarter AI search? Do you have the right insights to move ahead with the build ? Check out some thoughts and references that would get you started on building smarter search.

Did you catch my previous series on faceted navigation? Start the 5-part series on faceted navigation here

Want to learn more? Check out the OpenSearch Documentation

See you next Friday with another search solution. Until then, happy searching! 🔍