Stavros Macrakis & Eric Pugh – User Behavior Insights

Struggling to understand what your users are actually doing on the site when it comes to searching? Want to understand which queries are under performing and which are over performing? Want to understand which documents are seen but never clicked? You need User Behavior Insights!

Every search professional needs data about users' behavior. Data is fundamental for analyzing user behavior and improving search relevance, both with manual tuning and with machine learning. User Behavior Insights system provides a standard way to do just that.

Business product managers, UX designers, and relevance engineers need data to understand their users: What do they search for? What do they click on? What do they act on or buy? How do they use facets and filters? How do they refine their queries within a session? Engineers need data to improve search relevance and effectiveness, both manually and using AI/ML.

But until now, collecting user behavior data has been haphazard. Proprietary SaaS search systems collect data but don't share it with their customers. Open-source systems simply don't offer data collection mechanisms. Third-party analytics systems are designed primarily to track page-to-page flow, and not the flow of search results through the system -- and often make it difficult to extract the raw granular data needed for model training. Consequently many search teams develop and maintain their own ad hoc data collection systems and analysis tools.

Our open-source User Behavior Insights (UBI) system provides a client-side library for instrumenting web pages, a server-side library for collecting data, and analytical tools for understanding it. Critically, it defines a standard schema for behavior data so that the community can contribute additional analytical tools. We have also demonstrated its integration with personalization software.

With the emergence of even more ways of generating and ranking search results -- neural dense search, neural sparse search, model fine-tuning, hybrid search, RAG, ... -- choosing the best mix of approaches for your search application becomes even more critical.

UBI is a call to action to the Search Relevance community to make it simpler to seamlessly track, in an ethical and safe manner, the steps of a user’s search journey in order to build the experiences that the future requires.

Speaker:
Stavros Macrakis – https://opensearch.org/community/memb...
Eric Pugh – https://opensearch.org/community/memb...

More: https://opensearch.org/events/opensea...

OpenSearch is a community-driven, open-source search and analytics suite used by developers to ingest, search, visualize, and analyze data.

Learn more at:
Website: https://opensearch.org
Forum: https://forum.opensearch.org
GitHub: https://github.com/opensearch-project/
LinkedIn:   / opensearch-project