Скачать или смотреть Continuing the Conversation: Leveraging Gen AI Tools and Semantic Search for Digital Collections

Continuing the Conversation: Leveraging Generative Artificial Intelligence Tools and Semantic Search for Digital Collections

When OpenAI released ChatGPT 3.5, Northwestern University Libraries immediately saw the potential for a new kind of search-and-discovery tool. The team pivoted to develop a chat-based, retrieval-augmented generation proof of concept that allowed end users to ask questions in natural language and receive generated answers along with semantic search results. The proof of concept made it clear that large language models (LLMs) and semantic search drastically changed what was possible in search and discovery systems. The team presented its early findings widely and held workshops with the library community to share knowledge and start the conversation. Taking cues from the startup world, Northwestern prioritized moving to production and began reimagining how chat-based search would work within a digital collections context. The result is an integrated search and discovery system that leverages the power of generative artificial intelligence (AI) and semantic search providing users with a new way to explore and understand the collections. In August 2024, Northwestern was awarded an Institute of Museum and Library Services National Leadership Grant to expand on the work, generalize the tools, and ultimately release an installable cloud-based solution for experimentation. This presentation will cover the technical and design considerations of integrating semantic search and LLMs into digital collections, the challenges faced when developing solutions in a rapidly changing environment, and the lessons learned moving to production. It will also include a demonstration of the latest version of the production system which uses an agent-based approach and introspection to answer complex multi-faceted queries. Future plans for the tool and how the team plans to continue to leverage generative AI and semantic search to improve the user experience will also be discussed.

James Lee, Northwestern University
David Schober, Northwestern University

Learn more: https://www.cni.org/mm/spring-2025/pr...

Coalition for Networked Information (CNI)
Presented at Spring 2025 Membership Meeting
April 7-8, 2025

*Subscribe to our channel*: https://www.youtube.com/c/cnivideo?su...

Stay connected with us
Website: https://www.cni.org
Facebook:   / cni.org  
LinkedIn:   / coalition-for-networked-information  
BlueSky: https://bsky.app/profile/cni-org.bsky...
Mastodon: https://mastodon.social/@cni
Twitter:   / cni_org  
Subscribe to our listserv CNI-ANNOUNCE: https://www.cni.org/resources/follow-...