Postgres pgvector Extension - Vector Database with PostgreSQL / Langchain Integration

Blog Post: https://bugbytes.io/posts/vector-data...

In this video, we'll look at the pgvector extension for PostgreSQL, that allows you to turn your Postgres database into a vector data-store!

pgvector adds the vector data-type and distance computation operators (L2, inner product, and cosine distance) to allow you to query for "similar" items in the vector-space.

We'll see how to set pgvector up in a Docker container, and will see how to integrate it with Langchain via the PGVector object.

We'll look at how to take a piece of text, split it into chunks, create embeddings from those chunks using OpenAI, and then store the embeddings in the Postgres vector database. We'll also see how to query the database for vectors/documents that are similar to a text prompt/query.

☕️ 𝗕𝘂𝘆 𝗺𝗲 𝗮 𝗰𝗼𝗳𝗳𝗲𝗲:
To support the channel and encourage new videos, please consider buying me a coffee here:
https://ko-fi.com/bugbytes

📌 𝗖𝗵𝗮𝗽𝘁𝗲𝗿𝘀:
00:00 Intro
00:41 Introduction to pgvector for PostgreSQL
03:23 Splitting text file into chunks with Langchain RecursiveCharacterTextSplitter
06:10 Using OpenAI to get embeddings for each chunk with OpenAIEmbeddings object
10:54 Setting up pgvector and PostgreSQL in a Docker container
16:38 Using the Langchain PGVector object to connect to PostgreSQL
21:47 Finding similar vectors to a query in pgvector
25:29 Querying pgvector with SQL to get cosine distances

𝗦𝗼𝗰𝗶𝗮𝗹 𝗠𝗲𝗱𝗶𝗮:
📖 Blog: https://bugbytes.io/posts/vector-data...
👾 Github: https://github.com/bugbytes-io/
🐦 Twitter:   / bugbytesio  

📚 𝗙𝘂𝗿𝘁𝗵𝗲𝗿 𝗿𝗲𝗮𝗱𝗶𝗻𝗴 𝗮𝗻𝗱 𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻:
Blog Post: https://bugbytes.io/posts/vector-data...
pgvector: https://github.com/pgvector/pgvector
pgvector DockerHub image: https://hub.docker.com/r/ankane/pgvector
State of the Union text: https://github.com/hwchase17/chroma-l...
OpenAI Embeddings: https://platform.openai.com/docs/guid...
Langchain Vectorstores: https://python.langchain.com/docs/mod...

#python #langchain #datascience #postgresql