Скачать или смотреть StreamNative Cloud: Deploying Debezium Postgres Source using pulsarctl

This video was created using StreamNative Hosted, a fully-hosted Pulsar cluster option available from StreamNative.

Quickly deploy the Pulsar Debezium Postgres Source Connector to StreamNative Cloud. This source connector allows you to persist Postgres database changes to StreamNative Cloud. In this video we deploy the Debezium Postgres Connector using pulsarctl, make edits to a database table, and view those changes are persisted in topics using the Pulsar Java Client.

[00:00] Deploying Debezium Postgres Source Connector to StreamNativeCloud using pulsarctl
[00:31] Discuss versions
[01:45] Prepare AWS RDS Postgres Database
[04:18] Prepare StreamNative Cloud Cluster
[05:50] Connect to StreamNative Cloud Cluster as service account using pulsarctl
[08:02] Discuss pulsarctl command used to deploy Debezium Postgres Source Connector, deploy connector using pulsarctl
[11:32] Test connector by making database changes and consuming messages using Pulsar Java Client

At the time of this recording, StreamNative Cloud deploys version 2.9.2.17 of the Debezium Postgres Source Connector which uses Debezium 1.7.

Supported versions of Postgres for Debezium 1.7:
https://debezium.io/releases/1.7/

Debezium 1.7 configuration requirements for Postgres on Amazon RDS:
https://debezium.io/documentation/ref...

Create a service account and service account binding using snctl:
   • Prepare Your Cluster Part 1: Create a Serv...  

Providing service account sources permissions using pulsarctl if unavailable in the UI:
   • Prepare Your Cluster Part 2: Provide Servi...  

If you are unfamiliar with using pulsarctl, the following video provides another example of downloading pulsarctl, getting pulsarctl connection information from StreamNative Cloud UI, and connecting using your service account (needed to deploy the source as the service account):
   • Deploying Pulsar Functions to SN Hosted wi...  

Pulsarctl command used to deploy source:
pulsarctl sources create --archive builtin://debezium-postgres \
--tenant pgtenant \
--namespace pgnamespace \
--name pgsource \
--destination-topic-name pgtenant/pgnamespace/pgoutput \
--source-config '{"database.hostname":"*****",
"database.port":5432,
"database.user":"postgres",
"database.password":"*****",
"database.dbname":"postgres",
"table.whitelist": "public.users",
"database.server.name":"mydbinstance",
"plugin.name": "wal2json"}'

Source specific documentation:
https://docs.streamnative.io/hub/conn...

If you are unfamiliar with using Pulsar Java Client, the following video provides detailed directions on obtaining sample Java code from StreamNative UI and setting up Itellij:
   • Test Pulsar Functions with SNProducer and ...