StreamNative Cloud: Deploying Debezium Postgres Source using pulsarctl
This video was created using StreamNative Hosted, a fully-hosted Pulsar cluster option available from StreamNative.
Quickly deploy the Pulsar Debezium Postgres Source Connector to StreamNative Cloud. This source connector allows you to persist Postgres database changes to StreamNative Cloud. In this video we deploy the Debezium Postgres Connector using pulsarctl, make edits to a database table, and view those changes are persisted in topics using the Pulsar Java Client.
[00:00] Deploying Debezium Postgres Source Connector to StreamNativeCloud using pulsarctl
[00:31] Discuss versions
[01:45] Prepare AWS RDS Postgres Database
[04:18] Prepare StreamNative Cloud Cluster
[05:50] Connect to StreamNative Cloud Cluster as service account using pulsarctl
[08:02] Discuss pulsarctl command used to deploy Debezium Postgres Source Connector, deploy connector using pulsarctl
[11:32] Test connector by making database changes and consuming messages using Pulsar Java Client
At the time of this recording, StreamNative Cloud deploys version 2.9.2.17 of the Debezium Postgres Source Connector which uses Debezium 1.7.
Supported versions of Postgres for Debezium 1.7:
https://debezium.io/releases/1.7/
Debezium 1.7 configuration requirements for Postgres on Amazon RDS:
https://debezium.io/documentation/ref...
Create a service account and service account binding using snctl:
• Prepare Your Cluster Part 1: Create a Serv...
Providing service account sources permissions using pulsarctl if unavailable in the UI:
• Prepare Your Cluster Part 2: Provide Servi...
If you are unfamiliar with using pulsarctl, the following video provides another example of downloading pulsarctl, getting pulsarctl connection information from StreamNative Cloud UI, and connecting using your service account (needed to deploy the source as the service account):
• Deploying Pulsar Functions to SN Hosted wi...
Pulsarctl command used to deploy source:
pulsarctl sources create --archive builtin://debezium-postgres \
--tenant pgtenant \
--namespace pgnamespace \
--name pgsource \
--destination-topic-name pgtenant/pgnamespace/pgoutput \
--source-config '{"database.hostname":"*****",
"database.port":5432,
"database.user":"postgres",
"database.password":"*****",
"database.dbname":"postgres",
"table.whitelist": "public.users",
"database.server.name":"mydbinstance",
"plugin.name": "wal2json"}'
Source specific documentation:
https://docs.streamnative.io/hub/conn...
If you are unfamiliar with using Pulsar Java Client, the following video provides detailed directions on obtaining sample Java code from StreamNative UI and setting up Itellij:
• Test Pulsar Functions with SNProducer and ...
Auf dieser Seite können Sie das Online-Video StreamNative Cloud: Deploying Debezium Postgres Source using pulsarctl mit der Dauer stunde minuten sekunde in guter Qualität ansehen, das der Benutzer StreamNative Academy 25 August 2023 hochgeladen hat, den Link mit Freunden und Bekannten teilen, dieses Video wurde auf Youtube bereits 267 Mal angesehen und es wurde von 3 den Zuschauern gefallen. Viel Spaß beim Betrachtenden Zuschauern gefallen!
