Publish-Subscribe Architecture (Explained by Example)

In this video, I want to discuss the Pub sub architecture or publish subsribe architecture and talk about the pros and cons and differences over Request Response.


One of you guys asked me which is better REST or Message queue. I struggled to answer for a while but After thinking about it I discovered that both REST and Message queue does boil down to basic elements. REST being HTTP which is Request-Response and Message Queue being Publish subscribe.

Timecodes
0:00 Intro
3:00 Where does request response pattern breaks
7:00 Response request pros & cons
11:12 Pub/sub pattern
17:50 Pub/sub pros/cons


[ ] Request/ Response
[ ] Pros and Cons
[ ] Publish-Subscribe Definition
[ ] Pros and Cons

Req/res

Browsing
Fetch or XMLHttpRequest
SQL query

Pros
Simple to implement (REST)
Stateless (HTTP)
Sync or Async (service and client)

Uploading a youtube video
Compress video
format video
Notify subscribers

Upload breaks down the video into small chunks and uploads each one

Once all successfully uploaded send a whole video to the compress service

One video compressed, the compress service sends to the format service and notifications service

If you are only using pure request-response, The client will be waiting until all of that stuff is done and if the connection is broken or any failure happens you lose.

Plus let's assume You want to also add a feature where the video will need to be checked for copyrighted material.. so the compress service now need to make a request to the copyright service and make sure all of that requests return successfully

Cons
Doesn’t scale with multiple receivers
Creates extreme coupling between multiple services
The client and servers have to be running to communicate.
chaining, retries, circuit breaking
Chain effect and circuit breaking (microservices)


Meet the publish-subscribe model

Ok, this is starting to sound like a commercial.

Publish subscribe or pubsub for short is an architecture model where a publisher posts. a message to a topic (usually in the form of a queue) and clients optionally can subscribe to the topic and get notified when a new message is posted.

Upload breaks down the video into small chunks and uploads each one

Once all successfully uploaded send a whole video to message queue or topic

The compress service subscribes to the topic. The new video will get received by the compress service, compresses the video and publish it back to the topic or queue as a compressed.

The notification and copyright service subscribes to the topic.

Redis/kafka/RabbitMQ

Pub sub
Pros
Scales with many receivers
Loose coupling(publishers don’t know the subs, subs don't know the pubs)
Caching
Work while publishers or subscribers are offline
Great with microservices avoid chaining, retries, circuit breaking
Pairs with event-driven architecture


Is the pub/sub perfect though?

Decoupling side effect is a serious problem. You just decoupled the sub from the publisher, how do you guarantee to deliver a message?

Cons
Message delivery issues (exactly once delivery ar least once)    • The Two Generals’ Problem  
Network saturation- (depends on Pubsubs system) in push model clients might get overwhelmed, in pull model false attempts
scaling is still an issue


🐘 Postgres Videos
   • PostgresSQL  

🧮 Programming Pattern Videos
   • Programming Patterns  

🛡 Web Security Videos
   • Web Security  

🦠 HTTP Videos
   • HTTP  

🏰 Load Balancing and Proxies Videos
   • Proxies  

🏭 Software Architecture Videos
   • Software Architecture  

🛰 Network Engineering Videos
   • Network Engineering  

💾 Database Engineering Videos
   • Database Engineering  

🐍 Python Videos
   • Python by Example  

🔆 Javascript Videos
   • Javascript by Example  

Support me on PayPal
https://bit.ly/33ENps4


Stay Awesome,
Hussein