Скачать или смотреть Can You Send Pub/Sub Messages to Stackdriver Logs? Here's What You Need to Know

Discover the possibilities of integrating Google Cloud Pub/Sub with Stackdriver logs. Learn about the limitations and straightforward solutions to log your Pub/Sub messages efficiently.
---
This video is based on the question https://stackoverflow.com/q/70262501/ asked by the user 'red888' ( https://stackoverflow.com/u/1028270/ ) and on the answer https://stackoverflow.com/a/70263900/ provided by the user 'guillaume blaquiere' ( https://stackoverflow.com/u/11372593/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: Is it possible to send pubsub messages to stack driver logs?

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Can You Send Pub/Sub Messages to Stackdriver Logs? Here's What You Need to Know

When working with Google Cloud Platform (GCP), one common question that arises is whether you can directly send Pub/Sub messages to Stackdriver Logging (now known as Cloud Logging). It's an important consideration for those looking to streamline their logging processes and efficiently manage messages from Pub/Sub. Today, we will explore this topic and provide a clear solution that could help you achieve your logging goals without unnecessary complications.

The Challenge

Many developers have learned how to send logs from Stackdriver to Pub/Sub using log sinks. However, the opposite—sending Pub/Sub messages directly to Stackdriver Logs—poses a challenge. The core question boils down to:

Is it possible to configure Stackdriver Logs to subscribe to a Pub/Sub topic and log the messages directly?

This would ideally eliminate the need to develop a custom application that manages the reading and logging of Pub/Sub messages. Unfortunately, the answer to this query is somewhat disappointing.

The Solution: Creating a Custom Application

Direct Integration Not Available

As it stands, there is no built-in feature to sink Pub/Sub messages directly into Cloud Logging. You cannot simply set up a subscription that redirects messages from a Pub/Sub topic directly into your Stackdriver Logs. Instead, you will need to take a different approach.

Developing a Small Custom Application

While it may seem tedious, creating a minimal custom application is the best option available. Here's how you can develop a solution in just a few lines of code:

Options for Implementing the Custom Application

Cloud Functions: This is the preferred avenue, especially if you are dealing with a high volume of messages. Be sure to use the runtime V2 to minimize costs while still processing messages efficiently.

Cloud Run: This is another reliable option for handling Pub/Sub messages, allowing you to create a containerized application that processes and logs messages.

Implementation Steps

To get started, here’s a high-level outline of what your custom application would do:

Set Up Subscription: Create a subscription on your Pub/Sub topic.

Receive Messages: Use the Google Cloud SDK or appropriate libraries in your programming language to receive messages.

Log Messages: Log each message received to Cloud Logging using the available logging library (e.g., google-cloud-logging in Python).

Acknowledge Messages: Once logged, acknowledge the message in Pub/Sub to prevent it from being re-delivered.

Here is a basic example in Python:

[[See Video to Reveal this Text or Code Snippet]]

Key Takeaways

No Direct Configuration: You cannot configure Stackdriver Logs to subscribe to Pub/Sub topics directly.

Custom Application Required: To log Pub/Sub messages, you need a small custom application using Cloud Functions or Cloud Run.

Simple Code Implementation: With fewer than ten lines of code, you can effectively log your messages into Cloud Logging.

Conclusion

While Google Cloud does not currently support direct sinking of Pub/Sub messages to Cloud Logging, creating a lightweight custom application is a feasible workaround to achieve this goal. By using Cloud Functions or Cloud Run, you can efficiently log your Pub/Sub messages, ensuring that no important data gets lost in the process. Embrace this solution and take control of your logging infrastructure today!