Could you not have gotten all this information by changing the Envoy log format to add the required fields? In your blog post, you show the default log format, and suggest that there was no way whatsoever to get this data from the logs. But according to the documentation, Envoy has rich support for many additional fields including detailed latency statistics.
I wonder if you added complexity to the architecture when a simple log format change would have sufficed.
To my knowledge, Envoy does not provide this type of information for TCP proxying. I wanted request/response latency, but the available metrics are limited to connection-level information
Why aren’t you using HTTP proxying if the underlying protocol is HTTP?
TCP proxying, in my experience, is typically only used for routing TLS or other non-HTTP TCP requests. In the former case, the proxy should not be able to observe the requests and responses.
Judging by the font embedded in the SVG file, it looks like it was created using Excalidraw. It's a great tool if you're looking for that hand-drawn aesthetic!
Great post, thanks for sharing! I appreciate all the details, including context for the problem, the complete setup, and the results. Bookmarked for future reference.
Could you not have gotten all this information by changing the Envoy log format to add the required fields? In your blog post, you show the default log format, and suggest that there was no way whatsoever to get this data from the logs. But according to the documentation, Envoy has rich support for many additional fields including detailed latency statistics.
I wonder if you added complexity to the architecture when a simple log format change would have sufficed.
To my knowledge, Envoy does not provide this type of information for TCP proxying. I wanted request/response latency, but the available metrics are limited to connection-level information
Why aren’t you using HTTP proxying if the underlying protocol is HTTP?
TCP proxying, in my experience, is typically only used for routing TLS or other non-HTTP TCP requests. In the former case, the proxy should not be able to observe the requests and responses.
We currently only provide TCP load balancers, so HTTP-level proxying was not an option. Since the customer was not using TLS, this solution worked.
Great article! What is the name of tool used for writing the block diagram?
Judging by the font embedded in the SVG file, it looks like it was created using Excalidraw. It's a great tool if you're looking for that hand-drawn aesthetic!
Yes, I used Excalidraw!
Great post, thanks for sharing! I appreciate all the details, including context for the problem, the complete setup, and the results. Bookmarked for future reference.
I was also on the verge of installing OBI to troubleshoot networking issue on one of our clusters. So maybe next time there’s a similar incident