Skip to content

Fix raw byte array formatting in logs for json.RawMessage fields #3674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented May 19, 2025

Problem

When CNS logs structures containing json.RawMessage fields (such as OrchestratorContext), it uses Go's %+v format specifier which outputs raw bytes as integer arrays. This makes the logs very difficult to read and understand, as shown in this example:

OrchestratorContext:[123 34 80 111 100 78 97 109 101 34 58 34 122 116 117 110 110 101 108 45 57 122 54 55 100 34 44 34 80 111 100 78 97 109 101 115 112 97 99 101 34 58 34 105 115 116 105 111 45 115 121 115 116 101 109 34 125]

These bytes actually represent the JSON string:

{"PodName":"ztunnel-9z67d","PodNamespace":"istio-system"}

Solution

This PR introduces a new toJSONString helper function in both the base logger (log/logger.go) and CNS logger (cns/logger/cnslogger.go) that properly formats objects containing json.RawMessage fields.

The approach:

  1. Use json.Marshal instead of %+v to convert objects to strings for logging
  2. This ensures json.RawMessage fields are properly formatted as JSON
  3. Fall back to the original %+v formatting if JSON marshaling fails
  4. Update all the structured logging methods to use this new helper function

Example

Before:

Code:FailedToAllocateIpConfig, {DesiredIPAddresses:[] PodInterfaceID:c34a4c61-eth0 InfraContainerID:c34a4c61b6b2173f7cb62945a3e9f00ea33b99e4aa6b283e714b095b7875a87b OrchestratorContext:[123 34 80 111 100 78 97 109 101 34 58 34 122 116 117 110 110 101 108 45 57 122 54 55 100 34 44 34 80 111 100 78 97 109 101 115 112 97 99 101 34 58 34 105 115 116 105 111 45 115 121 115 116 101 109 34 125] ...}

After:

Code:FailedToAllocateIpConfig, {"DesiredIPAddresses":[],"PodInterfaceID":"c34a4c61-eth0","InfraContainerID":"c34a4c61b6b2173f7cb62945a3e9f00ea33b99e4aa6b283e714b095b7875a87b","OrchestratorContext":{"PodName":"ztunnel-9z67d","PodNamespace":"istio-system"} ...}

Fixes #3673.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • cdn.fwupd.org
    • Triggering command: /usr/bin/fwupdmgr refresh (dns block)

If you need me to access, download, or install something from one of these locations, you can either:


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: rbtr <2940321+rbtr@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] don't log raw bytes Fix raw byte array formatting in logs for json.RawMessage fields May 19, 2025
@Copilot Copilot AI requested a review from rbtr May 19, 2025 20:52
Copilot finished work on behalf of rbtr May 19, 2025 20:52
@rbtr rbtr marked this pull request as ready for review May 19, 2025 21:11
@Copilot Copilot AI review requested due to automatic review settings May 19, 2025 21:11
@rbtr rbtr requested review from a team as code owners May 19, 2025 21:11
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves log readability by replacing the default raw byte array formatting with a proper JSON string representation for logs containing json.RawMessage fields. The key changes include:

  • Introducing a new toJSONString helper function in both log/logger.go and cns/logger/cnslogger.go.
  • Updating structured logging methods to use toJSONString for formatting.
  • Falling back to the original %+v formatting if JSON marshaling fails.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
log/logger.go Added toJSONString helper and updated Request, Response, and ResponseEx formatting.
cns/logger/cnslogger.go Added toJSONString helper and updated Request, Response, and ResponseEx logging.

Copy link
Contributor

@rbtr rbtr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deduplicate and add tests

Co-authored-by: rbtr <2940321+rbtr@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 19, 2025

deduplicate and add tests

I've addressed this feedback in commit 3da0544:

  1. Deduplicated the JSON formatting code by:

    • Moving the function to the base logger package
    • Making it public as ToJSONString
    • Using it from CNS logger
  2. Added comprehensive tests for the new functionality:

    • Basic types and structures
    • Cases with json.RawMessage fields
    • Fallback behavior for non-JSON-marshalable objects

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • cdn.fwupd.org
    • Triggering command: /usr/bin/fwupdmgr refresh (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of rbtr May 19, 2025 21:36
Co-authored-by: rbtr <2940321+rbtr@users.noreply.github.com>
Copilot finished work on behalf of rbtr May 19, 2025 22:09
rbtr
rbtr previously requested changes May 19, 2025
Copy link
Contributor

@rbtr rbtr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is still a lint issue

log/logger.go:210: File is not `gci`-ed with --skip-generated -s standard -s default (gci)

Co-authored-by: rbtr <2940321+rbtr@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 19, 2025

there is still a lint issue

log/logger.go:210: File is not `gci`-ed with --skip-generated -s standard -s default (gci)

I fixed the lint issue in log/logger.go by running gci with the --skip-generated -s standard -s default flags. The issue was caused by an extra whitespace at line 210. Fixed in commit 3a8f2ed.

Copilot finished work on behalf of rbtr May 19, 2025 22:50
@rbtr
Copy link
Contributor

rbtr commented May 19, 2025

/azp run Azure Container Networking PR

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@rbtr rbtr enabled auto-merge May 21, 2025 16:14
@rbtr rbtr dismissed their stale review May 21, 2025 16:14

unblocking, can't approve

Copy link

github-actions bot commented Jun 5, 2025

This pull request is stale because it has been open for 2 weeks with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale due to inactivity. label Jun 5, 2025
@rbtr rbtr added exempt-stale Keep this fresh and removed stale Stale due to inactivity. labels Jun 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exempt-stale Keep this fresh
Projects
None yet
Development

Successfully merging this pull request may close these issues.

don't log raw bytes
3 participants