Skip to content

[otap-df-pdata/common] variable size protobuf placeholder#2589

Merged
jmacd merged 4 commits intoopen-telemetry:mainfrom
jmacd:jmacd/configurable-placeholder-size
Apr 9, 2026
Merged

[otap-df-pdata/common] variable size protobuf placeholder#2589
jmacd merged 4 commits intoopen-telemetry:mainfrom
jmacd:jmacd/configurable-placeholder-size

Conversation

@jmacd
Copy link
Copy Markdown
Contributor

@jmacd jmacd commented Apr 7, 2026

Change Summary

Add an optional fourth argument to proto_encode_len_delimited_of_size! macro to specify the number of bytes for the length placeholder. The existing 3-argument form continues to default to 4 bytes.

Update all callsites in the telemetry crate's self_tracing encoder to use 2-byte placeholders.

Part of #1746

How are these changes tested?

Are there any user-facing changes?

No

jmacd and others added 2 commits April 7, 2026 21:02
Add an optional fourth argument to proto_encode_len_delimited_unknown_size!
macro to specify the number of bytes (1-4) for the length placeholder varint.
The existing 3-argument form continues to default to 4 bytes.

Use const generics for encode_len_placeholder<N> and patch_len_placeholder<N>
so the placeholder size is known at compile time, enabling the compiler to
inline fixed-size arrays and unroll patch loops. Invalid sizes (outside 1-4)
are caught at compile time via a const assertion.

Add named convenience macros:
- proto_encode_len_delimited_small!  (2-byte, up to 16 KB)
- proto_encode_len_delimited_large!  (4-byte, up to 256 MB)

Update all 24 macro callsites in the telemetry crate's self_tracing encoder
(128-4096 bytes), well within the 16 KB ceiling of a 2-byte varint, saving
2 bytes of overhead per nesting level.

The pdata crate's OTAP encoding callsites (~40) continue using the default
4-byte placeholders since they encode full OTLP requests that can be large.

Part of open-telemetry#1746

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jmacd jmacd requested a review from a team as a code owner April 7, 2026 22:14
@github-actions github-actions bot added the rust Pull requests that update Rust code label Apr 7, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 7, 2026

Codecov Report

❌ Patch coverage is 97.67442% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.38%. Comparing base (b28727c) to head (c00de6e).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2589   +/-   ##
=======================================
  Coverage   88.37%   88.38%           
=======================================
  Files         622      622           
  Lines      230058   230115   +57     
=======================================
+ Hits       203318   203382   +64     
+ Misses      26216    26209    -7     
  Partials      524      524           
Components Coverage Δ
otap-dataflow 90.21% <97.67%> (+<0.01%) ⬆️
query_abstraction 80.61% <ø> (ø)
query_engine 90.74% <ø> (ø)
otel-arrow-go 52.45% <ø> (ø)
quiver 92.27% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Member

@albertlockett albertlockett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - assuming you'll fix the clippies

As much as I like the mysterious nature of the name proto_encode_len_delimited_unknown_size, I'll concede that proto_encode_len_delimited_of_size seems much more respectable

@jmacd jmacd force-pushed the jmacd/configurable-placeholder-size branch from b65b1ec to 392e83d Compare April 8, 2026 17:49
@jmacd
Copy link
Copy Markdown
Contributor Author

jmacd commented Apr 8, 2026

To avoid a large blast radius, I left the old _unknown_size helper in place for now.

@jmacd jmacd added this pull request to the merge queue Apr 8, 2026
Merged via the queue into open-telemetry:main with commit 4a8d85d Apr 9, 2026
68 of 70 checks passed
@jmacd jmacd deleted the jmacd/configurable-placeholder-size branch April 9, 2026 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rust Pull requests that update Rust code

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants