Add the public admin SDK and extract shared admin types

[lquerel]

Change Summary

This PR adds a public Rust admin SDK for the OTAP Dataflow Engine. This first version of the SDK relies on our existing HTTP admin endpoints. However, the SDK's design has been made with future support for other admin interfaces, such as OpAMP, in mind.

Concretely, this PR:

• adds otap-df-admin-api as the public integration crate for admin access
• makes the SDK configurable for HTTP and HTTPS, including custom CA and mTLS usage
• exposes the supported public admin endpoints through resource-scoped client methods
• introduces otap-df-admin-types as the shared schema crate and rewires otap-df-admin to depend on it directly
• updates validation to consume the new SDK
• expands the admin SDK documentation for integrators, including TLS, FIPS-oriented feature guidance, endpoint descriptions, examples, and future live reconfiguration direction.

What issue does this PR close?

No specific issue.

How are these changes tested?

These changes were validated with:

• cargo xtask check

Are there any user-facing changes?

Yes.

External Rust integrators now have a dedicated public crate, otap-df-admin-api, for admin access. The SDK uses a builder plus resource-client API, documents HTTP/TLS/custom-CA/mTLS usage, and re-exports the config model so users only need one public crate.

There are no intended wire-format changes to the existing admin HTTP endpoints. The new otap-df-admin-types crate is internal-only and is not intended as a public dependency.

[codecov]

Codecov Report

❌ Patch coverage is 86.44068% with 200 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.38%. Comparing base (b28727c) to head (4a31e61).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2574      +/-   ##
==========================================
+ Coverage   88.37%   88.38%   +0.01%     
==========================================
  Files         622      631       +9     
  Lines      230058   231409    +1351     
==========================================
+ Hits       203318   204541    +1223     
- Misses      26216    26344     +128     
  Partials      524      524              

Components	Coverage Δ
otap-dataflow	90.20% <86.44%> (+<0.01%)	⬆️
query_abstraction	80.61% <ø> (ø)
query_engine	90.74% <ø> (ø)
otel-arrow-go	52.45% <ø> (ø)
quiver	92.27% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[jmacd]

This has me sort of imagining a CLI like df_enginectl --shutdown

[lalitb]

This is very helpful, and close to something we’re doing on our side for embedded/managed collector integrations. One thing I’m trying to clarify: the PR description says the SDK is designed with future support for other admin interfaces in mind, but AdminBackend is currently internal-only. Is the intent that alternate backends remain internal for now, or should there eventually be a public extension point for non-HTTP backends?

[lalitb]

This has me sort of imagining a CLI like df_enginectl --shutdown

Love this use-case. A small df_enginectl-style CLI feels like a very natural fit for this SDK. I also feel this is relevant for MCP / agentic tooling. The OpenTelemetry community is already discussing official MCP-based workflows for the Collector (agentic-workflow.md), so I can see this SDK being useful as a foundation for external MCP servers on top of the engine admin surface.

[jmacd]

Please take @lalitb's feedback and please file issues for things that need to be resolved later.

[lquerel]

This is very helpful, and close to something we’re doing on our side for embedded/managed collector integrations. One thing I’m trying to clarify: the PR description says the SDK is designed with future support for other admin interfaces in mind, but AdminBackend is currently internal-only. Is the intent that alternate backends remain internal for now, or should there eventually be a public extension point for non-HTTP backends?

The use case I have in mind is to support OpAMP between the SDK and our engine once our engine exposes an OpAMP interface.