Skip to content

[Feature Request]: Support reading from BigTable Continuous Materialized Views in BigtableIO #38053

@psud

Description

@psud

What would you like to happen?

The BigTable connector via BigtableIO.Read currently only supports reading from BigTable tables via withTableId(). There is no way to read from the new continuous materialized view tables (CMV), which are a separate resource type (materializedViews/ vs tables/).

The underlying google-cloud-bigtable Java client already supports CMVs — Query.create() and sampleRowKeys() both accept materialized view resource names as of v2.55.0. However, BigtableIO.Read has no option to specify a materialized view name, so this capability is not accessible from Beam pipelines.

Use case:
We use BigTable CMVs to maintain pre-aggregated index views over our data. We need to read from these views in Dataflow pipelines for downstream processing. Currently we can only access CMVs via the REST API or the Java client directly, but not through BigtableIO.

Expected behavior:
A withMaterializedViewName(String) method (or similar) on BigtableIO.Read, mutually exclusive with withTableId(), that routes the read through the existing client support for materialized views.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions