Databricks MigrationRoomClickHouse Workshops

Governance boundaries

Identify which principal and policy plane controls each workshop data path.

There is no single identity spanning the full hybrid architecture. Review each path and the handoff between them.

Identity and control matrix

PathExecuting identityMetadata/control planeData authorizationWrite authority in this workshopEvidence and owner
Databricks source MCPRead-only Databricks runtime principal used by MigrationRoomUnity Catalog catalog/schema/table grantsDatabricks SQL warehouse executes authorized readsNone; runtime needs USE CATALOG, USE SCHEMA, and SELECT onlyMCP read test and source baseline; participant/source owner
Zero-copy Unity in ClickHouseWorkshop-scoped service principal or OAuth identity configured for the catalog attachmentUnity Catalog exposes namespaces, tables, and snapshotsCredential vending and/or scoped object-store credentials permit file readsNone through this lab pathSHOW TABLES, count/hash, redacted plan; participant and Unity/storage owners
Shared REST catalogInstructor-issued read-only fixture identityIceberg REST service exposes the fixture namespace and table metadataFixture's vending/storage policy permits ClickHouse Cloud to read objectsNonePublished invariant plus participant count/hash; instructor fixture owner
Native ClickHouse copyMigration writer during copy; participant ClickHouse user during queriesClickHouse databases, tables, grants, quotas, and settingsClickHouse RBAC controls copied rows on ClickHouse storageMigration and lab DDL write only to the participant targetValidation/invariant evidence and ClickHouse grants; participant/target owner

Handoff at copy time

When MigrationRoom reads Databricks and writes native tables, the resulting copy crosses a governance boundary. Unity continues to govern the source, but it does not automatically govern ClickHouse's copy. ClickHouse RBAC, retention, deletion, encryption, and audit controls apply to that copy. Policy translation, row filters, masks, and continuous CDC are not automated by this workshop.

Zero-copy access is different: Unity or the REST catalog continues to control catalog visibility and storage access for its registered data, while ClickHouse also controls who may use the attached database and execute the query. Describe this as composed controls, not “the same governance everywhere.”

Least privilege and evidence safety

  • Separate the provisioning principal from the read-only MigrationRoom runtime principal.
  • Use short-lived, workshop-scoped catalog credentials and revoke them after the session.
  • Grant the shared REST identity read-only access to a stable fixture, not administrative access.
  • Scope ClickHouse users to participant databases and remove temporary grants at teardown.
  • Store only hostnames, identifier suffixes, result hashes, row counts, and redacted errors in evidence. Tokens, passwords, full authorization headers, and signed URLs fail review.

Failure ownership

Route failures by layer: Databricks grants/warehouse to the source owner; Unity metadata or credential vending to the metastore owner; object reads to the storage owner; REST reachability/fixture content to the instructor; and native grants or lifecycle to the ClickHouse owner. This prevents broadening privileges merely to make a lab step pass.

ในหน้านี้

TH