Databricks to ClickHouse type map
Map the demo workload's eight Databricks augmentations by semantics rather than spelling.
The table lists every augmentation created by MigrationRoom's current Databricks sample workload. These are design prompts, not automatic one-to-one conversions.
| Databricks source | ClickHouse candidates | Required decision and validation |
|---|---|---|
orders.o_metadata VARIANT | JSON, or typed hot-path columns plus retained JSON | Keep dynamic paths in JSON; type frequently filtered paths. Validate missing, null, numeric, boolean, array, and nested-object behavior. |
lineitem.l_shipping_events ARRAY<STRUCT<status STRING, event_ts TIMESTAMP, location STRING>> | Nested(...) or Array(Tuple(...)) | Choose from element alignment and query semantics. Validate explode/LATERAL VIEW rewrites with ARRAY JOIN or arrayJoin. |
lineitem.l_attributes MAP<STRING, STRING> | Map(String, String) | Check missing-key semantics and map explosion; ClickHouse missing-key lookup can return the value type's default rather than NULL. |
lineitem.l_committed_at TIMESTAMP | DateTime64(6, 'UTC') | State source session/timezone assumptions and validate an instant across timezone settings. |
lineitem.l_committed_at_ntz TIMESTAMP_NTZ | DateTime64(6) | Preserve wall-clock semantics separately from an absolute instant; do not silently attach UTC. |
lineitem CLUSTER BY (l_shipdate, l_suppkey) | Workload-derived MergeTree ORDER BY | Do not copy the declaration mechanically. Use selective filters and measured cardinality, then validate with EXPLAIN indexes = 1. |
lineitem deletion vectors and Delta history | Application mutation design, ReplacingMergeTree, ClickPipes/CDC, or defer | There is no copied equivalent for Delta time travel. Define delete/update and audit requirements explicitly before choosing a replacement. |
orders.o_orderyear GENERATED ALWAYS AS (year(o_orderdate)) | MATERIALIZED, ALIAS, or normal column | Decide whether the value is stored, computed on read, or supplied. Validate insert behavior and staged-copy column handling. |
daily_order_summary Databricks materialized view | Incremental MV to AggregatingMergeTree, refreshable MV, or normal table | Choose from freshness semantics. For incremental MV, use state/merge functions and separate historical backfill. The source object is optional when serverless requirements are not met. |
TIMESTAMP and TIMESTAMP_NTZ form one workload augmentation but require two distinct
target choices, so both are shown separately.
Base scalar guidance
| Source family | Typical target | Guardrail |
|---|---|---|
DECIMAL(p,s) | Decimal(p,s) with supported precision | Preserve exact money arithmetic; do not substitute floating point silently. |
DATE | Date or Date32 | Choose from range and validate boundary dates. |
Repeated descriptive STRING | LowCardinality(String) candidate | Measure cardinality first; it is not a semantic replacement for arbitrary strings. |
| Integral keys | Signed or unsigned integer sized to observed and future range | Preserve negative-value and overflow semantics rather than choosing by sample alone. |
Physical and operational semantics
Type parity is not migration parity. Also decide ordering, partitioning, nullability, mutation behavior, column order for staged imports, and how existing rows enter any insert-triggered materialized view. MigrationRoom's reference DDL is a review aid, not an unquestionable result; the learner's workload and invariants decide acceptance.