A data pipeline uses Structured Streaming to ingest data from kafka to Delta Lake. Data is being stored in a bronze table, and includes the Kafka_generated timesamp, key, and value. Three months after the pipeline is deployed the data engineering team has noticed some latency issued during certain times of the day.
A senior data engineer updates the Delta Table's schema and ingestion logic to include the current timestamp (as recoded by Apache Spark) as well the Kafka topic and partition. The team plans to use the additional metadata fields to diagnose the transient processing delays.
Which limitation will the team face while diagnosing this problem?

• A. New fields cannot be added to a production Delta table.
• B. Spark cannot capture the topic partition fields from the kafka source.
• C. Updating the table schema will invalidate the Delta transaction log metadata.
• D. Updating the table schema requires a default value provided for each file added.
• E. New fields will not be computed for historic records.

A data engineer is configuring a Databricks Asset Bundle to deploy a job with granular permissions.
The requirements are:
- Grant the data-engineers group CAN_MANAGE access to the job.
- Ensure the auditors' group can view the job but not modify/run it.
- Avoid granting unintended permissions to other users/groups.
How should the data engineer deploy the job while meeting the requirements?

• A. resources:
  jobs:
  my-job:
  name: data-pipeline
  tasks: [...]
  job_clusters: [...]
  permissions:
  - group_name: data-engineers
  level: CAN_MANAGE
  - group_name: auditors
  level: CAN_VIEW
• B. resources:
  jobs:
  my-job:
  name: data-pipeline
  tasks: [...]
  job: [...]
  permissions:
  - group_name: data-engineers
  level: CAN_MANAGE
  permissions:
  - group_name: auditors
  level: CAN_VIEW
• C. permissions:
  - group_name: data-engineers
  level: CAN_MANAGE
  - group_name: auditors
  level: CAN_VIEW
  resources:
  jobs:
  my-job:
  name: data-pipeline
  tasks: [...]
  job_clusters: [...]
• D. resources:
  jobs:
  my-job:
  name: data-pipeline
  tasks: [...]
  job_clusters: [...]
  permissions:
  - group_name: data-engineers
  level: CAN_MANAGE
  - group_name: auditors
  level: CAN_VIEW
  - group_name: admin-team
  level: IS_OWNER

A junior data engineer on your team has implemented the following code block.

The view new_events contains a batch of records with the same schema as the events Delta table. The event_id field serves as a unique key for this table.
When this query is executed, what will happen with new records that have the same event_id as an existing record?

• A. They are ignored.
• B. They are deleted.
• C. They are updated.
• D. They are inserted.
• E. They are merged.

A data engineer needs to implement column masking for a sensitive column in a Unity Catalog- managed table. The masking logic must dynamically check if users belong to specific groups defined in a separate table (group_access) that maps groups to allowed departments. Which approach should the engineer use to efficiently enforce this requirement?

• A. Use a row filter to restrict access based on the user's group.
• B. Create a UDF that hardcodes allowed groups and apply it as a column mask.
• C. Create a view without selecting the sensitive column.
• D. Apply a column mask that references the group_access mapping table in its UDF.

A production workload incrementally applies updates from an external Change Data Capture feed to a Delta Lake table as an always-on Structured Stream job. When data was initially migrated for this table, OPTIMIZE was executed and most data files were resized to 1 GB. Auto Optimize and Auto Compaction were both turned on for the streaming production job. Recent review of data files shows that most data files are under 64 MB, although each partition in the table contains at least 1 GB of data and the total table size is over 10 TB.
Which of the following likely explains these smaller file sizes?

• A. Databricks has autotuned to a smaller target file size based on the overall size of data in the table
• B. Databricks has autotuned to a smaller target file size to reduce duration of MERGE operations
• C. Z-order indices calculated on the table are preventing file compaction C Bloom filler indices calculated on the table are preventing file compaction
• D. Databricks has autotuned to a smaller target file size based on the amount of data in each partition