Databricks Databricks-Certified-Professional-Data-Engineer Exam Prep Guide: Prep guide for the Databricks-Certified-Professional-Data-Engineer Exam
2023 New Preparation Guide of Databricks Databricks-Certified-Professional-Data-Engineer Exam
NEW QUESTION # 34
A Delta Live Table pipeline includes two datasets defined using STREAMING LIVE TABLE.
Three datasets are defined against Delta Lake table sources using LIVE TABLE . The table is configured to
run in Development mode using the Triggered Pipeline Mode.
Assuming previously unprocessed data exists and all definitions are valid, what is the expected outcome after
clicking Start to update the pipeline?
- A. All datasets will be updated once and the pipeline will shut down. The compute resources will be
terminated - B. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will
persist after the pipeline is stopped to allow for additional testing - C. All datasets will be updated once and the pipeline will shut down. The compute resources will persist to
allow for additional testing - D. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will
be deployed for the update and terminated when the pipeline is stopped - E. All datasets will be updated continuously and the pipeline will not shut down. The compute resources
will persist with the pipeline
Answer: C
NEW QUESTION # 35
Data science team has requested they are missing a column in the table called average price, this can be calculated using units sold and sales amt, which of the following SQL statements allow you to reload the data with additional column
- A. OVERWRITE sales AS SELECT *, salesAmt/unitsSold as avgPrice FROM sales
- B. 1.CREATE OR REPLACE TABLE sales
2.AS SELECT *, salesAmt/unitsSold as avgPrice FROM sales - C. 1.INSERT OVERWRITE sales
2.SELECT *, salesAmt/unitsSold as avgPrice FROM sales - D. COPY INTO SALES AS SELECT *, salesAmt/unitsSold as avgPrice FROM sales
- E. MERGE INTO sales USING (SELECT *, salesAmt/unitsSold as avgPrice FROM sales)
Answer: B
Explanation:
Explanation
1.CREATE OR REPLACE TABLE sales
2.AS SELECT *, salesAmt/unitsSold as avgPrice FROM sales
The main difference between INSERT OVERWRITE and CREATE OR REPLACE TABLE(CRAS) is that CRAS can modify the schema of the table, i.e it can add new columns or change data types of existing columns. By default INSERT OVERWRITE only overwrites the data.
INSERT OVERWRITE can also be used to overwrite schema, only when
spark.databricks.delta.schema.autoMerge.enabled is set true if this option is not enabled and if there is a schema mismatch command will fail.
NEW QUESTION # 36
You noticed that a team member started using an all-purpose cluster to develop a notebook and used the same all-purpose cluster to set up a job that can run every 30 mins so they can update un-derlying tables which are used in a dashboard. What would you recommend for reducing the overall cost of this approach?
- A. Change the cluster mode from all-purpose to single-mode
- B. Reduce the size of the cluster
- C. Reduce the number of nodes and enable auto scale
- D. Change the cluster all-purpose to job cluster when scheduling the job
- E. Enable auto termination after 30 mins
Answer: D
Explanation:
Explanation
While using an all-purpose cluster is ok during development but anytime you don't need to interact with a notebook, especially for a scheduled job it is less expensive to use a job cluster. Using an all-purpose cluster can be twice as expensive as a job cluster.
Please note: The compute cost you pay the cloud provider for the same cluster type and size be-tween an all-purpose cluster and job cluster is the same the only difference is the DBU cost.
The total cost of cluster = Total cost of VM compute(Azure or AWS or GCP) + Cost per DBU The per DBU cost varies between all-purpose and Job Cluster Here is the recent cost estimate from AWS between Jobs Cluster and all-purpose Cluster, for jobs compute its
$0.15 cents per DBU v$0.55 cents per DBU for all-purpose
Graphical user interface Description automatically generated
How do I check how much the DBU cost for my cluster?
When you click on an exister cluster or when you look at the cluster details you will see this in the top right corner Graphical user interface, text, application, email Description automatically generated
NEW QUESTION # 37
A data engineer has created a Delta table as part of a data pipeline. Downstream data analysts now need
SELECT permission on the Delta table.
Assuming the data engineer is the Delta table owner, which part of the Databricks Lakehouse Plat-form can
the data engineer use to grant the data analysts the appropriate access?
- A. Data Explorer
- B. Databricks Filesystem
- C. Repos
- D. Jobs
B Dashboards
Answer: C
NEW QUESTION # 38
You have written a notebook to generate a summary data set for reporting, Notebook was scheduled using the job cluster, but you realized it takes 8 minutes to start the cluster, what feature can be used to start the cluster in a timely fashion so your job can run immediatley?
- A. Disable auto termination so the cluster is always running
- B. Use Databricks Premium edition instead of Databricks standard edition
- C. Use the Databricks cluster pools feature to reduce the startup time
- D. Pin the cluster in the cluster UI page so it is always available to the jobs
- E. Setup an additional job to run ahead of the actual job so the cluster is running second job starts
Answer: C
Explanation:
Explanation
Cluster pools allow us to reserve VM's ahead of time, when a new job cluster is created VM are grabbed from the pool. Note: when the VM's are waiting to be used by the cluster only cost incurred is Azure. Databricks run time cost is only billed once VM is allocated to a cluster.
Here is a demo of how to setup a pool and follow some best practices,
Graphical user interface, text Description automatically generated
NEW QUESTION # 39
When using the complete mode to write stream data, how does it impact the target table?
- A. Entire stream waits for complete data to write
- B. Stream must complete to write the data
- C. Target table cannot be updated while stream is pending
- D. Delta commits transaction once the stream is stopped
- E. Target table is overwritten for each batch
Answer: E
Explanation:
Explanation
The answer is Target table is overwritten for each batch
Complete mode - The whole Result Table will be outputted to the sink after every trigger. This is supported for aggregation queries
NEW QUESTION # 40
You were asked to setup a new all-purpose cluster, but the cluster is unable to start which of the following steps do you need to take to identify the root cause of the issue and the reason why the cluster was unable to start?
- A. Data plane
- B. Check the cluster driver logs
- C. Workspace logs
- D. Storage account
- E. Check the cluster event logs
(Correct)
Answer: E
Explanation:
Explanation
Cluster event logs are very useful, to identify issues pertaining to cluster availability. Cluster may not start due to resource limitations or issues with the cloud providers.
Some of the common issues include a subnet for compute VM reaching its limits or exceeding the subscription or cloud account CPU quota limit.
Here is an example where the cluster did not start due to subscription reaching the quota limit on a certain type of cpu cores for a VM type.
Graphical user interface, text, application, email Description automatically generated
Click on event logs
Graphical user interface, text, application, email Description automatically generated
Click on the message to see the detailed error message on why the cluster did not start.
Graphical user interface, text, application, email Description automatically generated
NEW QUESTION # 41
An engineering manager uses a Databricks SQL query to monitor their team's progress on fixes related to
customer-reported bugs. The manager checks the results of the query every day, but they are manually
rerunning the query each day and waiting for the results.
Which of the following approaches can the manager use to ensure the results of the query are up-dated each
day?
- A. They can schedule the query to refresh every 1 day from the SQL endpoint's page in Databricks SQL
- B. They can schedule the query to refresh every 12 hours from the SQL endpoint's page in Databricks SQL
- C. They can schedule the query to refresh every 1 day from the query's page in Databricks SQL
- D. They can schedule the query to run every 1 day from the Jobs UI
- E. They can schedule the query to run every 12 hours from the Jobs UI
Answer: C
NEW QUESTION # 42
Which of the following SQL statements can replace a python variable, when the notebook is set in SQL mode
1.table_name = "sales"
2.schema_name = "bronze"
- A. spark.sql(f"SELECT * FROM {schem_name.table_name}")
- B. spark.sql(f"SELECT * FROM {schema_name}.{table_name}")
- C. spark.sql(f"SELECT * FROM f{schema_name.table_name}")
- D. spark.sql("SELECT * FROM schema_name.table_name")
- E. spark.sql(f"SELECT * FROM ${schema_name}.${table_name}")
Answer: B
Explanation:
Explanation
The answer is spark.sql(f"SELECT * FROM {schema_name}.{table_name}")
NEW QUESTION # 43
A dataset has been defined using Delta Live Tables and includes an expectations clause: CON-STRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') What is the expected behavior when a batch of data containing data that violates these constraints is processed?
- A. Records that violate the expectation cause the job to fail.
- B. Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
- C. Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
- D. Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
- E. Records that violate the expectation are added to the target dataset and flagged as in-valid in a field added to the target dataset.
Answer: D
Explanation:
Explanation
The answer is, Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
Delta live tables support three types of expectations to fix bad data in DLT pipelines Review below example code to examine these expectations, Diagram Description automatically generated with medium confidence
NEW QUESTION # 44
Two junior data engineers are authoring separate parts of a single data pipeline notebook. They are working on
separate Git branches so they can pair program on the same notebook simultaneously. A senior data engineer
experienced in Databricks suggests there is a better alternative for this type of collaboration.
Which of the following supports the senior data engineer's claim?
- A. Databricks Notebooks support the creation of interactive data visualizations
- B. Databricks Notebooks support the use of multiple languages in the same notebook
- C. Databricks Notebooks support real-time co-authoring on a single notebook
- D. Databricks Notebooks support automatic change-tracking and versioning
- E. Databricks Notebooks support commenting and notification comments
Answer: C
NEW QUESTION # 45
Which of the following python statement can be used to replace the schema name and table name in the query statement?
- A. 1.table_name = "sales"
2.schema_name = "bronze"
3.query = "select * from {schema_name}.{table_name}" - B. 1.table_name = "sales"
2.schema_name = "bronze"
3.query = f"select * from schema_name.table_name" - C. 1.table_name = "sales"
2.schema_name = "bronze"
3.query = f"select * from + schema_name +"."+table_name" - D. 1.table_name = "sales"
2.schema_name = "bronze"
3.query = f"select * from { schema_name}.{table_name}"
Answer: D
Explanation:
Explanation
Answer is
table_name = "sales"
query = f"select * from {schema_name}.{table_name}"
f strings can be used to format a string. f" This is string {python variable}"
https://realpython.com/python-f-strings/
NEW QUESTION # 46
Which one of the following is not a Databricks lakehouse object?
- A. Catalog
- B. Views
- C. Tables
- D. Functions
- E. Database/Schemas
- F. Stored Procedures
Answer: F
Explanation:
Explanation
The answer is, Stored Procedures.
Databricks lakehouse does not support stored procedures.
NEW QUESTION # 47
You have noticed the Data scientist team is using the notebook versioning feature with git integra-tion, you have recommended them to switch to using Databricks Repos, which of the below reasons could be the reason the why the team needs to switch to Databricks Repos.
- A. Databricks Repos allows merge and conflict resolution
- B. Databricks Repos has a built-in version control system
- C. Databricks Repos automatically saves changes
- D. Databricks Repos allows multiple users to make changes
- E. Databricks Repos allow you to add comments and select the changes you want to commit.
Answer: E
Explanation:
Explanation
The answer is Databricks Repos allow you to add comments and select the changes you want to commit.
NEW QUESTION # 48
You were asked to write python code to stop all running streams, which of the following command can be used to get a list of all active streams currently running so we can stop them, fill in the blank.
1.for s in _______________:
2. s.stop()
- A. getActiveStreams()
- B. Spark.getActiveStreams()
- C. spark.streams.active
- D. spark.streams.getActive
- E. activeStreams()
Answer: C
NEW QUESTION # 49
What is the main difference between the silver layer and the gold layer in medalion architecture?
- A. Silver may contain aggregated data
- B. Silver is a copy of bronze data
- C. Gold may contain aggregated data
- D. God is a copy of silver data
- E. Data quality checks are applied in gold
Answer: C
Explanation:
Explanation
Medallion Architecture - Databricks
Exam focus: Please review the below image and understand the role of each layer(bronze, silver, gold) in medallion architecture, you will see varying questions targeting each layer and its purpose.
Sorry I had to add the watermark some people in Udemy are copying my content.
A diagram of a house Description automatically generated with low confidence
NEW QUESTION # 50
A team member is leaving the team and he/she is currently the owner of the few tables, instead of transfering the ownership to a user you have decided to transfer the ownership to a group so in the future anyone in the group can manage the permissions rather than a single individual, which of the following commands help you accomplish this?
- A. GRANT OWNER On table_name to 'group'
- B. ALTER OWNER ON table_name to 'group'
- C. TRANSFER OWNER table_name to 'group'
- D. GRANT OWNER table_name to 'group'*
- E. ALTER TABLE table_name OWNER to 'group'
Answer: E
Explanation:
Explanation
The answer is ALTER TABLE table_name OWNER to 'group'
Assign owner to object
NEW QUESTION # 51
......
Databricks Certified Professional Data Engineer certification exam consists of 60 multiple-choice questions that need to be answered within 90 minutes. The questions are designed to test the candidate's knowledge of Databricks and its features, as well as their ability to apply this knowledge to real-world scenarios. To pass the exam, candidates need to achieve a minimum score of 70%. Databricks Certified Professional Data Engineer Exam certification is valid for two years and can be renewed by retaking the exam or by completing a set of continuing education activities. Obtaining the Databricks Certified Professional Data Engineer certification can help data professionals demonstrate their expertise in building and deploying data engineering solutions using Databricks, which can be a valuable asset in today's data-driven business landscape.
Latest Questions Databricks-Certified-Professional-Data-Engineer Guide to Prepare Free Practice Tests: https://www.actualvce.com/Databricks/Databricks-Certified-Professional-Data-Engineer-valid-vce-dumps.html