Professional-Data-Engineer Google Professional Data Engineer Exam Free Practice Exam Questions (2025 Updated)

When using Cloud Dataproc clusters, configure your browser to use the SOCKS proxy. The SOCKS proxy routes data intended for the Cloud Dataproc cluster through an SSH tunnel.

Partitioned tables include a pseudo column named _PARTITIONTIME that contains a date-based timestamp for data loaded into the table. To limit a query to particular partitions (such as Jan 1st and 2nd of 2017), use a clause similar to this:

WHERE _PARTITIONTIME BETWEEN TIMESTAMP('2017-01-01') AND TIMESTAMP('2017-01-02')

Using the Drain option to stop your job tells the Dataflow service to finish your job in its current state. Your job will immediately stop ingesting new data from input sources, but the Dataflow

service will preserve any existing resources (such as worker instances) to finish processing and writing any buffered data in your pipeline.

An authorized view allows you to share query results with particular users and groups without giving them read access to the underlying tables. Authorized views can only be created in a dataset that does not contain the tables queried by the view.

When you create an authorized view, you use the view's SQL query to restrict access to only the rows and columns you want the users to see.

Cloud DLP is a service that helps you discover, classify, and protect your sensitive data. It supports various de-identification techniques, such as masking, redaction, tokenization, and encryption. Format-preserving encryption (FPE) with FFX is a technique that encrypts sensitive data while preserving its original format and length. This allows you to join the encrypted data on the same field without revealing the actual values. FPE with FFX also supports partial encryption, which means you can encrypt only a portion of the data, such as the domain name of an email address. By using Cloud DLP to de-identify the email field with FPE with FFX, you can ensure that the analysts can join the booking and user profile data on the email field without accessing the PII. You can create a pipeline to de-identify the email field by usingrecordTransformations in Cloud DLP, which allows you to specify the fields and the de-identification transformations to apply to them. You can then load the de-identified data into a BigQuery table for analysis. References:

De-identify sensitive data | Cloud Data Loss Prevention Documentation

Format-preserving encryption with FFX | Cloud Data Loss Prevention Documentation

De-identify and re-identify data with the Cloud DLP API

De-identify data in a pipeline

To ensure that all employees can search and find tables with GDPR tags while restricting data access to sensitive tables only to the HR group, follow these steps:

Data Catalog Tag Template:

Use Data Catalog to create a tag template named "gdpr" with a boolean field "has sensitive data". Set the visibility to public so all employees can see the tags.

Roles and Permissions:

Assign the datacatalog.tagTemplateViewer role to the all employees group. This role allows users to view the tags and search for tables based on the "has sensitive data" field.

Assign the bigquery.dataViewer role to the HR group specifically on tables that contain sensitive data. This ensures only HR can access the actual data in these tables.

Steps to Implement:

Create the GDPR Tag Template:

Define the tag template in Data Catalog with the necessary fields and set visibility to public.

Assign Roles:

Grant the datacatalog.tagTemplateViewer role to the all employees group for visibility into the tags.

Grant the bigquery.dataViewer role to the HR group on tables marked as having sensitive data.

Reference Links:

Data Catalog Documentation

Managing Access Control in BigQuery

IAM Roles in Data Catalog

Comprehensive and Detailed Explanation:

This question describes a classic, scalable streaming analytics architecture on Google Cloud.

Option C is the correct answer as it combines the best-in-class managed services for each part of the pipeline.

Pub/Sub is a fully managed, highly scalable messaging service perfect for ingesting fluctuating volumes of streaming IoT data.

Dataflow is a fully managed, serverless service for stream and batch processing that automatically scales resources up and down to handle fluctuating data volumes.

BigQuery is a serverless, highly scalable data warehouse optimized for analytics.

Cloud KMS is the standard Google Cloud service for creating and managing your own cryptographic keys, which are then used to enable Customer-Managed Encryption Keys (CMEK) across services like Pub/Sub, Dataflow, and BigQuery, satisfying the security requirement.

Option A is incorrect because it specifies using a Google-managed encryption key, which violates the requirement for encryption with "your own key."

Option B is incorrect because Cloud SQL is a relational database (OLTP), not an analytical data warehouse (OLAP), making it unsuitable for storing and analyzing large volumes of streaming data. BigQuery is the appropriate choice.

Option D is incorrect because Dataproc is a managed Hadoop/Spark service. While powerful, it is less "managed" and serverless than Dataflow, as you still need to provision and manage clusters. For a fully elastic and managed solution, Dataflow is the preferred choice.

Reference (Google Cloud Documentation Concepts):

This architecture is a canonical pattern for streaming analytics on Google Cloud. The "Streaming analytics" solution guide frequently highlights the Pub/Sub -> Dataflow -> BigQuery pattern. Each of these services supports Customer-Managed Encryption Keys (CMEK) using keys from Cloud KMS, ensuring data is protected at rest with customer-controlled keys, which aligns with the principle of customer control over data security.

To provide a cost-effective storage and processing solution that allows data scientists to explore data similarly to using the on-premises HDFS cluster with SQL on the Hive query engine, deploying a Dataproc cluster is the best choice. Here’s why:

Compatibility with Hive:

Dataprocis a fully managed Apache Spark and Hadoop service that provides native support for Hive, making it easy for data scientists to run SQL queries on the data as they would in an on-premises Hadoop environment.

This ensures that the transition to Google Cloud is smooth, with minimal changes required in the workflow.

Cost-Effective Storage:

Storing the ORC files inCloud Storageis cost-effective and scalable, providing a reliable and durable storage solution that integrates seamlessly with Dataproc.

Cloud Storage allows you to store large datasets at a lower cost compared to other storage options.

Hive Integration:

Dataproc supports running Hive directly, which is essential for data scientists familiar with SQL on the Hive query engine.

This setup enables the use of existing Hive queries and scripts without significant modifications.

Steps to Implement:

Copy ORC Files to Cloud Storage:

Transfer the ORC files from the on-premises HDFS cluster to Cloud Storage, ensuring they are organized in a similar directory structure.

Deploy Dataproc Cluster:

Set up a Dataproc cluster configured to run Hive. Ensure that the cluster has access to the ORC files stored in Cloud Storage.

Configure Hive:

Configure Hive on Dataproc to read from the ORC files in Cloud Storage. This can be done by setting up external tables in Hive that point to the Cloud Storage location.

Provide Access to Data Scientists:

Grant the data scientist team access to the Dataproc cluster and the necessary permissions to interact with the Hive tables.

Reference Links:

Dataproc Documentation

Hive on Dataproc

Google Cloud Storage Documentation

A hopping window is a type of sliding window that advances by a fixed period of time, producing overlapping windows. This is suitable for the scenario where the system needs to aggregate data for the last 30 seconds, every 2 seconds, and provide real-time updates. A Dataflow pipeline can implement the hopping window logic using Apache Beam, and process both streaming and batch data sources. Memorystore is a low-latency, in-memory data store that can serve the aggregated data to the visualization layer. BigQuery is not a good choice for this scenario, as it is not optimized for low-latency queries and frequent updates.