About This Edition: A clean, intermediate-level architectural guide covering all 19 Subsections and 75 Official Considerations of the Google Cloud Certified Professional Cloud DevOps Engineer curriculum, illustrated with Mermaid architecture blueprints and 150 custom-designed concept & architecture diagrams.
Chapter Plan at a Glance
Ch # | Chapter Title | Subsections | Considerations | Custom Diagrams | Core Focus |
|---|---|---|---|---|---|
00 | Overview | — | Mermaid | The Architectural Planes, plain-English guide to in-scope services, and the 10 Core Architectural Principles. | |
01 | 1.1, 1.2, 1.3, 1.4, 1.5 | 21 | 42 | Resource Hierarchy, Shared VPC vs. Peering, Organization Policies, Terraform remote state/drift detection, Config Connector, multi-environment GKE Fleets, and secure Cloud Workstations. | |
02 | 2.1, 2.2, 2.3, 2.4 | 14 | 28 | Artifact Registry, Cloud Build private pools, Cloud Deploy progressive delivery (Canary, Blue/Green), Secret Manager vs. KMS, Artifact Analysis CVE scanning, SLSA provenance, and Binary Authorization. | |
03 | 3.1, 3.2, 3.3 | 9 | 18 | SLI selection (Request/Window), SLO targets, Error Budget policy enforcement, toil elimination, MIG/GKE/Cloud Run autoscaling, traffic draining, canary rollbacks, Incident Command System (ICS), and blameless postmortems. | |
04 | 4.1, 4.2, 4.3, 4.4, 4.5 | 23 | 46 | Ops Agent, OpenTelemetry Collector, Google Cloud Managed Service for Prometheus (GMP), Log Router exclusion filters/sinks/redaction, Multi-Window Multi-Burn-Rate SLO alerts, Cloud Trace, and production debugging. | |
05 | 5.1, 5.2 | 8 | 16 | Continuous CPU/heap profiling with Cloud Profiler, Active Assist right-sizing/idle recommendations, CUDs vs. Spot VMs, GKE cost allocation, and observability ingestion cost optimization. | |
06 | All Sections | — | — | Side-by-side Look-Alike Tool Comparison Matrix, 5-Step Architecture Decision Checklist, and Top 25 Golden Rules. |
Detailed Table of Contents
Chapter 0: The PDOE Reference Stack & 10 Core Principles
- 0.1 The Master Architectural Map: All In-Scope Services of the PDOE Stack
- 0.2 The 10 Core Architectural Principles
Chapter 1: Bootstrapping & Maintaining Google Cloud Organizations: Hierarchy, IaC, Fleets & Workstations
- Subsection 1.1: Designing the overall resource hierarchy for an organization (Section 1: Bootstrapping and Maintaining a Google Cloud Organization)
- Consideration 1.1a: Organizing resources (e.g., application-centric, projects, folders) (Diagrams:
pdoe-1.1a-1-folder-project-taxonomy.jpg,pdoe-1.1a-2-separate-projects-beat.jpg) - Consideration 1.1b: Shared networking (e.g., Shared VPC, VPC Network Peering, Private Service Connect) (Diagrams:
pdoe-1.1b-1-shared-vpc-default.jpg,pdoe-1.1b-2-vpc-network-peering-full.jpg) - Consideration 1.1c: Multi-project monitoring and logging (Diagrams:
pdoe-1.1c-1-multi-project-monitoring.jpg,pdoe-1.1c-2-multi-project-logging.jpg) - Consideration 1.1d: Identity and Access Management (IAM) roles and organization-level policies (Diagrams:
pdoe-1.1d-1-iam-role-selection.jpg,pdoe-1.1d-2-iam-conditions.jpg) - Consideration 1.1e: Creating and managing service accounts (Diagrams:
pdoe-1.1e-1-one-service-account-per.jpg,pdoe-1.1e-2-impersonation-service.jpg) - Consideration 1.1f: Data residency (Diagrams:
pdoe-1.1f-1-enforcing-data-residency.jpg,pdoe-1.1f-2-don-forget-telemetry.jpg) - Quick Decision Cheat Sheet (Subsection 1.1)
- Subsection 1.2: Managing infrastructure (Section 1: Bootstrapping and Maintaining a Google Cloud Organization)
- Consideration 1.2a: Infrastructure-as-code tooling and managed services (e.g., Infrastructure Manager, Cloud Foundation Toolkit, Config Connector, GitOps, Terraform, Helm) (Diagrams:
pdoe-1.2a-1-infrastructure-manager.jpg,pdoe-1.2a-2-foundation-toolkit-cft.jpg) - Consideration 1.2b: Making infrastructure changes using Google-recommended practices and blueprints (Diagrams:
pdoe-1.2b-1-recommended-iac-workflow.jpg,pdoe-1.2b-2-modularization.jpg) - Consideration 1.2c: Automation with scripting (e.g., Python, Go) (Diagrams:
pdoe-1.2c-1-official-client.jpg,pdoe-1.2c-2-authentication-via.jpg) - Quick Decision Cheat Sheet (Subsection 1.2)
- Subsection 1.3: Designing a CI/CD architecture stack in Google Cloud, hybrid, and multi-cloud environments (Section 1: Bootstrapping and Maintaining a Google Cloud Organization)
- Consideration 1.3a: Continuous integration (CI) with Cloud Build (Diagrams:
pdoe-1.3a-1-build-cloudbuild-yaml.jpg,pdoe-1.3a-2-optimizing-build-speed.jpg) - Consideration 1.3b: Continuous delivery (CD) with Cloud Deploy, including Kustomize and Skaffold (Diagrams:
pdoe-1.3b-1-deploy-delivery-pipeline.jpg,pdoe-1.3b-2-skaffold-kustomize.jpg) - Consideration 1.3c: Artifact Registry configuration (Diagrams:
pdoe-1.3c-1-artifact-registry.jpg,pdoe-1.3c-2-immutability-cleanup.jpg) - Consideration 1.3d: Widely used third-party tooling (e.g., Git, Jenkins, Argo CD, Packer, kpt) (Diagrams:
pdoe-1.3d-1-to-choose-each-third.jpg,pdoe-1.3d-2-packer-gke-compute-build.jpg) - Consideration 1.3e: Security of CI/CD tooling (Diagrams:
pdoe-1.3e-1-principle-of-least.jpg,pdoe-1.3e-2-never-build-secret.jpg) - Quick Decision Cheat Sheet (Subsection 1.3)
- Subsection 1.4: Managing multiple environments (e.g., staging, production) (Section 1: Bootstrapping and Maintaining a Google Cloud Organization)
- Consideration 1.4a: Managing ephemeral environments (Diagrams:
pdoe-1.4a-1-architectural-patterns.jpg,pdoe-1.4a-2-functions-scheduler.jpg) - Consideration 1.4b: Managing configuration and policy (Diagrams:
pdoe-1.4b-1-config-sync-gitops.jpg,pdoe-1.4b-2-policy-controller-opa.jpg) - Consideration 1.4c: Managing Google Kubernetes Engine (GKE) clusters across an enterprise (e.g., fleets) (Diagrams:
pdoe-1.4c-1-gke-fleets-environ.jpg,pdoe-1.4c-2-core-fleet-capabilities.jpg) - Consideration 1.4d: Safe and secure patching and upgrading practices (Diagrams:
pdoe-1.4d-1-gke-release-channels.jpg,pdoe-1.4d-2-maintenance-windows.jpg) - Quick Decision Cheat Sheet (Subsection 1.4)
- Subsection 1.5: Enabling secure cloud development environments (Section 1: Bootstrapping and Maintaining a Google Cloud Organization)
- Consideration 1.5a: Configuring and managing cloud development environments (e.g., Cloud Workstations, Cloud Shell) (Diagrams:
pdoe-1.5a-1-workstations-vs-shell.jpg,pdoe-1.5a-2-workstations-enterprise.jpg) - Consideration 1.5b: Bootstrapping environments with required tooling (e.g., custom images, IDE, Cloud SDK) (Diagrams:
pdoe-1.5b-1-bootstrapping.jpg,pdoe-1.5b-2-important-file.jpg) - Consideration 1.5c: Leveraging AI to assist with development and operations (e.g., Gemini Code Assist, Gemini Cloud Assist, Gemini CLI) (Diagrams:
pdoe-1.5c-1-disambiguating-gemini.jpg,pdoe-1.5c-2-gemini-code-assist-code.jpg) - Quick Decision Cheat Sheet (Subsection 1.5)
Chapter 2: Building & Securing CI/CD Pipelines: Cloud Build, Cloud Deploy, Secrets, SLSA & Binary Authorization
- Subsection 2.1: Designing pipelines (Section 2: Building and Implementing CI/CD Pipelines)
- Consideration 2.1a: CI/CD of applications and infrastructure (Diagrams:
pdoe-2.1a-1-unifying-application.jpg,pdoe-2.1a-2-application-ci-cd-build.jpg) - Consideration 2.1b: Artifact management with Artifact Registry (Diagrams:
pdoe-2.1b-1-artifact-registry-multi.jpg,pdoe-2.1b-2-artifact-registry.jpg) - Consideration 2.1c: Deployment to hybrid and multi-cloud environments (e.g., GKE) (Diagrams:
pdoe-2.1c-1-hybrid-multi-deployment.jpg,pdoe-2.1c-2-build-interconnect.jpg) - Consideration 2.1d: CI/CD pipeline triggers (Diagrams:
pdoe-2.1d-1-signal-build-triggers-ci.jpg,pdoe-2.1d-2-monorepo-path-filtering.jpg) - Consideration 2.1e: Configuring deployment processes (e.g., approval flows) (Diagrams:
pdoe-2.1e-1-governance-approval.jpg,pdoe-2.1e-2-deploy-target-approvals.jpg) - Quick Decision Cheat Sheet (Subsection 2.1)
- Subsection 2.2: Implementing and managing pipelines (Section 2: Building and Implementing CI/CD Pipelines)
- Consideration 2.2a: Auditing and tracking deployments (e.g., Artifact Registry, Cloud Build, Cloud Deploy, Cloud Audit Logs) (Diagrams:
pdoe-2.2a-1-pillars-of-deployment.jpg,pdoe-2.2a-2-deploy-gke-build-deploy.jpg) - Consideration 2.2b: Deployment strategies (e.g., canary, blue/green, rolling, traffic splitting, feature flags) and defining success metrics based on application or ML pipeline telemetry (Diagrams:
pdoe-2.2b-1-comparing-deployment.jpg,pdoe-2.2b-2-rolling-update-anthos.jpg) - Consideration 2.2c: Troubleshooting and mitigating deployment issues (Diagrams:
pdoe-2.2c-1-systematic-playbook.jpg,pdoe-2.2c-2-imagepullbackoff.jpg) - Quick Decision Cheat Sheet (Subsection 2.2)
- Subsection 2.3: Managing pipeline configuration and secrets (Section 2: Building and Implementing CI/CD Pipelines)
- Consideration 2.3a: Key management (e.g., Cloud Key Management Service) (Diagrams:
pdoe-2.3a-1-management-service-kms.jpg,pdoe-2.3a-2-customer-managed.jpg) - Consideration 2.3b: Configuration and secret management (e.g., Secret Manager, Certificate Manager, Parameter Manager, Workload Identity Federation) (Diagrams:
pdoe-2.3b-1-secret-manager-best.jpg,pdoe-2.3b-2-pin-to-explicit-secret.jpg) - Consideration 2.3c: Build versus runtime secret injection (Diagrams:
pdoe-2.3c-1-build-time-vs-runtime.jpg,pdoe-2.3c-2-secret-is-needed-by.jpg) - Quick Decision Cheat Sheet (Subsection 2.3)
- Subsection 2.4: Securing the deployment pipeline (Section 2: Building and Implementing CI/CD Pipelines)
- Consideration 2.4a: Artifact Analysis and vulnerability scanning (Diagrams:
pdoe-2.4a-1-artifact-analysis-on.jpg,pdoe-2.4a-2-automated-scanning-in.jpg) - Consideration 2.4b: Software supply chain security (e.g., Binary Authorization, Supply-chain Levels for Software Artifacts [SLSA] framework) (Diagrams:
pdoe-2.4b-1-binary-authorization.jpg,pdoe-2.4b-2-attestations-work.jpg) - Consideration 2.4c: IAM policies based on environment (Diagrams:
pdoe-2.4c-1-environment-based-iam.jpg,pdoe-2.4c-2-folder-project.jpg) - Quick Decision Cheat Sheet (Subsection 2.4)
Chapter 3: Applying Site Reliability Engineering (SRE): SLIs/SLOs, Error Budgets, Autoscaling & Incident Command
- Subsection 3.1: Balancing change, velocity, and reliability of the service (Section 3: Applying Site Reliability Engineering Practices)
- Consideration 3.1a: Defining SLIs (e.g., availability, latency), SLOs, and SLAs (Diagrams:
pdoe-3.1a-1-defining-signal-slis-sli.jpg,pdoe-3.1a-2-request-response.jpg) - Consideration 3.1b: Error budgets (e.g., Cloud Service Mesh definitions) (Diagrams:
pdoe-3.1b-1-error-budget-math.jpg,pdoe-3.1b-2-error-budget-formula.jpg) - Consideration 3.1c: Opportunity cost of risk and reliability (e.g., number of "nines") (Diagrams:
pdoe-3.1c-1-opportunity-cost-of.jpg,pdoe-3.1c-2-over-performance-too.jpg) - Quick Decision Cheat Sheet (Subsection 3.1)
- Subsection 3.2: Managing service lifecycle (Section 3: Applying Site Reliability Engineering Practices)
- Consideration 3.2a: Service management (e.g., planning, deployment, maintenance, retirement) (Diagrams:
pdoe-3.2a-1-full-service-lifecycle.jpg,pdoe-3.2a-2-planning-readiness.jpg) - Consideration 3.2b: Capacity planning (e.g., quotas, limits, reservations, Dynamic Workload Scheduler) (Diagrams:
pdoe-3.2b-1-capacity-planning.jpg,pdoe-3.2b-2-organic-growth.jpg) - Consideration 3.2c: Autoscaling (e.g., managed instance groups, Cloud Run, GKE) (Diagrams:
pdoe-3.2c-1-gke-autoscaling-hpa-vpa.jpg,pdoe-3.2c-2-horizontal-pod.jpg) - Quick Decision Cheat Sheet (Subsection 3.2)
- Subsection 3.3: Mitigating incident impact on users (Section 3: Applying Site Reliability Engineering Practices)
- Consideration 3.3a: Draining/redirecting traffic (Diagrams:
pdoe-3.3a-1-graceful-traffic.jpg,pdoe-3.3a-2-load-balancer-connection.jpg) - Consideration 3.3b: Adding capacity (Diagrams:
pdoe-3.3b-1-adding-capacity.jpg,pdoe-3.3b-2-emergency-compute.jpg) - Consideration 3.3c: Rollback strategies (Diagrams:
pdoe-3.3c-1-rapid-deterministic.jpg,pdoe-3.3c-2-deploy-gke-build-run.jpg) - Quick Decision Cheat Sheet (Subsection 3.3)
Chapter 4: Implementing Cloud Observability: OpenTelemetry, Cloud Logging, PromQL, SLO Burn Alerts & Trace
- Subsection 4.1: Instrumenting and collecting telemetry (Section 4: Implementing Observability Practices and Troubleshooting Issues)
- Consideration 4.1a: Collecting and importing logs (e.g., Ops Agent, OpenTelemetry, Cloud Audit Logs, VPC Flow Logs, Cloud Service Mesh) (Diagrams:
pdoe-4.1a-1-collecting-importing.jpg,pdoe-4.1a-2-compute-engine-ops-agent.jpg) - Consideration 4.1b: Optimizing logs (e.g., filtering, sampling, exclusions, cost management, source considerations) (Diagrams:
pdoe-4.1b-1-optimizing-logs-cost.jpg,pdoe-4.1b-2-log-router-exclusion.jpg) - Consideration 4.1c: Collecting metrics (e.g., from applications, platforms, networking, Cloud Service Mesh, Google Cloud Managed Service for Prometheus, hybrid/multi-cloud environments) (Diagrams:
pdoe-4.1c-1-collecting-metrics-gmp.jpg,pdoe-4.1c-2-managed-service.jpg) - Consideration 4.1d: Creating synthetic monitors to proactively probe application endpoints and workflows (Diagrams:
pdoe-4.1d-1-proactive-probing-uptime.jpg,pdoe-4.1d-2-uptime-checks-operations.jpg) - Consideration 4.1e: Creating custom metrics, including log-based metrics (Diagrams:
pdoe-4.1e-1-custom-metrics-log-based.jpg,pdoe-4.1e-2-types-of-log-based.jpg) - Quick Decision Cheat Sheet (Subsection 4.1)
- Subsection 4.2: Managing and analyzing logs (Section 4: Implementing Observability Practices and Troubleshooting Issues)
- Consideration 4.2a: Analyzing logs using the Logs Explorer and the Logging query language (Diagrams:
pdoe-4.2a-1-mastering-logs-explorer.jpg,pdoe-4.2a-2-know-payload-fields.jpg) - Consideration 4.2b: Exporting and retaining logs (e.g., routing to BigQuery, Pub/Sub, Cloud Storage) (Diagrams:
pdoe-4.2b-1-right-log-export.jpg,pdoe-4.2b-2-custom-logging-bucket.jpg) - Consideration 4.2c: Handling sensitive data (e.g., using log processors to redact personally identifiable information [PII], protected health information [PHI]) (Diagrams:
pdoe-4.2c-1-redacting-pii-phi.jpg,pdoe-4.2c-2-pattern-redact-before.jpg) - Consideration 4.2d: Using Gemini Cloud Assist for AI-powered log analysis (Diagrams:
pdoe-4.2d-1-gemini-assist-ai-powered.jpg,pdoe-4.2d-2-natural-language-to-lql.jpg) - Quick Decision Cheat Sheet (Subsection 4.2)
- Subsection 4.3: Managing metrics, dashboards, and alerts (Section 4: Implementing Observability Practices and Troubleshooting Issues)
- Consideration 4.3a: Analyzing metrics using the Metrics Explorer (Diagrams:
pdoe-4.3a-1-time-series-in-metrics.jpg,pdoe-4.3a-2-step-temporal-alignment.jpg) - Consideration 4.3b: Managing dashboards (e.g., creating, filtering, sharing, playbooks, PromQL) (Diagrams:
pdoe-4.3b-1-dashboards-promql.jpg,pdoe-4.3b-2-widget-selection.jpg) - Consideration 4.3c: Configuring alerting and alerting policies (e.g., SLIs, SLOs, cost control) (Diagrams:
pdoe-4.3c-1-signal-alerting-policies.jpg,pdoe-4.3c-2-multi-window-burn-rate.jpg) - Consideration 4.3d: Integrating with third-party alerting tools (e.g., webhooks, PagerDuty, Rootly) (Diagrams:
pdoe-4.3d-1-integrating-third-party.jpg,pdoe-4.3d-2-multi-channel-tiered.jpg) - Consideration 4.3e: Leveraging Gemini Cloud Assist for metrics interpretation (Diagrams:
pdoe-4.3e-1-leveraging-gemini-assist.jpg,pdoe-4.3e-2-operations-monitoring.jpg) - Quick Decision Cheat Sheet (Subsection 4.3)
- Subsection 4.4: Capturing and analyzing distributed traces (Section 4: Implementing Observability Practices and Troubleshooting Issues)
- Consideration 4.4a: Utilizing tracing frameworks (e.g., OpenTelemetry) (Diagrams:
pdoe-4.4a-1-utilizing-opentelemetry.jpg,pdoe-4.4a-2-opentelemetry-is.jpg) - Consideration 4.4b: Analyzing trace waterfalls and spans (Diagrams:
pdoe-4.4b-1-reading-trace-waterfall.jpg,pdoe-4.4b-2-pattern-staircase.jpg) - Consideration 4.4c: Correlating trace IDs with structured logs (Diagrams:
pdoe-4.4c-1-correlating-trace-ids.jpg,pdoe-4.4c-2-operations-logging-run.jpg) - Consideration 4.4d: Employing Gemini Cloud Assist for trace analysis (Diagrams:
pdoe-4.4d-1-employing-gemini-assist.jpg,pdoe-4.4d-2-operations-trace-gemini.jpg) - Quick Decision Cheat Sheet (Subsection 4.4)
- Subsection 4.5: Troubleshooting issues (Section 4: Implementing Observability Practices and Troubleshooting Issues)
- Consideration 4.5a: Infrastructure issues (Diagrams:
pdoe-4.5a-1-troubleshooting-gke.jpg,pdoe-4.5a-2-pod-stuck-in-pending.jpg) - Consideration 4.5b: CI/CD pipeline issues (Diagrams:
pdoe-4.5b-1-troubleshooting-ci-cd.jpg,pdoe-4.5b-2-build-step-timeout-or.jpg) - Consideration 4.5c: Application issues (Diagrams:
pdoe-4.5c-1-troubleshooting.jpg,pdoe-4.5c-2-error-reporting.jpg) - Consideration 4.5d: Observability issues (Diagrams:
pdoe-4.5d-1-troubleshooting.jpg,pdoe-4.5d-2-vms-or-gke-nodes.jpg) - Consideration 4.5e: Performance and latency issues (Diagrams:
pdoe-4.5e-1-troubleshooting.jpg,pdoe-4.5e-2-cpu-cfs-quota-throttling.jpg) - Quick Decision Cheat Sheet (Subsection 4.5)
Chapter 5: Optimizing Performance & Cost: Cloud Profiler, Active Assist & Cloud FinOps Practices
- Subsection 5.1: Collecting performance information in Google Cloud (Section 5: Optimizing Performance and Cost)
- Consideration 5.1a: Application performance monitoring (Diagrams:
pdoe-5.1a-1-application-performance.jpg,pdoe-5.1a-2-profiler-core-mechanics.jpg) - Consideration 5.1b: Active Assist insights and recommendations (Diagrams:
pdoe-5.1b-1-active-assist-insights.jpg,pdoe-5.1b-2-insights-vs.jpg) - Quick Decision Cheat Sheet (Subsection 5.1)
- Subsection 5.2: Implementing FinOps practices for optimizing resource utilization and costs (Section 5: Optimizing Performance and Cost)
- Consideration 5.2a: Observability costs (Diagrams:
pdoe-5.2a-1-controlling.jpg,pdoe-5.2a-2-logging-cost-levers.jpg) - Consideration 5.2b: Spot virtual machines (VMs) (Diagrams:
pdoe-5.2b-1-mastering-spot-vms-spot.jpg,pdoe-5.2b-2-spot-vms-vs-legacy.jpg) - Consideration 5.2c: Optimizing resource usage for cost and efficiency (Diagrams:
pdoe-5.2c-1-optimizing-resource.jpg,pdoe-5.2c-2-labels-tags-gke-cost.jpg) - Consideration 5.2d: Infrastructure cost planning (e.g., committed-use discounts, sustained-use discounts, network tiers) (Diagrams:
pdoe-5.2d-1-infrastructure-cost.jpg,pdoe-5.2d-2-resource-based-committed.jpg) - Consideration 5.2e: Leveraging Google Cloud recommenders (e.g., cost, security, performance, manageability, reliability) (Diagrams:
pdoe-5.2e-1-leveraging-recommenders.jpg,pdoe-5.2e-2-vm-machine-type.jpg) - Consideration 5.2f: Optimizing individual workload costs (e.g., GKE, Cloud Run, Compute Engine) (Diagrams:
pdoe-5.2f-1-optimizing-individual.jpg,pdoe-5.2f-2-gke-cost-optimization.jpg) - Quick Decision Cheat Sheet (Subsection 5.2)
Chapter 6: Master Look-Alike Tool Comparison & Decision Playbook
- 6.1 The Master Look-Alike Tool Comparison Matrix
- 6.2 The 5-Step Architecture Decision Checklist
- 6.3 Top 25 Golden Rules for Final Review
Generating the Custom Illustrations (pdoe-image-prompts.json)
All 150 custom image prompts and output filenames for this book are stored in pdoe-image-prompts.json (and mirrored in openai-image-gen/out/9011-openai-prompts-pdoe-book.json). To generate the illustrations using openai-image-gen/images-from-prompts.py, run:
cd openai-image-gen uv run images-from-prompts.py -file ../books/pdoe-architecture-takeaways/pdoe-image-prompts.json -simple -quality medium
AwesomeGCP.com Learning
The approach taken is a question-led learning approach: you explore realistic project scenarios where you need to make decisions or take actions to achieve business goals within specific constraints. Instead of passive learning, you are challenged to think and work through the options to master the Professional Cloud DevOps Engineer (PDOE) certification!