Book TOC · Previous · Chapter 1 of 6 · Next

Chapter 1: Generative AI Fundamentals, Data Strategy, Gen AI Layers & Google Foundation Models

Subsections Covered: 1.1, 1.2, 1.3, 1.4 (20 Considerations · 40 Custom Visual Diagrams)

Chapter Focus: AI/ML/Gen AI hierarchy, supervised/unsupervised/RLHF lifecycles, structured vs. unstructured data, the 5-layer Gen AI stack, and Gemini, Gemma, Imagen & Veo selection.

Subsection 1.1: Describe core generative AI (gen AI) concepts and use cases

Section 1: Fundamentals of gen AI

The Big Picture

Executive teams routinely derail multi-million-dollar AI transformations in the first week because they conflate Discriminative (Predictive) Machine Learning with Generative AI, or because they attempt to train a foundation model from scratch when an existing pre-trained model adapted via prompt engineering would deliver 95% of the value at 1% of the cost.

In Subsection 1.1, production architecture focuses on whether you can think like a strategic Generative AI Leader across the entire conceptual spectrum—from the hierarchy of AI/ML/NLP/GenAI and the three core ML learning paradigms, through the 5-stage ML lifecycle on Google Cloud, to selecting the right foundation model and aligning data modalities with high-ROI business use cases:

1. The "Everything Is an LLM" Anti-Pattern (1.1a, 1.1b, 1.1e): When a CFO asks to predict next quarter's churn rate from structured CRM rows, deploying a generative Large Language Model (LLM) is expensive, slow, and prone to arithmetic hallucinations. That is a Supervised Learning classification/regression problem. Conversely, when a support organization needs to synthesize 50-page legal contracts into executive briefs or draft empathetic customer emails, a discriminative classifier cannot generate novel tokens—you need Generative AI powered by a Foundation Model (Transformer-based LLM for text/multimodal, or Diffusion Model for iterative image denoising).
2. The "Train From Scratch" Budget Sinkhole (1.1c, 1.1d): Training a foundation model from scratch requires petabytes of unlabeled data, thousands of TPUs, and months of compute. Across the ML Lifecycle (Data Ingestion → Data Preparation → Model Training → Model Deployment → Model Management), Data Preparation consumes the vast majority of engineering effort when dealing with messy legacy logs. Smart leaders select an existing Foundation Model by evaluating Modality, Context Window, Latency/Cost vs. Performance, Security, and lightweight customization (Prompt Engineering or Parameter-Efficient Prompt Tuning).
3. The Garbage-In, Hallucination-Out Data Trap (1.1f, 1.1g, 1.1h, 1.1i): Foundation models are initially pre-trained via self-supervised/unsupervised learning on massive unlabeled, unstructured corpora, then refined using labeled instruction/preference datasets (Supervised Fine-Tuning and RLHF). In production, feeding models incomplete, inconsistent, or outdated enterprise data directly degrades output accuracy regardless of model size.

Consideration 1.1a: Core gen AI concepts: AI, NLP, ML, generative AI, foundation models, multimodal models, diffusion models, prompt tuning, prompt engineering, LLMs

• The Russian-Doll Hierarchy (AI ⊃ ML ⊃ Deep Learning ⊃ Gen AI): Artificial Intelligence (AI) is the overarching discipline of building smart machines; Machine Learning (ML) is the subset where systems learn statistical patterns from data without explicit if/then rules; Generative AI is a specialized subset of deep learning that creates new, original artifacts (text, code, images, audio, video) rather than merely predicting a category or number.
• Foundation Models & Transformers: A Foundation Model is a massive neural network pre-trained on broad, web-scale data that can be adapted to dozens of downstream tasks. Modern Large Language Models (LLMs) rely on the Transformer architecture, whose self-attention mechanism processes entire token sequences in parallel to capture long-range context—overcoming the sequential bottlenecks of legacy RNNs/LSTMs.
• Unimodal vs. Multimodal vs. Diffusion Architectures: Unimodal models handle a single data type (e.g., text-to-text), whereas Multimodal Foundation Models natively ingest, correlate, and generate across multiple modalities (text, images, audio, video) in a unified embedding space. Diffusion Models generate high-resolution images/video by learning to iteratively reverse Gaussian noise back into coherent pixels conditioned on a text prompt.
• Prompt Engineering vs. Prompt Tuning vs. Full Fine-Tuning: Prompt Engineering crafts natural-language instructions and examples inside the context window without touching model weights. Prompt Tuning (a Parameter-Efficient Fine-Tuning / PEFT method) freezes the billions of base model weights and trains only a tiny vector of continuous "soft prompt" embeddings—offering huge compute and storage savings over full fine-tuning.

Key Implementation Patterns

• Generative vs. Discriminative AI: Use Use a traditional classification model for the churn prediction and a generative AI model for the newsletter drafting. Generative AI creates new content (newsletters), while traditional ML predicts labels or probabilities (churn). (Avoid alternative: Classification models predict labels and cannot generate text content like a newsletter.)
• Foundation Models Strategy: Use Choose a pre-trained foundation model and customize it for legal summarization. Foundation models are already pre-trained on vast data, allowing you to adapt them cost-effectively without training from scratch. (Avoid alternative: Training from scratch requires massive compute and data resources, which the startup lacks.)

Key Takeaways (1.1a)

• Generative AI is used to create new content such as text, images, audio, and code.
• Traditional ML (Discriminative AI) is used to classify data, predict numerical values, or cluster groups.
• Always match the business goal (creation vs. prediction) to the model type to ensure efficiency and accuracy.
• Foundation Models serve as the base for most modern Generative AI applications, eliminating the need to train from scratch.

Consideration 1.1b: Machine learning approaches: supervised, unsupervised, reinforcement

• Supervised Learning (Task-Driven via Labeled Pairs): Trains on explicit input-output pairs (X, Y) where every record has a ground-truth label (e.g., historical emails tagged Spam vs. Not Spam, or house features paired with sale prices). Used for classification (discrete categories) and regression (continuous numbers), as well as Supervised Fine-Tuning (SFT) of LLMs.
• Unsupervised / Self-Supervised Learning (Data-Driven Pattern Discovery): Operates on unlabeled data to discover latent structures, clusters, or statistical distributions (e.g., customer segmentation via $k$-means clustering, anomaly detection, or pre-training a foundation model via next-token prediction on billions of raw web documents).
• Reinforcement Learning (Goal-Driven Trial-and-Error): An autonomous agent interacts with an environment, taking actions and receiving numerical rewards or penalties to learn an optimal policy over time. In generative AI, Reinforcement Learning from Human Feedback (RLHF) aligns LLMs with human safety, helpfulness, and tone preferences using a reward model trained on human rankings.
• Architectural Tie-Breaker: If the scenario gives you historical records with known outcomes → Supervised. If it gives you raw uncategorized records to group by similarity → Unsupervised. If it describes reward signals, human preference rankings, or sequential environment feedback → Reinforcement Learning.

Key Implementation Patterns

• Foundation Model Pre-training Strategy: Use Use unsupervised learning to identify patterns and probabilities within the raw text. Unsupervised learning is the standard approach for pre-training foundation models on unstructured, unlabeled text. (Avoid alternative: Supervised learning requires labeled data, which is not available in this scenario.)
• Aligning Model Safety with Human Feedback: Use Reinforcement learning from human feedback (RLHF). RLHF uses human rankings to train a reward model that guides the generative model's behavior. (Avoid alternative: Unsupervised learning finds patterns in unlabeled data and does not use human feedback or rankings.)
• Fine-Tuning for Customer Support: Use Apply supervised learning to fine-tune the model using the inquiry-response pairs. Supervised learning uses labeled data (input-output pairs) to train or adapt models. (Avoid alternative: Reinforcement learning relies on rewards from actions, not a static dataset of input-output pairs.)

Key Takeaways (1.1b)

• Unsupervised Learning is used for pre-training foundation models on large, unlabeled datasets.
• Supervised Learning requires labeled input-output pairs.
• Reinforcement Learning uses rewards and is typically used for fine-tuning rather than initial training.
• Unstructured data (like raw text) usually implies an unsupervised or self-supervised approach initially.

Consideration 1.1c: Machine learning lifecycle and Google Cloud tools: ingestion, preparation, training, deployment, management

• Stage 1 — Data Ingestion: Collecting and importing raw batch or streaming data from operational sources into Google Cloud using Cloud Storage (unstructured blobs), BigQuery Data Transfer Service (batch warehouse loads), or Pub/Sub (real-time event streams).
• Stage 2 — Data Preparation (Frequently the Most Time-Intensive Stage): Cleaning missing values, normalizing inconsistent legacy schemas, deduplicating records, de-identifying PII (Cloud DLP / Sensitive Data Protection), and running scalable ETL pipelines via Dataflow, Dataproc, or BigQuery SQL.
• Stage 3 — Model Training & Customization: Selecting an algorithm or foundation model and executing training/tuning jobs using Vertex AI Training (custom code), Vertex AI AutoML (no-code tabular/image/text training), or Vertex AI Studio (supervised fine-tuning and prompt tuning).
• Stage 4 & Stage 5 — Model Deployment & Model Management: Exposing trained models in Vertex AI Model Registry to production Vertex AI Endpoints (real-time low-latency online prediction) or Batch Prediction jobs, followed by continuous Vertex AI Model Monitoring for data/concept drift, lineage tracking, and Vertex AI Feature Store management.

Key Implementation Patterns

• Unstructured Data Ingestion Strategy: Use Upload the raw PDF and audio files to a Cloud Storage bucket. Cloud Storage is the standard object storage for unstructured data (blobs) and integrates natively with Vertex AI for data ingestion. (Avoid alternative: Cloud SQL is optimized for structured relational data, not for storing terabytes of unstructured binary files like PDFs and audio.)
• RAG Implementation and Vectorization: Use Upload the embeddings to an index in Vertex AI Vector Search to perform approximate nearest neighbor queries. Vertex AI Vector Search provides managed, low-latency similarity search for embeddings. (Avoid alternative: Cloud Storage scan is too slow for real-time search.)

Key Takeaways (1.1c)

• Unstructured data (PDFs, images, audio) requires Object Storage.
• Google Cloud Storage (GCS) is the default landing zone for raw data in Gen AI pipelines.
• Vertex AI integrates natively with GCS for reading datasets.
• Relational databases (Cloud SQL) and Data Warehouses (BigQuery) are primarily for structured data, not raw file storage.

Consideration 1.1d: Choosing foundation models based on modality, context window, security, reliability, cost, performance, tuning, customization

• Modality & Context Window First: Always filter candidate foundation models by input/output modality (e.g., if users upload accident photos + voice notes, eliminate text-only LLMs in favor of a multimodal model) and context window capacity (the maximum combined input + output tokens a model can attend to in a single call, such as analyzing a 300-page PDF without chunking).
• The Latency / Cost / Intelligence Triangle: High-parameter reasoning models (Gemini Pro) excel at complex multi-step logic and coding but incur higher latency and per-token cost. Lightweight models (Gemini Flash) deliver sub-second time-to-first-token and drastically lower cost for high-volume summarization, extraction, and Level-1 chat.
• Security, Sovereignty & Enterprise SLAs: In regulated industries (finance, healthcare, public sector), model selection is governed by data residency, zero training on customer data, VPC Service Controls, IAM, and indemnification/reliability SLAs—steering leaders toward managed enterprise platforms (Vertex AI) or self-hosted open-weight models (Gemma) when air-gapped execution is required.
• Customization Spectrum: Choose a base model that supports the minimum viable adaptation level—starting with in-context prompting, adding RAG/grounding for private knowledge, and reserving fine-tuning only when domain-specific terminology or strict stylistic alignment requires weight adaptation.

Key Implementation Patterns

• Multimodal Input for Insurance Claims: Use Choose a multimodal foundation model that supports both image and text data as input. Multimodal models can process text and images together, making them suitable for analyzing claims with photos and descriptions. (Avoid alternative: Text-only models cannot process image data, which is required for this scenario.)
• High-Volume Customer Support Bot: Use Select Gemini Flash to prioritize response speed and cost-effectiveness. Gemini Flash is optimized for speed and cost, making it ideal for high-volume, simple queries. (Avoid alternative: Gemini Ultra is too expensive and slow for simple, high-volume tasks.)
• Legal Contract Summarization: Use Select Gemini Pro for its large context window capability. Gemini Pro features a large context window (1M+ tokens) capable of processing 400+ pages of text in a single prompt. (Avoid alternative: Gemini 1.0 Pro has a context window limit (approx 32k tokens) that is too small for a 400-page document.)

Key Takeaways (1.1d)

• Identify the input data types (modalities) required for your use case (e.g., text, image, audio).
• Select a Multimodal foundation model (like Gemini) when you need to reason across different inputs simultaneously (e.g., asking questions about an image).
• Avoid using specialized single-modality models (like text-only or image-generation) for tasks requiring mixed-media analysis.
• Match the model size to the complexity of the task.

Consideration 1.1e: Business use cases: creation, summarization, discovery, automation (text, image, code, video, analytics, personalization)

• The Four Pillars of Gen AI Business Value (Create · Summarize · Discover · Automate):
• Create: Drafting personalized marketing copy, generating product imagery (Imagen), producing promotional video clips (Veo), and synthesizing boilerplate application code (Gemini Code Assist).
• Summarize: Distilling 100-page earnings transcripts, doctor-patient clinical audio notes, or multi-thread support tickets into concise, actionable executive briefs.
• Discover: Enabling natural-language semantic search across millions of internal HR policies, engineering wikis, and PDFs so employees find exact answers rather than keyword links.
• Automate: Orchestrating multi-step business workflows—such as extracting invoice line items, validating purchase orders in an ERP system, and routing exceptions to human reviewers.
• Hyper-Personalization at Scale: Unlike static rule engines that bucket users into 5 rigid segments, Generative AI dynamically tailors product recommendations, learning paths, and outreach messages to each individual's real-time context and history.

Key Implementation Patterns

• Executive Briefing Automation: Use Implement a text summarization model to generate abstractive summaries of the reports. Text summarization condenses long content into concise summaries, directly addressing the business need. (Avoid alternative: Classification organizes documents but does not shorten them or extract key insights.)
• Marketing Asset Generation: Use Use a text-to-image model to generate the backgrounds and composite the products into them. Text-to-image models generate new visuals from descriptions, fitting the creation use case perfectly. (Avoid alternative: Discriminative models classify data rather than create new content.)

Key Takeaways (1.1e)

• Identify the specific business constraint (e.g., lack of time to read) to select the right AI capability.
• Text Summarization is the primary tool for condensing information for executive review.
• Distinguish between search (finding data) and summarization (condensing data).
• Generative AI is used to create new content (images, text, audio) rather than just analyze existing data.

Consideration 1.1f: Use of various data types in gen AI and business implications

• Unstructured Data Is the Native Fuel of Gen AI: Traditional BI and analytics only tap the ~20% of enterprise data stored in structured relational rows. Generative AI unlocks the remaining 80% of dark, unstructured enterprise data—contracts, call recordings, emails, slide decks, images, and videos—turning passive storage buckets into queryable intelligence.
• Multimodal Business Workflows: Modern business processes rarely live in a single data type. An insurance claim combines structured policy tables (BigQuery), unstructured text adjuster notes, images of vehicle damage, and audio customer statements. Multimodal models process these heterogeneous inputs natively in a single pass, eliminating brittle multi-model stitching pipelines.
• Embeddings as the Universal Translator: Behind the scenes, embedding models project text fragments, images, and audio clips into high-dimensional numerical vectors (embeddings) where geometric proximity reflects semantic similarity—powering cross-modal search, recommendations, and RAG.

Key Implementation Patterns

• Multimodal Input for Retail Inventory: Use Use a multimodal model to analyze the images and generate text descriptions. Multimodal models can process image inputs and generate text outputs, making them suitable for visual classification tasks. (Avoid alternative: Text-to-image models generate images from text, they do not analyze existing images to produce text.)
• Generative Images for Marketing Campaigns: Use Use a Text-to-Image model to generate visual assets from your text descriptions. Text-to-Image models allow you to create visual assets directly from written descriptions. (Avoid alternative: Text-to-Text models generate text, not images.)
• Audio Transcription for Quality Assurance: Use Convert the audio files to text using a Speech-to-Text service, and then process the transcripts with the LLM. You must transcribe audio into text using a Speech-to-Text service before a text-based LLM can analyze the content.

Key Takeaways (1.1f)

• Unimodal models process only one type of data (e.g., text-to-text).
• Multimodal models can process mixed inputs, such as images and text, to generate outputs.
• Use multimodal models for tasks requiring visual analysis, such as image captioning, classification from photos, or visual question answering.
• Text-to-Image models generate visual content from natural language descriptions.

Consideration 1.1g: Importance of data quality and accessibility: completeness, consistency, relevance, availability, cost, format

• The Six Dimensions of Enterprise Data Quality:
• Completeness: Are critical fields, documents, or historical segments missing?
• Consistency: Do customer records or product definitions conflict across CRM, ERP, and support silos?
• Relevance ( & Timeliness): Is the dataset applicable and current for the business problem (e.g., pre-pandemic 2019 pricing data is irrelevant for 2026 inflation forecasting)?
• Availability & Accessibility: Can authorized AI pipelines and RAG retrievers programmatically access the data with low latency and proper IAM permissions, or is it locked in offline silos?
• Format: Is the data stored in clean, machine-parsable formats (UTF-8 JSON, Parquet, clean Markdown) rather than corrupted scans or proprietary binary blobs?
• Cost: Does the cost of acquiring, cleaning, storing, and labeling the dataset justify the incremental model accuracy gain?

Key Implementation Patterns

• Data Completeness in RAG Systems: Use Upload the new Remote Work Policy document to the data store and refresh the index. The bot lacks the new policy document; adding it to the data store ensures the model has the correct information to reference. (Avoid alternative: Lowering temperature reduces randomness but does not provide the missing information required to answer correctly.)
• Data Consistency and Conflicting Sources: Use Remove the transcripts generated before the policy change from the training dataset. The training data contains conflicting information due to a policy change; removing outdated data restores consistency. (Avoid alternative: Lowering temperature reduces randomness but does not correct factually inconsistent training data.)

Key Takeaways (1.1g)

• Data Completeness is Key: Gen AI models cannot answer questions about private data if that data has not been ingested into the system.
• RAG over Fine-tuning: For knowledge updates (like new policies), update the search index (RAG) rather than fine-tuning the model.
• Grounding: To prevent hallucinations, ensure the model is grounded in the most current and relevant documents.
• Data Consistency is crucial: Generative models memorize patterns; conflicting patterns in training data lead to conflicting outputs.

Consideration 1.1h: Structured vs unstructured data and real-world examples

• Structured Data: Highly organized data that conforms to a predefined, rigid schema of rows, columns, and data types—stored in relational databases (Cloud SQL, AlloyDB) or enterprise data warehouses (BigQuery). Examples: point-of-sale transaction tables, inventory SKU ledgers, customer age/zip-code columns. Queryable via deterministic SQL.
• Unstructured Data: Data that lacks a predefined tabular schema and exists in its native free-form format—typically stored in object storage (Cloud Storage). Examples: PDF legal contracts, customer support call MP3 audio files, medical X-ray JPEGs, surveillance MP4 videos, and free-text social media posts.
• Semi-Structured Bridge: Formats like JSON, XML, and CSV logs that carry self-describing tags or loose hierarchy without strict relational normalization.

Key Implementation Patterns

• Identifying Unstructured Data for Summarization: Use Classify the notes as unstructured data because they lack a rigid data model. Explain that LLMs can interpret natural language semantics to generate summaries. Free-text notes are unstructured data; LLMs excel at processing this type of data to extract meaning and generate summaries.
• Text-to-SQL for Structured Data Interaction: Use Implement a foundation model capable of Text-to-SQL conversion to generate queries based on the database schema. Text-to-SQL models translate unstructured natural language into structured queries required by relational databases.
• Structured Data for Predictive vs. Generative AI: Use Use a predictive AI model, such as a time-series forecasting model. Predictive AI is best suited for forecasting numerical values from structured historical data. (Avoid alternative: Generative AI creates new content like text or images, which does not solve the numerical forecasting problem.)

Key Takeaways (1.1h)

• Unstructured data includes free text, images, and audio, and lacks a predefined schema.
• Generative AI (LLMs) excels at tasks requiring semantic understanding of unstructured data, such as summarization and extraction.
• Traditional databases and SQL are optimized for structured data, not for interpreting the nuance of free text.
• Relational databases store structured data that requires SQL for interaction.

Consideration 1.1i: Labeled vs unlabeled data

• Unlabeled Data: Raw text, images, audio, or logs without human-annotated target tags. It is abundant and cheap to collect at petabyte scale, forming the foundation of self-supervised pre-training (where an LLM learns grammar, facts, and reasoning by predicting masked or subsequent tokens across billions of unlabeled documents) and unsupervised clustering.
• Labeled Data: Data explicitly annotated with ground-truth target answers, categories, bounding boxes, or human preference scores. It is expensive and slow to curate (often requiring domain experts like radiologists or attorneys), but comparatively small volumes (hundreds to thousands of high-quality examples) are essential for Supervised Learning, Supervised Fine-Tuning (SFT), and RLHF alignment.

Key Implementation Patterns

• Foundation Model Pre-training Requirements: Use Gather a massive dataset of unlabeled text from diverse sources. Pre-training an LLM requires a massive corpus of unlabeled text to learn general language patterns via self-supervised learning. (Avoid alternative: Structured numerical data lacks the linguistic context required to train a Large Language Model.)
• Supervised Fine-Tuning (SFT) Implementation: Use Create a dataset of labeled pairs containing raw logs and the corresponding JSON output. Supervised Fine-Tuning requires labeled input-output pairs to teach the model how to transform specific inputs into specific outputs.

Key Takeaways (1.1i)

• Pre-training requires massive amounts of unlabeled data to learn general patterns via self-supervised learning.
• Fine-tuning requires smaller sets of labeled data to specialize the model for tasks.
• Foundation models are the output of the pre-training phase.
• Unlabeled Data is used for pre-training to teach general knowledge and language patterns.

Quick Decision Cheat Sheet (Subsection 1.1)

Further Reading (Subsection 1.1)

• Introduction to Generative AI — Provides a fundamental overview of what Generative AI is, how it differs from traditional AI, and its primary use cases like content generation.
• Foundation models in Vertex AI — Explains the concept of Foundation Models available in Vertex AI Model Garden and how they can be used for various modalities like text and images.
• Introduction to Generative AI — This introductory course explains the fundamentals of how Large Language Models are trained, distinguishing between supervised, unsupervised, and reinforcement learning in the context of GenAI.
• RLHF on Google Cloud | Google Cloud Blog — Documentation on how RLHF is used in Vertex AI to tune models using human preference data.
• Supervised Learning — A clear overview of supervised learning, explaining how it uses labeled datasets to train algorithms to classify data or predict outcomes accurately.
• Cloud Storage: Overview — Provides an overview of Cloud Storage, highlighting its use cases for storing unstructured data like blobs and objects, which is critical for ML datasets.

Subsection 1.2: Describe how various data types are used in gen AI and the business implications

Section 1: Fundamentals of gen AI

The Big Picture

A foundation model is only as trustworthy as the enterprise data it ingests during training, fine-tuning, and retrieval-augmented generation (RAG). Many executive teams assume that because Large Language Models were pre-trained on trillions of public web tokens, they can simply point an agent at a messy internal shared drive and get instant enterprise intelligence.

In production, three data failures routinely sink generative AI deployments:

1. The Data Quality & Accessibility Blind Spot (1.2a): If an internal HR policy repository contains conflicting 2021 and 2025 PDF versions (poor consistency and relevance), missing regional addendum pages (poor completeness), or locked departmental permissions (poor accessibility), a RAG-powered chatbot will confidently synthesize obsolete policy advice.
2. The Structured vs. Unstructured Tool Mismatch (1.2b): Using an unconstrained text LLM to calculate exact sum-of-sales across 10 million rows of structured relational data in BigQuery leads to severe calculation errors and token bloat; conversely, traditional SQL engines cannot extract clauses from unstructured PDF invoices or customer call audio without AI perception (Document AI, Speech-to-Text, or multimodal Gemini).
3. The Labeling Economics Trap (1.2c): Organizations waste months paying subject-matter experts to manually label millions of documents for pre-training when unlabeled data powers self-supervised pre-training and RAG indexing—whereas small, surgical sets of labeled data (100–1,000 curated examples) are reserved strictly for Supervised Fine-Tuning (SFT), evaluation golden datasets, and RLHF reward modeling.

Consideration 1.2b: Structured vs unstructured data and examples

• Core Architectural Distinction (1.2a Data Quality + 1.2b Structured vs. Unstructured): Before selecting a storage engine or AI pipeline, audit both Data Quality (Completeness, Consistency, Relevance, Availability, Cost, Format) and Data Structure. Structured data fits neatly into fixed relational rows and columns with strict data types (INT64, FLOAT64, TIMESTAMP) in BigQuery or Cloud SQL. Unstructured data has no predefined tabular schema and encompasses free-text documents, PDFs, emails, audio recordings, images, and video files stored in Cloud Storage.
• Production Use Cases:
• Structured: Inventory SKU quantities, credit card transaction timestamps and amounts, employee salary tables, sensor telemetry timestamps.
• Unstructured: Doctor-patient clinical notes, customer support phone call recordings, scanned vendor PDF invoices, product review paragraphs, CCTV video feeds.
• Bridging the Divide with Generative AI: Generative AI and Google Cloud pre-built APIs (Document AI, Speech-to-Text, Gemini Multimodal) act as a Structure-Extraction Engine—ingesting messy unstructured PDFs or audio calls and outputting clean, structured JSON records that can be loaded directly into BigQuery for SQL analytics.
• When NOT to Use Gen AI on Structured Data: If a CFO wants exact deterministic aggregations (e.g., Q4 revenue forecasting or exact sum/average queries over tabular sales rows), use SQL in BigQuery or tabular AutoML / BigQuery ML—never dump millions of raw CSV numbers into an LLM prompt hoping it will do mental math.

Key Implementation Patterns

• Customer Sentiment Analysis Source Identification: Use The repository of customer support emails and product review comments. Emails and reviews contain unstructured natural language text, which is the required input for AI-driven sentiment analysis. (Avoid alternative: Sales transaction data is structured and quantitative, making it unsuitable for extracting qualitative sentiment.)
• Predictive Sales Forecasting Strategy: Use Advise the CFO to use traditional machine learning forecasting models suited for structured data. Structured numerical data requires traditional predictive models for accuracy, not Generative AI. (Avoid alternative: LLMs are probabilistic and linguistic tools, not mathematical calculation engines.)
• Invoice Processing Automation: Use Prompt the model to extract the specific fields and output them as structured JSON. Extraction converts unstructured documents into structured formats like JSON, which is ideal for database storage. (Avoid alternative: Summarization produces unstructured text, which does not fit the requirement for structured SQL database entry.)

Key Takeaways (1.2b)

• Structured Data consists of numbers and categories in rows/cols, best for quantitative metrics.
• Unstructured Data includes text, images, and audio, best for qualitative AI tasks like sentiment analysis.
• Generative AI models require natural language input (unstructured data) to discern emotion and intent.
• Generative AI is best for unstructured data (text, images, audio) and tasks like summarization, creation, and translation.

Consideration 1.2c: Labeled vs unlabeled data

• Unlabeled Data (The Scale Engine): Raw data collected naturally from business operations without human-assigned target tags (e.g., 10 million raw support articles, unannotated product photos, raw web crawl text). Because it requires zero human annotation labor, it is used in massive volumes during self-supervised pre-training of foundation models (where the model teaches itself language structure by predicting the next token) and as the knowledge base for RAG / Vertex AI Search data stores.
• Labeled Data (The Precision Steering Wheel): Data where every example is paired with a verified ground-truth label, target summary, classification tag, or human preference score (e.g., 2,000 customer emails manually tagged as Phishing vs. Legitimate, or 500 medical prompts paired with physician-approved gold-standard responses).
• Strategic Business Implications:
• Pre-training from scratch demands trillions of unlabeled tokens.
• Supervised Fine-Tuning (SFT) and Supervised Classification require curated, high-quality labeled input-output pairs.
• RLHF (Reinforcement Learning from Human Feedback) requires human raters to label/rank multiple candidate model outputs from best to worst to train a reward model.

Key Implementation Patterns

• Foundation Model Pre-training Requirements: Use Compile a massive dataset of unlabeled internal documents and product descriptions. Pre-training uses massive amounts of unlabeled data to learn language structure and domain vocabulary. (Avoid alternative: Q&A pairs are used for instruction tuning after pre-training, not for learning the initial domain vocabulary.)
• Supervised Fine-Tuning (SFT) Strategy: Use Create a dataset of labeled pairs with accident reports as inputs and compliant denial letters as outputs to fine-tune the model. Supervised Fine-Tuning (SFT) using labeled input-output pairs is necessary to teach a model a specific task and output format.

Key Takeaways (1.2c)

• Foundation Models are pre-trained on massive amounts of unlabeled data using self-supervised learning.
• Unlabeled data allows the model to learn general language structure and domain vocabulary.
• Labeled data is resource-intensive and primarily used for fine-tuning specific tasks after pre-training.
• Domain adaptation requires training on data specific to that domain (e.g., internal documents) to capture unique jargon.

Quick Decision Cheat Sheet (Subsection 1.2)

Further Reading (Subsection 1.2)

• Introduction to Generative AI — This foundational course defines Generative AI and discusses the types of data (text, images, audio) that Gen AI models generate and analyze.
• What is Unstructured Data? — Google Cloud's definition of unstructured data, contrasting it with structured data and highlighting examples like emails and social media.
• Predictive AI vs Generative AI — An article detailing the differences between predictive models (analyzing past data to predict future outcomes) and generative models (creating new data).
• Overview of prompting strategies — This guide explains how to design prompts to extract entities from text and output them in formats like JSON.
• Introduction to Generative AI — Provides a high-level overview of how foundation models are trained and the difference between pre-training and fine-tuning.
• Tuning foundation models — Explains the process of tuning models using labeled datasets, contrasting it with the base model capabilities.

Subsection 1.3: Identify the core layers of the gen AI landscape and the business implications

Section 1: Fundamentals of gen AI

The Big Picture

When executives evaluate generative AI investments, vendors pitch everything from bare-metal GPU clusters to consumer chat apps using the same generic "AI solution" label. A Google Cloud Certified Generative AI Leader decomposes the generative AI ecosystem into Five Distinct Architectural Layers:

1. Infrastructure Layer (1.3a): The physical and virtualized compute, networking, and storage foundation—Google's custom Cloud TPUs, NVIDIA GPUs, AI Hypercomputer, and high-throughput data centers.
2. Models Layer (1.3b): The neural network brains—first-party proprietary models (Gemini, Imagen, Veo), open-weight models (Gemma, Llama, Mistral), and task-specific embeddings.
3. Platforms Layer (1.3c): The managed developer and MLOps control plane (Vertex AI / Agent Platform) that provides model discovery (Model Garden), tuning studios, RAG pipelines, evaluation, security, and endpoint scaling.
4. Agents Layer (1.3d): Autonomous reasoning loops that combine a foundation model with memory, planning, grounding data stores, and external action tools (Extensions/Functions) to execute multi-step business workflows.
5. Applications Layer (1.3e): The end-user-facing software experience—either turnkey SaaS (Gemini for Google Workspace, Gemini App, NotebookLM, Customer Engagement Suite) or custom web/mobile interfaces.

Understanding which layer solves a problem prevents two classic executive blunders: building at too low a layer (provisioning raw GPU clusters when a managed Platform API or turnkey Application exists) or buying at too rigid a layer (buying a closed consumer chat app when you need an Agent layer that can invoke internal ERP APIs).

Consideration 1.3a: Infrastructure

• What Lives in the Infrastructure Layer: Purpose-built AI accelerators (Google Cloud TPUs and NVIDIA GPUs), high-bandwidth low-latency cluster networking, liquid-cooled data centers, and distributed storage systems unified under the Google Cloud AI Hypercomputer architecture.
• Business Implications of the Infrastructure Layer: Training and serving billion-parameter models is bound by compute FLOPs, High-Bandwidth Memory (HBM), and inter-chip interconnect speed. By consuming cloud AI infrastructure elastically (pay-as-you-go or committed reservation) rather than buying on-premises GPU racks, enterprises avoid multi-million-dollar CapEx depreciation and 6-month hardware supply-chain delays.

Key Implementation Patterns

• Hardware Selection for Large-Scale Training: Use Select Cloud TPUs (Tensor Processing Units). Cloud TPUs are custom silicon optimized for TensorFlow and massive scaling, offering high throughput-per-dollar. (Avoid alternative: GPUs are powerful but often have a higher cost per unit of throughput compared to TPUs for specific TensorFlow workloads.)
• Cost Optimization for Fault-Tolerant Training: Use Use Spot VMs for the compute instances. Spot VMs provide the lowest cost for workloads that can handle interruptions, such as fault-tolerant training jobs. (Avoid alternative: On-demand VMs are full-price instances and do not offer the cost savings needed for a strict budget scenario.)
• Orchestration for Portable Workloads: Use Deploy the model on Google Kubernetes Engine (GKE). GKE aligns with the existing Kubernetes environment, supports rapid scaling, and ensures container portability to on-premise infrastructure. (Avoid alternative: Compute Engine requires manual management of scaling and container orchestration, lacking the automation provided by Kubernetes.)

Key Takeaways (1.3a)

• TPUs are Google's custom silicon optimized for machine learning workloads.
• TPUs typically offer better cost-efficiency (throughput-per-dollar) for TensorFlow workloads compared to generic GPUs.
• Large-scale training often requires specialized hardware accelerators rather than standard CPUs.
• Spot VMs are the most cost-effective option for stateless or fault-tolerant batch workloads.

Consideration 1.3b: Models

• What Lives in the Models Layer: Pre-trained Foundation Models across modalities and licensing models—including Google's proprietary frontier models (Gemini, Imagen, Veo), Google's open-weight models (Gemma), partner models (Anthropic Claude), and open-source community models (Llama, Mistral).
• Business Implications of the Models Layer: No single model is optimal for every workload. Strategic leaders adopt a Model-Agnostic / Multi-Model Portfolio—routing simple, high-frequency tasks to fast, inexpensive small models (Gemini Flash or Gemma) while reserving frontier reasoning models (Gemini Pro) for complex multi-modal synthesis.

Key Implementation Patterns

• Model Selection for Multimodal Inputs: Use Select a multimodal model like Gemini. Multimodal models can accept and process multiple input types, such as text and images, at the same time. (Avoid alternative: Speech-to-text models process audio, which is not one of the input types (text and photos) in this scenario.)
• Latency vs. Model Size Trade-off: Use Choose a small foundation model optimized for on-device deployment. Small models are optimized for the limited resources and low latency needs of mobile devices. (Avoid alternative: Large models require too much compute power and memory for mobile devices.)

Key Takeaways (1.3b)

• Modalities: Data types like text, images, and audio are called modalities.
• Multimodal Models: Models like Gemini can process multiple modalities simultaneously (e.g., Image + Text).
• Use Case Alignment: Always match the model's input capabilities (text-only vs. multimodal) to your specific business requirements.
• Match model size to the use case complexity and deployment target.

Consideration 1.3c: Platforms

• What Lives in the Platforms Layer: Vertex AI (Agent Platform)—the unified enterprise software fabric that sits between raw models/infrastructure and application teams. It provides Model Garden, Vertex AI Studio, RAG & Vector Search, AutoML, Model Registry, Pipelines, Model Monitoring, and enterprise security controls (IAM, VPC Service Controls, CMEK).
• Business Implications of the Platforms Layer: Without a unified platform layer, data scientists waste 80% of their time stitching together brittle open-source libraries, custom Docker containers, and ad-hoc logging. The Platform layer standardizes governance, accelerates time-to-production, and prevents shadow-IT sprawl.

Key Implementation Patterns

• Model Discovery and Selection: Use Use Vertex AI Model Garden to discover and test the models. Vertex AI Model Garden is the managed hub for discovering and interacting with both proprietary and open-source foundation models. (Avoid alternative: Compute Engine requires manual infrastructure management and does not offer a built-in catalog for model discovery.)
• Enterprise Search Implementation: Use Create a data store agent using Vertex AI Agent Builder and ingest the PDF documents from Cloud Storage. Vertex AI Agent Builder provides an out-of-the-box solution to ingest documents and build a grounded chatbot with minimal effort.
• Rapid Prototyping Interface: Use Use Vertex AI Studio to test and save prompts directly in the Google Cloud console. Vertex AI Studio offers a GUI for prompting and testing models without requiring code. (Avoid alternative: Vertex AI Workbench requires coding skills, which does not fit the scenario of a user with limited coding experience.)

Key Takeaways (1.3c)

• Model Garden is the single destination in Vertex AI to search, discover, and interact with foundation models.
• It supports both Google-proprietary models (1st party) and open-source models (3rd party).
• It abstracts infrastructure management, allowing for quick testing and deployment.
• Vertex AI Agent Builder is the fastest way to build search and chat apps based on your own data.

Consideration 1.3d: Agents

• What Lives in the Agents Layer: Goal-directed systems (Vertex AI Agent Builder / Conversational Agents) that wrap a foundation model in an iterative Reason + Act (ReAct) loop equipped with short/long-term memory, grounding data stores, and executable tools (Extensions, Function Calling, Plugins).
• Passive Model vs. Active Agent : A standalone Model is stateless and passive—given a prompt, it predicts text tokens and stops. An Agent is autonomous and action-oriented—given a high-level goal ("Process this customer's return and issue a replacement order"), it plans subtasks, queries an order database, calls a shipping API, verifies the result, and updates system state.

Key Implementation Patterns

• Defining the Agent Layer vs. Model Layer: Use Design an AI agent configured with tools to access the database and website API. Agents extend LLM capabilities by using tools to interact with external systems and execute tasks autonomously. (Avoid alternative: Fine-tuning improves the content generation quality but does not provide the capability to execute external actions or connect to databases.)
• Connecting Agents to External Data: Use Configure the agent with a tool or extension that connects to the mainframe's API. Tools and extensions allow agents to perform actions and query external APIs for real-time, deterministic data. (Avoid alternative: Fine-tuning cannot handle real-time data updates and is prone to hallucinations for exact number lookups.)

Key Takeaways (1.3d)

• Models generate content; Agents execute tasks. When a requirement shifts from 'write text' to 'perform an action' (like updating a database), you need an Agent.
• Tools extend capabilities. Agents rely on defined tools (functions/APIs) to interact with the outside world.
• LLMs are the reasoning engine. In an agent architecture, the LLM acts as the brain that plans which tools to use.
• Agents extend the capabilities of LLMs by using Tools to interact with external systems.

Consideration 1.3e: Applications

• What Lives in the Applications Layer: The tangible user interfaces, workflows, and business solutions where employees or external customers consume AI—such as Gemini for Google Workspace (side panels in Gmail/Docs/Sheets), Gemini Enterprise / NotebookLM, Customer Engagement Suite, or custom-built enterprise portals.
• Business Implications of the Applications Layer: Users do not care about vector embeddings or TPU pods; they care about frictionless workflows. Delivering AI directly inside the applications employees already use every day (like Google Docs or a CRM desktop) drives immediate adoption and measurable productivity ROI.

Key Implementation Patterns

• Customer Support Automation: Use Implement a generative AI conversational agent capable of understanding intent and generating context-aware responses. Generative AI agents understand intent and context, handling repetitive queries more effectively than rule-based systems.
• Developer Productivity Tools: Use Deploy an AI-powered code assistant. Code assistants are specialized Gen AI applications that integrate into IDEs to generate code, write tests, and document APIs. (Avoid alternative: Chatbots are often separate from the IDE and lack the seamless integration required for autocomplete and context-aware coding tasks.)
• Enterprise Knowledge Retrieval: Use Deploy a search and synthesis application that retrieves relevant information and generates a natural language summary. A search and synthesis application uses Gen AI to read retrieved documents and generate a concise summary answer. (Avoid alternative: Improving indexing helps keyword search but does not provide the generative synthesis required to answer questions directly.)

Key Takeaways (1.3e)

• Generative AI excels at understanding natural language variation and intent compared to keyword-based systems.
• Conversational agents act as the application layer interface between users and LLMs.
• Deploying Gen AI for repetitive queries increases ticket deflection and operational efficiency.
• Code Assistants are specialized Gen AI applications for developer productivity.

Quick Decision Cheat Sheet (Subsection 1.3)

Further Reading (Subsection 1.3)

• Cloud TPU — Overview of Cloud TPU features, highlighting cost-effectiveness and optimization for machine learning workloads.
• Introduction to Cloud TPU — Documentation explaining when to use TPUs versus GPUs or CPUs.
• Spot VMs — Overview of Spot VMs, explaining how they offer significant cost savings for fault-tolerant workloads and how preemption works.
• Architecture of the Generative AI landscape — Provides context on the layers of Gen AI, including the infrastructure layer where compute decisions are made.
• AI Infrastructure Options — Google Cloud offers various infrastructure options for AI. GKE is highlighted for providing flexibility, portability, and control over containerized AI workloads.
• Gemini - Google DeepMind — Overview of the Gemini model family, highlighting its native multimodal capabilities to understand text, images, video, and audio.

Subsection 1.4: Identify the use cases and strengths of Google’s foundation models

Section 1: Fundamentals of gen AI

The Big Picture

Google's foundation model portfolio is purpose-built so that leaders never have to force a single monolithic model onto every business problem. Yet in Production—and in real enterprise architecture reviews—teams frequently make four costly errors:

1. Stiching Separate OCR + Audio Transcription + Text Models When Native Multimodality Exists (1.4a — Gemini): Legacy pipelines required chaining a speech-to-text model, a vision OCR model, and a text LLM—losing cross-modal nuance (like sarcastic tone of voice paired with a visual slide). Gemini was trained natively from the ground up across text, code, images, audio, and video with an industry-leading 1M to 2M+ token context window. Furthermore, teams often deploy heavyweight Gemini Pro for simple, high-volume classification where Gemini Flash delivers 10x better cost-latency efficiency, or forget Gemini Nano for on-device mobile execution.
2. Using a Closed Cloud API When Open Weights or Offline Execution Are Required (1.4b — Gemma): When a defense contractor, field researcher on an airplane, or edge IoT deployment requires running a lightweight model locally without internet connectivity—or requires full control to inspect and modify model weights on custom GKE/local hardware—calling the cloud-hosted Gemini API fails. That is the exact role of Gemma, Google's family of lightweight, state-of-the-art open-weight models built from the same research and technology as Gemini.
3. Confusing Image Synthesis (1.4c — Imagen) with Video Synthesis (1.4d — Veo): When a creative studio needs photorealistic studio product photography, brand-safe image inpainting/outpainting, or sharp typography rendering inside static graphics, Imagen (Google's text-to-image diffusion model family) is the right tool. When the requirement shifts to generating high-definition (1080p+) cinematic video clips with consistent temporal physics, camera pans, and motion from text or image prompts, Veo is required.

Consideration 1.4a: Gemini

• Native Multimodality from Pre-Training: Unlike models that bolt a vision adapter onto a text-only LLM, Gemini is natively multimodal—capable of ingesting and reasoning across interleaved text, code, images, audio, and video within a single prompt.
• Massive Context Window (1M–2M+ Tokens): Gemini can process hours of video, thousands of lines of repository code, or hundreds of pages of dense financial/legal PDFs in a single request without losing "needle-in-a-haystack" recall.
• The Gemini Size & Speed Tiers :
• Gemini Pro: Optimized for complex reasoning, cross-modal synthesis, advanced coding, and deep agentic planning.
• Gemini Flash ( & Flash-Lite): Engineered for high-volume, high-frequency, low-latency, and cost-sensitive workloads (e.g., millions of daily customer sentiment classifications, real-time chat routing, or rapid data extraction).
• Gemini Nano: Engineered for on-device mobile/edge execution (e.g., Android AICore) where zero network latency and local device privacy are paramount.

Key Implementation Patterns

• Native Multimodal Video Analysis: Use Use the Gemini API with the video file provided as a prompt input. Gemini is natively multimodal and can accept video files directly as input for analysis. (Avoid alternative: PaLM 2 is text-based and cannot process video files natively.)
• Model Selection for High-Volume Summarization: Use Configure the application to use the Gemini Flash model. Gemini Flash is the most cost-effective and low-latency model designed specifically for high-volume, high-frequency tasks with lower complexity. (Avoid alternative: Gemini Pro is optimized for complex reasoning and scaling, making it more expensive and less efficient for simple, high-volume tasks compared to Flash.)

Key Takeaways (1.4a)

• Gemini is a natively multimodal model capable of understanding text, images, audio, and video simultaneously.
• Native video support eliminates the need for separate frame extraction or speech-to-text pipelines.
• Legacy LLMs (like PaLM 2) are text-centric and require external tools to process non-text data.
• Gemini Flash is the go-to model for high-volume, low-latency, and cost-sensitive tasks.

Consideration 1.4b: Gemma

• What Makes Gemma Unique: Gemma is Google's family of lightweight, open-weight models derived from the same research and technology used to build Gemini. Unlike the closed-weight Gemini API, Gemma weights can be downloaded, inspected, fine-tuned, and run anywhere—on a developer's offline laptop, on-premises servers, or on Google Kubernetes Engine (GKE) / Vertex AI GPUs and TPUs.
• Ideal Business Use Cases for Gemma:
• Offline / Disconnected Environments: Running local summarization or coding assistance on laptops, ships, retail stores, or edge devices without internet access.
• Strict Weight Sovereignty & Custom Fine-Tuning: Regulated enterprises that require full ownership of fine-tuned model weights and custom serving stacks without external API dependencies.
• Specialized Variants: Includes CodeGemma (code completion/generation), PaliGemma (vision-language tasks), and ShieldGemma (safety content moderation).

Key Implementation Patterns

• Local Prototyping on Consumer Hardware: Use Download the Gemma model weights and run the model locally. Gemma models are open, lightweight models designed to run locally on developer hardware without internet access. (Avoid alternative: Gemini Pro is accessed via an API over the internet and does not run locally on consumer hardware.)
• Framework Agnostic Customization: Use Download a Gemma model and use the ready-to-use PyTorch implementation. Gemma is an open model family that natively supports PyTorch, JAX, and Keras 3.0. (Avoid alternative: Gemini is a managed model accessed via API, not an open model for local PyTorch workflows.)
• Air-Gapped Environment Deployment: Use Download the Gemma model weights and deploy them directly on your isolated infrastructure. Gemma offers open weights that can be downloaded and self-hosted in offline environments. (Avoid alternative: Gemini requires API connectivity (even if private), which violates the strict air-gapped requirement.)

Key Takeaways (1.4b)

• Gemma models are 'open' models designed for responsible AI development on local hardware.
• Gemini models are typically accessed via cloud APIs for high-performance, multimodal tasks.
• Local execution is ideal for prototyping without internet access or cloud costs.
• Gemma is Google's family of lightweight, open models.

Consideration 1.4c: Imagen

• Imagen (Text-to-Image & Visual Editing): Imagen is Google's flagship diffusion-based foundation model family for generating photorealistic images, studio product backgrounds, digital illustrations, and accurate embedded typography from natural-language prompts.
• Enterprise Capabilities on Vertex AI: Supports inpainting (inserting or replacing objects inside a masked region of an existing product photo), outpainting (expanding an image's canvas/aspect ratio for different ad formats), subject-driven customization (generating new scenes featuring a brand's specific product with just a few reference photos), and invisible SynthID digital watermarking for provenance.

Key Implementation Patterns

• Marketing Asset Generation: Use Use the Imagen model. Imagen is the specific Google Cloud foundation model designed for text-to-image generation. (Avoid alternative: Gemini is primarily a multimodal model for understanding and reasoning, not the specialized model for generating images.)
• Product Catalog Background Replacement: Use Use the image editing feature with a mask identifying the background and a text prompt for a white studio setting. Image editing with a mask (in-painting) allows you to replace specific parts of an image, such as the background, while preserving the rest.

Key Takeaways (1.4c)

• Identify the input and output modality (e.g., text-to-image) to select the correct foundation model.
• Imagen is the dedicated model family on Vertex AI for generating high-quality images from text.
• Image Editing (In-painting) is used to modify specific areas of an image while preserving the rest.
• Masks are binary or alpha-channel overlays that define which parts of an image a model should edit.

Consideration 1.4d: Veo

• Veo (Text-to-Video & Image-to-Video): Veo is Google's state-of-the-art generative video foundation model on Vertex AI, designed to synthesize high-definition (1080p+) video clips beyond a minute in length from text prompts or reference images.
• Strengths & Business Value: Understands cinematic terminology ("timelapse", "aerial drone shot", "shallow depth of field", "panning shot"), maintains temporal consistency (characters, objects, and physics remain coherent across frames), and enables marketing, media, and e-commerce teams to storyboard and produce studio-grade video assets in minutes—protected by SynthID watermarking.

Key Implementation Patterns

• Cinematic Vocabulary Understanding: Use Use Veo. Include the phrase "dolly zoom" in your text prompt to direct the camera movement. Veo is specifically trained to understand and execute cinematic vocabulary like "dolly zoom" in video generation. (Avoid alternative: Gemini Pro is a multimodal model for analysis and text generation, not specialized for generative video creation like Veo.)
• Masked Video Editing: Use Use the masked editing feature in Veo to select the car area and apply a text prompt for the change. Veo's masked editing feature allows you to modify specific objects in a video via text prompts without altering the rest of the scene. (Avoid alternative: Regenerating a video from scratch creates a new variation of the scene, changing the background and motion.)
• Image-to-Video Generation: Use Use Veo with the static image as input and a text prompt to animate the condensation. Veo supports Image-to-Video generation, allowing you to animate specific elements while preserving the visual fidelity of the input image. (Avoid alternative: Imagen is optimized for static image generation, not for creating temporally consistent video sequences from a reference image.)

Key Takeaways (1.4d)

• Veo is Google's generative video model capable of creating 1080p+ video clips.
• Veo understands advanced cinematic terms (e.g., "pan", "tilt", "dolly zoom") included in text prompts.
• Use Imagen for static images and Veo for moving video content.
• Distinguish between generative models (creating content) and analytical APIs (analyzing content).

Quick Decision Cheat Sheet (Subsection 1.4)

Further Reading (Subsection 1.4)

• Gemini - Google DeepMind — Overview of Gemini's capabilities, specifically highlighting its sophisticated multimodal reasoning and ability to make sense of complex written and visual information.
• Gemini Models — Overview of the Gemini model family, describing the differences between Ultra, Pro, Flash, and Nano variants.
• Gemma: State-of-the-art open models — Google's official announcement introducing Gemma, explaining its relationship to Gemini and its availability in 2B and 7B sizes for local development.
• Get started with Gemma using Keras — Technical documentation showing how to download and run Gemma locally.
• Welcome to Gemma — The official Gemma landing page describing it as a family of lightweight, state-of-the-art open models built from the same research and technology used to create the Gemini models.
• Model Garden on Vertex AI — Documentation detailing how Gemma can be used within Vertex AI and confirming its support for various frameworks.

Book TOC · Previous · Chapter 1 of 6 · Next