Microsoft Foundry for AI-103: Hubs, Projects, Model Catalog, Serverless vs Managed Compute, Azure OpenAI, Playground, Prompt Flow, Connections, Endpoints, Quotas, and the AI Development Lifecycle

Leave a Comment / AI/ML, Azure

Table of Contents

In the AI-103 Study Guide, we mapped all five exam domains. This post dives deep into the platform that underpins everything: Microsoft Foundry. Every AI-103 domain assumes you know Foundry — agents are built in Foundry, RAG pipelines are orchestrated through Foundry, evaluations run in Foundry, and models are deployed from Foundry. Understanding the Foundry architecture is like understanding the Azure portal before taking any Azure exam — it is the foundation everything else sits on.

Analogy — A professional kitchen. Microsoft Foundry is a professional restaurant kitchen. The Hub is the building — it provides plumbing (networking), electrical (compute), fire safety (content safety), and health inspections (Responsible AI). The Project is an individual chef’s station within that kitchen — each chef has their own prep area, their own ingredients (connections to data), and their own dishes being prepared (model deployments). The Model Catalog is the wholesale supplier’s catalog — 1,800+ ingredients (models) from dozens of suppliers (OpenAI, Meta, Mistral, Microsoft). You pick what you need, and it gets delivered to your station (deployed to your project). The Playground is the tasting counter where you try a dish before putting it on the menu. And Prompt Flow is the recipe card that chains steps together: prep the ingredients (retrieve data), cook the dish (call the model), plate it (format the output), and inspect it (evaluate quality).

What Is Microsoft Foundry

Microsoft Foundry is the unified platform for building, deploying,
and managing AI applications and agents on Azure.

What it brings together:
  - Model Catalog: discover and deploy 1,800+ models from OpenAI, Meta, Mistral, and others
  - Agent Service: build AI agents with tools, memory, and multi-agent orchestration
  - Prompt Flow: visual and code-based AI workflow orchestration
  - Evaluation: built-in quality and safety evaluators
  - Content Safety: content filters, prompt shields, groundedness detection
  - Monitoring: tracing, logging, and observability via Application Insights
  - Connections: link to Azure AI Search, Blob Storage, databases, and external APIs
  - Deployment: managed endpoints with quotas, scaling, and versioning

What it replaces:
  - Azure AI Studio (renamed to Microsoft Foundry in 2025-2026)
  - Azure OpenAI Studio (now part of Foundry)
  - Individual service portals (Vision, Language, Speech now accessible through Foundry)
  - Azure Machine Learning Studio (ML model management now integrated)

The key insight:
  Before Foundry, you managed each Azure AI service separately.
  With Foundry, everything is in one place -- one portal, one SDK,
  one set of connections, one deployment model.

The Evolution — Azure AI Studio to Microsoft Foundry

Timeline:
  2023: Azure AI Studio launched (preview)
       Unified portal for Azure OpenAI + Azure AI services

  2024: Azure AI Studio renamed to Azure AI Foundry (preview)
       Hub/Project architecture introduced
       Model Catalog expanded to 1,000+ models
       Agent Service introduced (preview)

  2025-2026: Azure AI Foundry renamed to Microsoft Foundry (GA)
       1,800+ models in catalog
       Responses API for agents (GA)
       Content Understanding (new service)
       Foundry resources replace Hub-based resources for new projects
       AI-103 exam launched (April 2026)

Why the name changes matter for AI-103:
  The exam uses "Microsoft Foundry" throughout
  Documentation may still reference "Azure AI Studio" or "Azure AI Foundry"
  They are the same platform -- just different names at different points in time
  If a question mentions "Foundry portal" or "AI Foundry" -- it is the same thing

Foundry Architecture — Hubs, Projects, and Resources

Two architecture models coexist in 2026:

  MODEL 1: Hub-Based (classic)
    AI Hub (shared governance container)
      ├── AI Project A (team/app workspace)
      ├── AI Project B (another team)
      └── AI Project C (another app)

    Hub provides: shared compute, networking, connections, managed identity
    Projects provide: isolated deployments, connections, endpoints per team

  MODEL 2: Foundry Resource (new, recommended for new projects)
    Foundry Resource (replaces Hub for simpler setups)
      └── Project (directly inside the Foundry resource)

    No separate Hub needed
    Simpler setup for single-team or single-app scenarios

  When to use which:
    Hub-Based: enterprise teams with multiple projects sharing governance
    Foundry Resource: single project, quick start, simpler architecture
    AI-103 tests BOTH models -- know the difference

  Hierarchy comparison:
    Hub-Based:    Azure Subscription → Resource Group → AI Hub → AI Project → Deployments
    Foundry:      Azure Subscription → Resource Group → Foundry Resource → Project → Deployments

Analogy — Apartments vs houses. A Hub-based architecture is like an apartment building: multiple tenants (projects) share the building’s infrastructure (networking, security, compute). A Foundry Resource is like a single-family house: one project, self-contained, simpler to manage. The apartment building is better for enterprise teams; the house is better for individual projects.

What lives where:

  At the Hub / Foundry Resource level:
    - Networking configuration (private endpoints, VNets)
    - Managed identity (system-assigned or user-assigned)
    - Shared connections (Azure AI Search, Blob Storage)
    - Content Safety configuration
    - Compute resources

  At the Project level:
    - Model deployments (endpoints)
    - Project-specific connections
    - Prompt Flow definitions
    - Evaluation runs
    - Agent configurations
    - Project-specific RBAC

Creating a Foundry Resource and Project

Method 1: Azure Portal
  1. Navigate to Azure Portal → Create a resource
  2. Search "Azure AI Foundry" or "Microsoft Foundry"
  3. Choose: Foundry resource (new) or AI Hub (classic)
  4. Configure:
     - Name: "foundry-mycompany-dev"
     - Region: Canada Central (choose based on model availability)
     - Resource Group: rg-ai-dev
     - Managed Identity: System-assigned (recommended)
  5. Create → navigate to the Foundry portal (ai.azure.com)
  6. Create a Project within the resource

Method 2: Foundry Portal (ai.azure.com)
  1. Go to ai.azure.com
  2. Click "Create project"
  3. Provide project name and select/create a Foundry resource
  4. Configure connections (Azure AI Search, Storage)
  5. Start building

Method 3: Azure CLI / Bicep / Terraform
  az ai foundry create --name "foundry-dev" --resource-group "rg-ai-dev" --location "canadacentral"
  az ai project create --name "myproject" --resource "foundry-dev"

  Or via Terraform (from our Terraform post):
  resource "azurerm_ai_services" "foundry" {
    name                = "foundry-dev"
    resource_group_name = azurerm_resource_group.main.name
    location            = "canadacentral"
    sku_name            = "S0"
  }

The Model Catalog — 1,800+ Models in One Place

The Model Catalog is the searchable directory of every deployable model in Foundry. It includes models from Microsoft, OpenAI, Meta, Mistral, Cohere, Anthropic, and many others.

Analogy — A supermarket with 1,800 products. You do not need to visit separate stores for bread (OpenAI GPT-4o), milk (Meta Llama 3.1), and eggs (Mistral Large). The catalog is a one-stop shop. Each product has a label (model card) with ingredients (capabilities), nutrition facts (benchmarks), and pricing. Some products are sold by the item (pay-per-token serverless), others require a subscription (managed compute).

Model categories in the catalog:

  Language models (text generation, chat, reasoning):
    OpenAI: GPT-4o, GPT-4o-mini, o1, o3-mini
    Microsoft: Phi-4, Phi-3.5
    Meta: Llama 3.1 (8B, 70B, 405B), Llama 3.2
    Mistral: Mistral Large, Mixtral 8x7B
    Cohere: Command R+
    DeepSeek: DeepSeek-V2

  Embedding models (for RAG / vector search):
    OpenAI: text-embedding-3-large, text-embedding-3-small
    Cohere: embed-v3

  Image models (generation and analysis):
    OpenAI: DALL-E 3
    Stability AI: Stable Diffusion

  Speech models:
    OpenAI: Whisper (speech-to-text)

  Multimodal models (text + image):
    OpenAI: GPT-4o (accepts images and text)
    Meta: Llama 3.2 Vision

  Each model card shows:
    - Capabilities (what it can do)
    - Benchmarks (performance on standard tests)
    - Deployment options (serverless, managed compute)
    - Pricing (per 1M input/output tokens or per compute hour)
    - License terms (open-weight, commercial, restricted)
    - Region availability

Deployment Types — Serverless API vs Managed Compute

Once you choose a model, you deploy it to an endpoint. There are two deployment types:

Serverless API (recommended for most use cases):
  What: model hosted by Microsoft/provider in managed infrastructure
  Billing: pay-per-token (e.g., $2.50 per 1M input tokens for GPT-4o)
  Setup: click deploy → get an endpoint URL → start calling the API
  Scaling: automatic (Microsoft handles scaling)
  Infrastructure: zero (no GPUs to manage)
  Best for: GPT-4o, Mistral, Cohere -- any model where you just need the API

  Advantages:
    - Zero infrastructure management
    - Pay only for what you use
    - Instant deployment
    - Same API across all models (Azure AI Model Inference API)

  Limitations:
    - Cannot fine-tune (some models support fine-tuning separately)
    - Cannot inspect model weights
    - Token-per-minute (TPM) quotas apply
    - Limited to models that support serverless

Managed Compute (for advanced scenarios):
  What: model deployed on dedicated GPU VMs in YOUR subscription
  Billing: per compute hour (VM running time)
  Setup: select VM size → deploy model → manage scaling
  Scaling: manual or autoscale rules
  Infrastructure: you manage (VM sizes, disk, networking)
  Best for: open-weight models you want to fine-tune, custom models, high-throughput

  Advantages:
    - Full control over compute
    - Fine-tune models on your data
    - No token quotas (limited by GPU capacity)
    - Run completely within your VNet (data never leaves your network)

  Limitations:
    - Higher cost (paying for VMs even when idle)
    - More complex setup and management
    - Requires GPU quota approval from Microsoft
    - You handle scaling, monitoring, and failover
FeatureServerless APIManaged Compute
BillingPay-per-tokenPay-per-hour (VM)
SetupClick deploy, get URLSelect VM, configure scaling
InfrastructureZero (Microsoft manages)You manage GPUs
ScalingAutomaticManual or autoscale
Fine-tuningLimitedFull support
NetworkingPublic endpoint (or private)VNet-integrated
QuotaTPM limitsGPU quota
Best forAPI consumption, chat, RAGFine-tuning, custom models, high-throughput

Azure OpenAI Within Foundry

Azure OpenAI is now a PART of Microsoft Foundry, not a separate service.

Before Foundry:
  Azure OpenAI had its own portal (oai.azure.com)
  Separate resource, separate keys, separate management
  You managed Azure OpenAI and other AI services independently

With Foundry:
  Azure OpenAI models appear in the Foundry Model Catalog
  Deployments happen within a Foundry project
  Same API (Azure AI Model Inference API) for OpenAI and non-OpenAI models
  Content Safety applies uniformly
  One portal for everything

Azure OpenAI-specific deployment types within Foundry:
  Standard: shared compute, pay-per-token, best general option
  Provisioned (PTU): reserved throughput, pay-per-provisioned-unit
  Global: routes to the nearest available region for lowest latency
  Data Zone: keeps data within a geographic boundary (EU, US)

Choosing Azure OpenAI deployment type:
  Low/variable traffic → Standard (pay-per-token)
  High/consistent traffic → Provisioned (reserved capacity, predictable cost)
  Global users → Global deployment (lowest latency)
  Data residency requirements → Data Zone deployment

Model Selection — Choosing the Right Model for the Task

Decision framework:

  TASK: General chat, instruction following, reasoning
    → GPT-4o (best quality, highest cost)
    → GPT-4o-mini (good quality, lower cost)
    → Phi-4 (small, fast, cost-effective for simpler tasks)
    → Llama 3.1 70B (open-weight alternative to GPT-4)

  TASK: Embedding text for vector search (RAG)
    → text-embedding-3-large (highest quality, 3072 dimensions)
    → text-embedding-3-small (good quality, 1536 dimensions, lower cost)
    → Cohere embed-v3 (multilingual alternative)

  TASK: Code generation and analysis
    → GPT-4o (strong at code across languages)
    → DeepSeek Coder (specialized for code)
    → Phi-4 (good for code at lower cost)

  TASK: Image understanding (describe, analyze, OCR)
    → GPT-4o (multimodal -- accepts images and text)
    → Llama 3.2 Vision (open-weight multimodal)

  TASK: Image generation
    → DALL-E 3

  TASK: Speech-to-text
    → Whisper (OpenAI)
    → Azure AI Speech service (more configuration options)

  Cost vs quality tradeoff:
    GPT-4o: highest quality, ~$2.50/$10 per 1M input/output tokens
    GPT-4o-mini: 80% quality, ~$0.15/$0.60 per 1M tokens (16x cheaper)
    Phi-4: 70% quality, ~$0.07/$0.14 per 1M tokens (35x cheaper)

  For AI-103: know WHEN to recommend each model type, not just WHAT they are.

The Foundry Playground — Testing Before Building

The Playground is an interactive testing environment within the Foundry portal.
No code required -- type a prompt, see the response, iterate.

Playground modes:
  Chat: multi-turn conversation with a deployed model
  Completion: single-turn text generation
  Image: generate or analyze images
  Speech: speech-to-text and text-to-speech

What you can configure in the Playground:
  System message: "You are a helpful data engineering assistant..."
  Temperature: 0.0 (deterministic) to 2.0 (creative)
  Top-p: nucleus sampling (0.1 = conservative, 1.0 = full vocabulary)
  Max tokens: limit response length
  Stop sequences: tokens that end generation
  Frequency penalty: reduce repetition
  Presence penalty: encourage topic diversity
  Seed: reproducible outputs (same seed = same output)
  Response format: text or JSON mode

Testing RAG in the Playground:
  1. Add your data: connect Azure AI Search index or upload files
  2. Configure retrieval: vector search, semantic search, or hybrid
  3. Ask questions: the model uses your data for grounding
  4. Evaluate: check if responses are grounded in your data

  This is the "On Your Data" feature -- quick RAG without building a pipeline.
  Useful for prototyping. For production, use Prompt Flow or the Agent Service.

Connections — Linking External Data and Services

Connections link your Foundry project to external data sources and services.

Connection types:
  Azure AI Search: for RAG retrieval (vector/semantic/hybrid search)
  Azure Blob Storage: for document ingestion and file access
  Azure Cosmos DB: for structured data
  Azure SQL Database: for relational data
  Azure AI Services: for Vision, Language, Speech, Document Intelligence
  Azure Key Vault: for secrets management
  Custom: any REST API endpoint

Creating a connection:
  Foundry portal → Project → Settings → Connections → New Connection
  Or via SDK:

  from azure.ai.projects import AIProjectClient
  client = AIProjectClient(endpoint="https://your-foundry.api.azureml.ms",
                            credential=DefaultAzureCredential())

Authentication for connections:
  Managed Identity (recommended): no secrets to manage
  API Key: simple but requires rotation
  Connection String: for specific services (SQL, Cosmos DB)
  Service Principal: for CI/CD and automation

For AI-103: know when to use each connection type
  and how to authenticate securely (managed identity preferred).

Prompt Flow — Orchestrating AI Workflows

Analogy — A factory assembly line for AI. Each station on the line does one thing: Station 1 retrieves documents (AI Search node), Station 2 formats the prompt (Python node), Station 3 calls the model (LLM node), Station 4 checks safety (Content Safety node), Station 5 formats the response (Python node). The assembly line runs automatically, and you can swap out any station without rebuilding the whole line.

Prompt Flow is the orchestration engine in Foundry.
It chains together multiple steps into a single AI workflow.

Node types:
  LLM: call a language model (GPT-4o, Llama, Mistral)
  Python: run custom Python code (data processing, formatting)
  Tool: call an external tool (AI Search, Bing, custom API)
  Prompt: template a prompt with variables

Flow types:
  Standard Flow: input → process → output (single request)
  Chat Flow: multi-turn conversation with history
  Evaluation Flow: run quality/safety checks on model outputs

Example RAG flow:
  1. Input: user question
  2. AI Search node: retrieve relevant documents
  3. Python node: format retrieved documents + question into a prompt
  4. LLM node: call GPT-4o with the prompt
  5. Content Safety node: check response for harmful content
  6. Output: return the safe, grounded response

Deploying a Prompt Flow:
  Build and test in the Foundry portal
  → Deploy as a managed endpoint (gets a URL)
  → Call from your application via REST API
  → Monitor with Application Insights

Endpoints and APIs — Consuming Deployed Models

After deploying a model, you get an endpoint URL to call via REST API or SDK.

REST API example:
  POST https://your-foundry.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21
  Headers:
    Authorization: Bearer <token>  (or api-key: <key>)
    Content-Type: application/json
  Body:
    {
      "messages": [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain Delta Lake in one sentence."}
      ],
      "temperature": 0.7,
      "max_tokens": 150
    }

Python SDK example:
  from openai import AzureOpenAI

  client = AzureOpenAI(
      azure_endpoint="https://your-foundry.openai.azure.com",
      api_key="your-api-key",  # or use DefaultAzureCredential
      api_version="2024-10-21"
  )

  response = client.chat.completions.create(
      model="gpt-4o",
      messages=[
          {"role": "system", "content": "You are a helpful assistant."},
          {"role": "user", "content": "Explain Delta Lake in one sentence."}
      ],
      temperature=0.7,
      max_tokens=150
  )

  print(response.choices[0].message.content)

Azure AI Model Inference API:
  Unified API that works across ALL models in the catalog
  Same code for GPT-4o, Llama, Mistral -- just change the deployment name
  Simplifies switching models without rewriting application code

Quotas, TPM Limits, and Cost Management

Quotas control how much you can use:

  Tokens Per Minute (TPM): limits how many tokens per minute per deployment
    Standard: varies by model and region (e.g., GPT-4o: 30K-450K TPM)
    Provisioned: you purchase a fixed amount (PTUs)
    Shared across all users of the deployment

  Requests Per Minute (RPM): limits API calls per minute
    Typically tied to TPM (e.g., 1K TPM ≈ 6 RPM for short prompts)

  Increasing quotas:
    Azure Portal → Azure OpenAI → Quotas → Request increase
    Or contact Microsoft support for large-scale increases

  Cost management:
    Monitor spend: Azure Cost Management → filter by AI Foundry resource
    Set budgets and alerts: alert at 80% of monthly budget
    Use GPT-4o-mini or Phi-4 for development/testing (10-35x cheaper)
    Reserve capacity (PTU) for predictable workloads (30-50% savings)
    Delete unused deployments (managed compute bills even when idle)

  For AI-103:
    Know the difference between Standard and Provisioned deployments
    Know that TPM quotas are per-deployment, per-region
    Know that managed compute costs are per-hour (even when idle)
    Know how to request quota increases

The AI Development Lifecycle in Foundry

The complete lifecycle in one platform:

  1. DISCOVER
     Browse Model Catalog → compare models → read benchmarks
     Try models in the Playground → quick experiments

  2. BUILD
     Create a Project → connect data sources
     Build Prompt Flow (orchestration chain)
     Configure agents with tools and memory
     Set up RAG with Azure AI Search connection

  3. EVALUATE
     Run evaluations: groundedness, relevance, coherence, fluency, safety
     Compare model configurations (GPT-4o vs Phi-4, temperature 0.3 vs 0.7)
     Use evaluation flows on test datasets

  4. DEPLOY
     Deploy as managed endpoint (Prompt Flow or model deployment)
     Configure scaling (serverless auto-scales, managed compute manual)
     Set up content safety filters

  5. MONITOR
     Application Insights: tracing, latency, error rates
     Content Safety logs: blocked prompts and responses
     Cost monitoring: token consumption, compute hours
     Model performance: track evaluation metrics over time

  6. ITERATE
     A/B test new model versions
     Update Prompt Flow with improved prompts
     Retrain or fine-tune based on evaluation results
     Promote through environments (dev → staging → prod)

Common Mistakes

  1. Confusing Foundry Resources with AI Hubs. Foundry Resources are the newer, simpler architecture for single-project setups. AI Hubs are the classic architecture for multi-project enterprise scenarios with shared governance. AI-103 tests both. Know when to use each: Hub-based for teams sharing networking and compute, Foundry Resource for standalone projects.

  2. Deploying to managed compute when serverless would suffice. Managed compute requires GPU quota, costs per hour even when idle, and requires scaling management. For most chat, RAG, and agent scenarios, serverless API deployment (pay-per-token) is simpler and cheaper. Use managed compute only for fine-tuning, custom models, or when you need data to stay within your VNet.

  3. Using the Playground for production. The Playground is for testing and prototyping. For production, deploy models as endpoints via Prompt Flow or the Agent Service. The Playground does not provide the reliability, monitoring, or scaling needed for production traffic.

  4. Not understanding the Model Inference API. The Azure AI Model Inference API provides a unified interface across all models in the catalog. You can switch from GPT-4o to Llama 3.1 by changing the deployment name, not your code. AI-103 expects you to know this unified API pattern.

  5. Ignoring TPM quotas during capacity planning. A deployment with a 30K TPM limit can handle roughly 180 short requests per minute. If your application serves 500 users simultaneously, you need a higher quota or multiple deployments with load balancing. AI-103 tests whether you can calculate and plan for token quotas.

  6. Choosing models based on benchmarks alone without considering cost. GPT-4o scores highest on most benchmarks, but at $2.50/1M input tokens, it is 16x more expensive than GPT-4o-mini. For internal tools, development, and non-critical use cases, smaller models provide 80% of the quality at a fraction of the cost. AI-103 tests cost-aware model selection.

  7. Not configuring connections with managed identity. Using API keys for connections means secrets to rotate, potential leaks, and manual key management. Managed identity (system-assigned or user-assigned) eliminates secrets entirely and is the recommended authentication method for all Foundry connections. AI-103 consistently prefers managed identity answers.

  8. Skipping the Foundry Playground during exam preparation. The Playground lets you experiment with every configurable parameter (temperature, top-p, system message, response format, content filters) in minutes. Exam questions reference these parameters — understanding them practically (not just theoretically) makes scenario questions much easier.

Interview Questions

Q: What is Microsoft Foundry and what does it replace? A: Microsoft Foundry is the unified platform for the entire AI application lifecycle on Azure: model discovery, deployment, orchestration, evaluation, safety, and monitoring. It replaces Azure AI Studio (the original name), Azure OpenAI Studio (now integrated), and individual service portals for Vision, Language, Speech, and Document Intelligence. Everything is managed from one portal (ai.azure.com) with one SDK, one set of connections, and unified identity management.

Q: What is the difference between a Foundry Resource and an AI Hub? A: A Foundry Resource is the newer, simpler architecture for single-project setups — one resource directly contains one project. An AI Hub is the classic architecture for enterprise scenarios where multiple projects share governance, networking, compute, and connections. The Hub provides shared infrastructure; each project within it gets isolated deployments and endpoints. New projects should use Foundry Resources unless multi-project governance is needed.

Q: What are the two deployment types for models and when would you use each? A: Serverless API deploys models on Microsoft-managed infrastructure with pay-per-token billing. It requires zero infrastructure management and auto-scales. Use it for most chat, RAG, and agent scenarios. Managed Compute deploys models on dedicated GPU VMs in your subscription with per-hour billing. Use it for fine-tuning, custom models, high-throughput workloads, and when data must stay within your VNet. The choice is cost vs control: serverless is simpler and cheaper for variable workloads; managed compute is better for sustained high-volume or fine-tuning.

Q: How does the Model Catalog help with model selection? A: The Model Catalog lists 1,800+ models with model cards showing capabilities, benchmarks, pricing, deployment options, license terms, and region availability. It enables side-by-side comparison without deploying models first. For AI-103, the key is knowing when to recommend which model: GPT-4o for highest quality, GPT-4o-mini for cost efficiency, text-embedding-3-large for RAG embeddings, Phi-4 for lightweight tasks, and open-weight models (Llama, Mistral) when data must stay in your infrastructure.

Q: What is Prompt Flow and how does it fit into the AI development lifecycle? A: Prompt Flow is Foundry’s orchestration engine for chaining multiple AI steps into a single workflow. It supports LLM nodes (model calls), Python nodes (custom code), and tool nodes (AI Search, APIs). Flow types include standard (single request), chat (multi-turn), and evaluation (quality testing). You build flows visually or in code, test in the Foundry portal, deploy as managed endpoints with a REST API, and monitor with Application Insights. It sits between model deployment and production serving — it is how you turn a raw model into a complete AI application.

Q: What are connections in Foundry and how should they be authenticated? A: Connections link a Foundry project to external data sources and services: Azure AI Search (for RAG), Blob Storage (for documents), SQL Database, Cosmos DB, Key Vault, and custom APIs. Authentication options include managed identity (recommended — no secrets to manage), API keys (simple but requires rotation), connection strings, and service principals (for CI/CD). AI-103 consistently prefers managed identity for connections because it eliminates secret management, reduces security risk, and integrates natively with Azure RBAC.

Q: How do you manage cost and quotas for Azure OpenAI deployments? A: Serverless deployments are limited by Tokens Per Minute (TPM) quotas per deployment per region. Standard deployments bill per token consumed. Provisioned deployments (PTUs) offer reserved throughput at lower per-token cost for high-volume workloads. To manage cost: use smaller models (GPT-4o-mini, Phi-4) for development, set Azure budget alerts, delete unused managed compute deployments, and request quota increases for production workloads. Monitor consumption through Azure Cost Management filtered by the Foundry resource.

Wrapping Up

Microsoft Foundry is the platform that AI-103 is built on. Every domain — from planning to agents to vision to extraction — works through Foundry. The Model Catalog gives you 1,800+ models. Serverless and managed compute give you deployment options. Connections link your data. Prompt Flow chains it all together. And the Playground lets you experiment before committing.

The key Foundry concepts for AI-103: Hub vs Project vs Foundry Resource architecture, serverless vs managed compute deployment, the unified Model Inference API, connections with managed identity, and the discover-build-evaluate-deploy-monitor lifecycle. Understand these, and every subsequent domain makes sense.

In the next post, we cover the other half of Domain 1: security, networking, monitoring, Responsible AI, and Content Safety — the governance layer that wraps around everything Foundry does.

Related posts: – AI-103 Study Guide – Azure Key Vault – Azure RBAC – Azure Networking – Fine-Tuning LLMs

Related Posts

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top