Documentation: Azure OpenAI & AI Services

This document provides further details and context for the Azure AI section of the Azure Digital Natives Guide.

  • Follow guidance for using your own data with AOAI

  • Why: Retrieval-Augmented Generation (RAG) patterns allow you to ground LLM responses in your specific data, improving relevance and accuracy. Implementing this securely and effectively is crucial.
  • How: Explore Azure OpenAI’s “on your data” feature or implement custom RAG solutions using services like Azure AI Search to index your data and provide relevant context to the LLM during generation.
  • Resources:

  • Understand AOAI data processing and storage

  • Why: It’s essential to know how your prompts, completions, embeddings, and training data (if applicable for fine-tuning) are processed and stored by the Azure OpenAI service to meet compliance and privacy requirements.
  • How: Review the official Azure OpenAI data privacy and security documentation.
  • Resources:

  • Monitor AOAI data residency, concurrency, and cost

  • Why: As you scale your use of AOAI, these operational factors become critical.
  • How:
    • Data Residency: Understand where your data is processed and stored based on the Azure region you deploy AOAI to.
    • Concurrency: Monitor token usage (Prompt + Completion tokens) and manage quotas (Tokens-Per-Minute, Requests-Per-Minute) to ensure your application scales appropriately. Implement retry logic and potentially provisioned throughput for high-scale scenarios.
    • Cost: Track token consumption closely as it directly impacts cost. Optimize prompts and leverage different models based on cost/performance trade-offs.
  • Resources:

  • Implement Responsible AI practices

  • Why: AI systems can produce harmful, biased, or inaccurate content. Responsible AI practices ensure your applications are fair, transparent, and safe for users.
  • How: Enable content filtering on Azure OpenAI deployments. Implement human-in-the-loop patterns for high-stakes decisions. Review the Microsoft Responsible AI Standard and apply it to your AI workloads. Use Azure AI Content Safety to detect harmful content.
  • Resources:

  • Explore Azure AI Foundry for end-to-end AI development

  • Why: Azure AI Foundry provides a unified platform for building, evaluating, and deploying AI applications, including model catalog, prompt flow, and evaluation tools — going beyond raw model access.
  • How: Use Azure AI Foundry to explore the model catalog, build prompt flows for orchestration, evaluate model outputs for quality and safety, and deploy AI solutions with built-in monitoring.
  • Resources:

📚 Recommended Reading