1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model
17. Dec 17: Microsoft Azure AI – Azure OpenAI service
18. Dec 18: Microsoft Azure AI – Azure AI Hub and Azure AI Project
19. Dec 19: Microsoft Azure AI – Azure AI Foundry management center
20. Dec 20: Microsoft Azure AI – Models and endpoints in Azure AI Foundry
21. Dec 21: Microsoft Azure AI – Prompt flow in Azure AI Foundry
22. Dec 22: Microsoft Azure AI – Prompt flow using VS Code and Python
23. Dec 23: Microsoft Azure AI – Tracing in Azure AI Foundry
24. Dec 24: Microsoft Azure AI – Evaluation in Azure AI Foundry

Documentation and material for Azure AI Foundry are plentiful and growing on a daily basis, since the topic on AI and GenAI is evermore so popular.

General and best way to start is with Microsoft own product introduction website: https://azure.microsoft.com/en-us/products/ai-studio and later you can scale with different material from many different websites.

Detailed product documentation is on Microsoft learn: https://learn.microsoft.com/en-us/azure/ai-studio/

Video series:

• Udemy classes: https://www.udemy.com/course/azure-ai-studio-mastery-llmops-and-more/
• LinkedIn classes: https://www.linkedin.com/learning/topics/azure-ai-services
• Youtube: https://www.youtube.com/watch?v=nC766APIZFM

Learning paths:

• Introduction to AI Foundry: https://learn.microsoft.com/en-us/training/modules/introduction-to-azure-ai-studio/
• Udacity: https://www.udacity.com/course/azure-generative-ai-engineer–nd444

Books:

• Programming Large Language Models with Azure Open AI (https://www.amazon.com/Programming-Large-Language-Models-Azure/dp/0138280371)
• Azure OpenaAI Services for Cloud Native Applications (https://www.amazon.com/Azure-OpenAI-Service-Native-Applications/dp/1098154991)
• Microsoft Azure AI Fundamentals AI-900 Exam guide (https://www.amazon.com/Microsoft-Azure-Fundamentals-AI-900-Guide/dp/1835885667)
• Getting started with Azure OpenAI (https://www.amazon.com/Getting-Started-Azure-OpenAI-Deploying/dp/B0D8XRQKN9)

Code and samples:

• Github: https://github.com/MicrosoftLearning/mslearn-ai-studio
• Github: https://github.com/MicrosoftDocs/azure-ai-docs

Many introduction articles are available as well as many listings for the use cases.

All of the code samples will be available on my Github.

Merry Christmas and happy new year!

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model
17. Dec 17: Microsoft Azure AI – Azure OpenAI service
18. Dec 18: Microsoft Azure AI – Azure AI Hub and Azure AI Project
19. Dec 19: Microsoft Azure AI – Azure AI Foundry management center
20. Dec 20: Microsoft Azure AI – Models and endpoints in Azure AI Foundry
21. Dec 21: Microsoft Azure AI – Prompt flow in Azure AI Foundry
22. Dec 22: Microsoft Azure AI – Prompt flow using VS Code and Python
23. Dec 23: Microsoft Azure AI – Tracing in Azure AI Foundry

With evaluation you performing iterative, systematic evaluations with the right evaluators and measure and address potential response quality, safety, or security concerns throughout the AI development lifecycle, from initial model selection through post-production monitoring.

With the Evaluation in Azure AI Foundry, you can evaluation the GenAI Ops Lifecycle production. In addition, it also gives you the ability to  assess the frequency and severity of content risks or undesirable behavior in AI responses.

With each model, you can consider and evaluate:

• accuracy and quality of the model generated responses
• performance of specific tasks, as handling the prompts and content required for specific use cases
• bias and consider ethical bias to address or avoid harmful stereotypes or avoid promoting any other bias
• risk and safety to track and evaluate the unsafe or malicious content.

You also have:

• pre-production (with processes for testing datasets, finding edge cases, assessing robustness and measuring key metrics)
• post-production (with processes for finding edge cases and reporting it back, tracking and monitoring performance, analysing inappropriate outputs)

evaluation of the models.

You can evaluate:

With the model and prompt, you can select many different metrics:

And you can start with evaluation when the deployment is ready, all relevant metrics selected, prompts added and data integration (input and output) defined.

And the results of evaluation

And you can analyse the results with the Python SDK as well. Under the code, you will get the standard overview of the files, runs, logs, metrics definitions and displaying the results.

Tomorrow we will look into documentation for Azure AI Foundry.

All of the code samples will be available on my Github.

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model
17. Dec 17: Microsoft Azure AI – Azure OpenAI service
18. Dec 18: Microsoft Azure AI – Azure AI Hub and Azure AI Project
19. Dec 19: Microsoft Azure AI – Azure AI Foundry management center
20. Dec 20: Microsoft Azure AI – Models and endpoints in Azure AI Foundry
21. Dec 21: Microsoft Azure AI – Prompt flow in Azure AI Foundry
22. Dec 22: Microsoft Azure AI – Prompt flow using VS Code and Python

Tracing is a powerful tool that offers developers an in-depth understanding of the execution process of their generative AI applications. Though still in preview (in the time of writing this post), It provides a detailed view of the execution flow of the application and the essential information for debugging or optimisations.

Tracing with the Azure AI Inference SDK offers enhanced visibility and simplified troubleshooting for LLM-based applications, effectively supporting development, iteration, and production monitoring. Tracing follows the OpenTelemetry semantic conventions, capturing and visualizing the internal execution details of any AI application, enhancing the overall development experience.

Key advances of using tracing are:

• users can collect feedback logs and evaluate data to enrich trace data with additional insights
• get clear insights into the GenAI application
• gives options on sensitive data logging
• use Azure App insights
• simplified process for enabling traces for better improvements and productivity

When starting the tracing, create a new app or connect to an existing one.

The Kusto language for analysing the logs for attributes, content, spans, evaluation events and others. Excerpt for retrieving span attributes:

// Retrieve Gen AI span attributes within the specified date range
let gen_ai_spans = dependencies
| where isnotnull(customDimensions["gen_ai.system"]) 
| where timestamp between (datetime(2024-12-16T18:55:29.840Z) .. datetime(2024-12-23T18:56:18.214Z))
| project 
    operation_ParentId,
    operation_Id,
    customDimensions,
    id,
    duration,
    success,
    gen_ai_operation_start = timestamp,
    name,
    span_type = "GenAI",
    gen_ai_operation_name = tostring(customDimensions["gen_ai.operation.name"]),
    gen_ai_response_id = tostring(customDimensions["gen_ai.response.id"]),
    gen_ai_response_model = tostring(customDimensions["gen_ai.response.model"]),
    gen_ai_usage_input_tokens = tostring(customDimensions["gen_ai.usage.input_tokens"]),
    gen_ai_usage_output_tokens = tostring(customDimensions["gen_ai.usage.output_tokens"]);
// Helper function to determine Gen AI role based on event name
let get_role = (event_name: string) { 
    iff(event_name == "gen_ai.choice", "assistant", split(event_name, ".")[1])
};

With Python SDK you can also collect and analyse the log traces.

import os
from opentelemetry import trace

# Install opentelemetry with command "pip install opentelemetry-sdk".
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage, CompletionsFinishReason
from azure.core.credentials import AzureKeyCredential

# [START trace_setting]
from azure.core.settings import settings

settings.tracing_implementation = "opentelemetry"
# [END trace_setting]

# Setup tracing to console
# Requires opentelemetry-sdk
span_exporter = ConsoleSpanExporter()
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(span_exporter))
trace.set_tracer_provider(tracer_provider)

# [START trace_function]
from opentelemetry.trace import get_tracer

tracer = get_tracer(__name__)

This code part can be added to your regular Python code in order for tracing to be activated.

Tomorrow we will look into Evaluation in Azure AI Foundry.

All of the code samples will be available on my Github.

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model
17. Dec 17: Microsoft Azure AI – Azure OpenAI service
18. Dec 18: Microsoft Azure AI – Azure AI Hub and Azure AI Project
19. Dec 19: Microsoft Azure AI – Azure AI Foundry management center
20. Dec 20: Microsoft Azure AI – Models and endpoints in Azure AI Foundry
21. Dec 21: Microsoft Azure AI – Prompt flow in Azure AI Foundry

Prompt Flow is particularly beneficial for organisations leveraging AI to streamline operations, enhance customer experiences, and innovate in digital transformation projects.

Why Use Prompt Flow?

• Ease of Use: Simplifies the complex process of prompt engineering and integration.
• Scalability: Makes it easier to deploy and scale AI workflows across applications.
• Cost-Effectiveness: Helps optimize prompts, reducing unnecessary API calls and improving efficiency.

With Python you can start using prompt flow by installing the package

pip install promptflow

and create a prompty file (save it as my_prompty_file.yaml):

---
name: Minimal Chat
model:
  api: chat
  configuration:
    type: azure_openai
    azure_deployment: gpt-35-turbo
    api_key: ${env:AZURE_OPENAI_API_KEY}
    api_version: ${env:AZURE_OPENAI_API_VERSION}
    azure_endpoint: ${env:AZURE_OPENAI_ENDPOINT}
  parameters:
    temperature: 0.2
    max_tokens: 1024
inputs:
  question:
    type: string
sample:
  question: "Where can I get the most famous pasta in the world?"
---
system:
You are an AI assistant who helps people find information.
As the assistant, you answer questions briefly, succinctly,
and in a personable manner using markdown and even add some personal flair with appropriate emojis.

# Safety
- You **should always** reference factual statements to search results based on [relevant documents]
- Search results based on [relevant documents] may be incomplete or irrelevant. You do not make assumptions
# Customer
You are helping to find answers to their questions.
Use their name to address them in your responses.

user:
{{question}}

And for the environment variables:

from promptflow.core import Prompty

# Load prompty with dict override
override_model = {
    "configuration": {
        "api_key": "${env:AZURE_OPENAI_API_KEY}",
        "api_version": "${env:AZURE_OPENAI_API_VERSION}",
        "azure_endpoint": "${env:AZURE_OPENAI_ENDPOINT}"
    },
    "parameters": {"max_tokens": 512}
}
prompty = Prompty.load(source="path/to/my_prompty_file.yaml", model=override_model)

And the prompty file (YAML) file is to be specified in the Prompty.load() function. To orchestrate and call:

from promptflow.core import Prompty

prompty_obj = Prompty.load(source="path/to/my_prompty_file.yaml")
result = prompty_obj(first_name="John", last_name="Doh", question="What is the capital of France?")

Prompty gives you the ability to create an end-to-end solution, like RAG where you can chat with LLM over an article or document, where you can ask to classify the input data (list of URLs,…)

Prompty is a markdown file, structured in YAML and encapsulates a series of metadata fields pivotal for defining the model’s configuration and the inputs. After this front matter is the prompt template, articulated in the Jinja format.

You can run this from VS Code using SDK or as CLI and also use Trace UI to analyse the flow and analyse the runs.

Tomorrow we will look into feature Tracing in Azure AI Foundry.

All of the code samples will be available on my Github.

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model
17. Dec 17: Microsoft Azure AI – Azure OpenAI service
18. Dec 18: Microsoft Azure AI – Azure AI Hub and Azure AI Project
19. Dec 19: Microsoft Azure AI – Azure AI Foundry management center
20. Dec 20: Microsoft Azure AI – Models and endpoints in Azure AI Foundry

Prompt flow in Azure AI Foundry is development tool for designing the flows (streamlines) for the complete end-to-end development cycle of LLM’s AI application. You can create, iterate, test, orchestrate, debug, and monitor your flows.

 For additional information on Prompt flow: go to: https://microsoft.github.io/promptflow/index.html

With the prompt flow you will be able to do:

• collaborate with your team by using cloud version of Prompt flow (you can also clone the repository and use it on-prem)
• create flows that link LLMs, prompts and any other tools together for executable workflows
• use the power of Python language with the promptflow package
• evaluate your flows, calculate quality and performance metrics
• Integrate the testing and evaluation into your CI/CD systems
• Deploy flows to your serving platform (either prototype, experiment or finalised solution)

The main building blocks of Prompt flow are:

• A flow is an executable instruction set that can implement the AI logic.​​ Flows can be created or run via multiple tools, like a prebuilt canvas, LangChain, etcetera. Iterations of a flow can be saved as assets; once deployed a flow becomes an API. Not all flows are prompt flows; rather, prompt flow is one way to create a flow.
• A prompt is a package of input sent to a model, consisting of the user input, system message, and any examples. User input is text submitted in the chat window. System message is a set of instructions to the model scoping its behaviors and functionality.
• A sample flow is a simple, prebuilt orchestration flow that shows how flows work, and can be customized.
• A sample prompt is a defined prompt for a specific scenario that can be copied from a library and used as-is or modified in prompt design.

You can start building flow with the three predefined types (standard flow, chat flow or evaluation flow) or clone the prepared flows and deploy it based on your needs.

Web Classification

Web classification flow is an example available in the prompt flow gallery, for you to clone and experiment or deploy it to your application. It uses LLM to classify URL addresses into multiple categories.

And the graphical presentation of the flow:

Tomorrow we will look into building prompt flow in VS Code using Python.

All of the code samples will be available on my Github.

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model
17. Dec 17: Microsoft Azure AI – Azure OpenAI service
18. Dec 18: Microsoft Azure AI – Azure AI Hub and Azure AI Project
19. Dec 19: Microsoft Azure AI – Azure AI Foundry management center

Models from the model catalog can be deployed using programming languages or using the Foundry studio.

Model deployment has two types: Deploy from the base model or deploy from the fine-tuned model. The difference is that fine-tuned model is model taken from the model catalog and later tuned to an additional dataset, as the base model is the model as it is available in Azure AI Foundry.

Each model has all the necessary detail information, as well all the metrics to check number of requests, prompt token counts, number of requests and additional information on model usage.

Model from the model catalog can also be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription. Models deployed in Azure Machine Learning (AMLS) and Azure AI Foundry in Serverless API endpoints support the Azure AI Model inference API (read more here: https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-api ) and can be used by developers from inference or consuming the predictions.

Deployment with Python SDK is straightforward with Python and serverless API endpoint can be set with the MLClient:

from azure.ai.ml import MLClient
from azure.identity import InteractiveBrowserCredential
from azure.ai.ml.entities import MarketplaceSubscription, ServerlessEndpoint

client = MLClient(
    credential=InteractiveBrowserCredential(tenant_id="<tenant-id>"),
    subscription_id="<subscription-id>",
    resource_group_name="<resource-group>",
    workspace_name="<project-name>",
)

And an endpoint for the serverless:

endpoint_name="My_endpoint"

serverless_endpoint = ServerlessEndpoint(
    name=endpoint_name,
    model_id=model_id
)

created_endpoint = client.serverless_endpoints.begin_create_or_update(
    serverless_endpoint
).result()

endpoint_keys = client.serverless_endpoints.get_keys(endpoint_name)
print(endpoint_keys.primary_key)
print(endpoint_keys.secondary_key)

Tomorrow we will look into the prompt flow in Azure AI Foundry.

All of the code samples will be available on my Github.

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model
17. Dec 17: Microsoft Azure AI – Azure OpenAI service
18. Dec 18: Microsoft Azure AI – Azure AI Hub and Azure AI Project

The management center is a part of the Azure AI Foundry portal. It gives you the overview on activities and governance. From management center you can manage Azure AI Foundry hubs, projects, resources, access, endpoints, models and other resources and general settings.

General overview of management center:

Quota gives you capability to monitor and track your usage by subscription and region and per different models, like Azure OpenAI Standard, Azure OpenAI Provisioned, Azure OpenAI Global-Standard, Azure OpenAI Global-Batch.

You can use management center to create and configure both hubs and projects within hubs and view the resources. Both Hub and Project are similar within management center. You can access Users, Models, connected resources and compute.

Models + Endpoints will give you overview and monitoring of model versions, state additional filters, fine-tuning, capacities and rate limits (tokens per minute).

In connected resources you can create new connections or list all resources connected in selected hub.

In Compute you can spin up or shut down compute (VM machines) for purposes of your projects in the hub.

Creating is common task (same as in Fabric or Azure Machine Learning service).

And you can list all the compute instances or serverless instances.

Tomorrow we will look into the Models and endpoints in Azure AI Foundry.

All of the code samples will be available on my Github.

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model
17. Dec 17: Microsoft Azure AI – Azure OpenAI service

Azure AI Hub

In AI Foundry portal, hubs provide the environment for a team to collaborate and organize work, and help you as a team lead or IT admin centrally set up security settings and govern usage and spend. You can create and manage a hub from the Azure portal or from the AI Foundry portal, and then your developers can create projects from the hub.

In essence, Hubs are the primary top-level Azure resource for AI Foundry. Their purpose is to to govern security, connectivity, and computing resources across playgrounds and projects.

Once a hub is created, developers can create multiple projects from it and access any other shared shared resources.

Many of the resource can be created in hub and shared:

• Security configuration including public network access, V-net, VPN, customer-managed key encryption, and privileged access to whom can create projects for customization. Security settings configured on the hub automatically pass down to each project. A managed virtual network is shared between all projects that share the same hub.
• Connections are named and authenticated references to Azure and non-Azure resources like data storage providers. Use a connection as a means for making an external resource available to a group of developers without having to expose its stored credential to an individual.
• Compute and quota allocation is managed as shared capacity for all projects in AI Foundry portal that share the same hub. This quota includes compute instance as managed cloud-based workstation for an individual. The same user can use a compute instance across projects.
• AI services access keys to endpoints for prebuilt AI models are managed on the hub scope. Use these endpoints to access foundation models from Azure OpenAI, Speech, Vision, and Content Safety with one API Key.
• Policy enforced in Azure on the hub scope applies to all projects managed under it.
• Dependent Azure resources are set up once per hub and associated projects and used to store artifacts you generate while working in AI Foundry portal such as logs or when uploading data.

A hub will also provides the hosting environment for projects.

Azure AI Projects

Project workspaces that are created using a hub inherit the same security settings and shared resource access. Teams can create project workspaces as needed to organize their work, isolate data, and/or restrict access.

A project is also an organisational container that has tools for AI customisation and orchestration. It lets you organise your work, save state across different tools like prompt flow, and collaborate with others. For example, you can share uploaded files and connections to data sources.

Multiple projects can use a hub, and multiple users can use a project. A project also helps you keep track of billing, and manage access and provides data isolation. Every project uses dedicated storage containers to let you upload files and share it with only other project members when using the ‘data’ experiences.

Projects let you create and group reusable components that can be used across tools in AI Foundry portal:

• Data – all datasets that can be used to create indexes, fine-tune models, and evaluate models.
• Flows – are an executable instruction set that can implement the AI logic.
• Evaluations – of a model or flow. You can run manual or metrics-based evaluations.
• Indexes – Vector search indexes generated from your data.

You will find in your Azure resources type of resources: Hub and Project.

or within the Azure AI Foundry:

Tomorrow we will look into the the management center in Azure AI Foundry.

All of the code samples will be available on my Github.

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model

Let’s first address the elephant in the room. We have explored the Azure AI Foundry and the we have also Azure OpenAI. So what is the core difference? Let’s take a look:

The services in the back:

• Azure AI Services has much broader AI capabilities and simpler integration into applications and usage of the real world. With mostly pre-build API for all services (face recognition, document recognition, speech recognition, computer vision, image recognition, and more) that will allow better interoperabilty and and connection to machine learning services (Azure Machine Learning Service).
• Azure OpenAI is focusing primarly on OpenAI LLM models (Azure AI services supports many others) and provides great agents for conversations, content tools, RAG and natural language services.

UI:

• Azure AI Foundry (PreviousAzure AI Studio) contains AI Hubs, Hubs contain projects and projects contain AI resources (Search, Computer Vision, Models, etc). Azure OpenAI gives you access to the OpenAI models to use in the Studio as a part of a project.
• Azure OpenAI will give you the endpoints listed in the Azure OpenAI service details. And access to the various AI resources.

GenAI models:

• Models in Azure AI Services offers a variety of AI models and tools that can be fine-tuned, customised and applied (embedded) to your application or service. AI Service also offers many of the pre-trained models that can be used out-of-the-box or even later fine-tuned. Models include variety of services for image analysis, translation, and text analytics
• Models in Azure OpenAI are mainly the OpenAI’s LLMs and along those models are also the tasks one can use to create. From RAG systems. text generation, chat bots, summarisations, also to fine-tuning and some additional customisation. You will also find capabilities for image analysis and video analysis. But in general more limited in comparison to Azure AI Services with the model offerings.

There are also differences when it comes to integration with other azure services but in general it derives from the models each is offering.

Creating Azure OpenAI service, you can create a new by opening Azure OpenAI service in Azure portal.

and once you have the service up and running you will enter to the same UI (interface), that is Azure AI Foundry. Here you will have all the with the API capabilities, playground and other functionalities.

Tomorrow we will look into the Azure AI Project and Azure AI Hub.

All of the code samples will be available on my Github.

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK

Fine-tuning is the process of optimizing a pretrained model by training it on your specific dataset, which often contains more examples than you can typically fit in a prompt. Fine-tuning helps you achieve higher quality results for specific tasks, save on token costs with shorter prompts, and improve request latency.

The following models support fine-tuning:

• babbage-002
• davinci-002
• gpt-35-turbo (0613)
• gpt-35-turbo (1106)
• gpt-35-turbo (0125)
• gpt-4 (0613)
• gpt-4o (2024-08-06)
• gpt-4o-mini (2024-07-18)

First we need to find the model we would like to fine-tune. Under the model catalog, select the Fine-tune option:

As you get comfortable and begin building your solution, it’s important to understand where prompt engineering falls short and that will help you realize if you should try fine-tuning.

• Is the base model failing on edge cases or exceptions?
• Is the base model not consistently providing output in the right format?
• Is it difficult to fit enough examples in the context window to steer the model?
• Is there high latency?

Make sure that you are in the right region where fine-tune is available. Sweden-central usually works

Upload your data as shared resource:

or directly into the fine-tune

and change the task parameters:

The parameters explained:

And the fine-tune is available:

With the fine-tuned model available (after you trained it and you get the overview metrics of your fine-tuned model), you can deploy the model and start using it in the Chat -playground.

Overview of the running model:

Once completed, you will be able to get the fine-tune metrics.

Metrics of the fine-tuned model:

And you can deploy the model:

and start using it in chat playground:

Tomorrow we will look into the Azure OpenAI service.

All of the code samples will be available on my Github.