Duration
1 day
Overview
Azure AI Vision is a branch of artificial intelligence (AI) in which software interprets visual input, often from images or video feeds.
In this module, you'll learn how to use the Azure AI Vision service to extract information from images.
After completing this module, you’ll be able to:
- Provision an Azure AI Vision resource.
- Analyze an image.
- Generate a smart cropped thumbnail.
Outline
- Provision an Azure AI Vision resource
- The Azure AI Vision service is designed to help you extract information from images. It provides functionality that you can use for:
- Description and tag generation - determining an appropriate caption for an image, and identifying relevant "tags" that can be used as keywords to indicate its subject.
- Object detection - detecting the presence and location of specific objects within the image.
- People detection - detecting the presence, location, and features of people in the image.
- Image metadata, color, and type analysis - determining the format and size of an image, its dominant color palette, and whether it contains clip art.
- Category identification - identifying an appropriate categorization for the image, and if it contains any known landmarks.
- Background removal - detecting the background in an image and output the image with the background transparent or a greyscale alpha matte image.
- Moderation rating - determine if the image includes any adult or violent content.
- Optical character recognition - reading text in the image.
- Smart thumbnail generation - identifying the main region of interest in the image to create a smaller "thumbnail" version.
You can provision Azure AI Vision as a single-service resource, or you can use the Azure AI Vision API in a multi-service Azure AI Services resource.
- Analyze an image
- To analyze an image, you can use the Analyze Image REST method or the equivalent method in the SDK for your preferred programming language, specifying the visual features you want to include in the analysis (and if you select categories, whether or not to include details of celebrities or landmarks). This method returns a JSON document containing the requested information.
- Available visual features are contained in the VisualFeatures enum:
- VisualFeatures.Tags: Identifies tags about the image, including objects, scenery, setting, and actions
- VisualFeatures.Objects: Returns the bounding box for each detected object
- VisualFeatures.Caption: Generates a caption of the image in natural language
- VisualFeatures.DenseCaptions: Generates more detailed captions for the objects detected
- VisualFeatures.People: Returns the bounding box for detected people
- VisualFeatures.SmartCrops: Returns the bounding box of the specified aspect ratio for the area of interest
- VisualFeatures.Read: Extracts readable text
- Specifying the visual features you want analyzed in the image determines what information the response will contain. Most responses will contain a bounding box (if a location in the image is reasonable) or a confidence score (for features such as tags or captions).
- Generate a smart-cropped thumbnail and remove background
- Thumbnails are often used to provide smaller versions of images in applications and websites. For example, a tourism site might display a list of tourist attractions in a city with a small, representative thumbnail image for each attraction; and only display the full image when the user selects the "details" page for an individual attraction.
- The Azure AI Vision service enables you to create a thumbnail with different dimensions (and aspect ratio) from the source image, and optionally to use image analysis to determine the region of interest in the image (its main subject) and make that the focus of the thumbnail. This ability to determine the region of interest is especially useful when cropping the image to change its aspect ratio.
- Remove image background
- The background removal feature can split the image into the subject in the foreground, and everything else that is considered background. Azure AI Vision achieves this feature by creating an alpha matte of the foreground subject, which is then used to return either the foreground or the background.
- Lab Exercise - Analyze images with Azure AI Vision
Summary
In this module, you learned how to use the Azure AI Vision service to extract information from images.
Now that you've completed this module, you can:
- Provision an Azure AI Vision resource.
- Analyze an image.
- Generate a smart cropped thumbnail.
Programme scope, delivery arrangements and suitability are confirmed during enquiry.