4. Deploying to Azure¶

Now that we’ve got a working continuous deployment pipeline taking changes to our code and pushing Docker images up into the GitHub Container Registry, we can deploy our application to the cloud as a cloud-native web service.

To do this, we’re going to use Microsoft’s Azure cloud computing service - in particular, a service called Azure App Service.

Recap - cloud and Azure¶

Before we dig into Azure App Service, let’s have a quick recap on what we discussed earlier about the cloud in general and Azure specifically.

“Cloud” is a phrase that gets thrown around a lot, but what does it actually mean?

Some people like to say that “the cloud is other people’s computers” - and this is kind of true. It’s a massive network of computers linked together by complex software-based networking and redundancy, but it’s still other people’s computers.

These computers are physical things you can go visit. If you can get past Microsoft’s security team, that is. (Which you probably can’t.) I’d recommend visiting virtually, instead - you can do a full virtual tour of an Azure data centre - it’s pretty neat!

Microsoft’s Des Moines data centre, a.k.a. “US Central”️
© Google Maps

Economies of scale¶

The main advantage of the cloud is economy of scale. If I wanted to start a business that does machine learning, I’d need to buy a bunch of expensive GPUs (i.e.a big capital investment) before I could start to do any good research which I could then sell on. With the cloud, I can just tap into some of Microsoft’s existing GPUs and pay an hourly rate for just the time I need.

Because Microsoft have thousands of servers and GPUs, they’re able to manage them much more cost and time efficiently, and thus pass on those scaling efficiencies to end users.

Managed services and SEP theory¶

One of the main advantages of using cloud service providers like Microsoft’s Azure is that you can utilise what are referred to as managed services. This means that Microsoft take care of all of the effort of maintaining a particular application like a database or web server or whatever, and give you a nice self-service interface to be able to use this service.

For instance, if you want to host your own PostgreSQL database, you could spin up a fresh VM and install PostgreSQL on it - sounds simple, right? Managing a production database at scale with high available is immensely difficult. With Azure’s “Azure Databases for PostgreSQL” managed service, Microsoft handle all the management, maintenance, replication, failover, etc. and give you a button that says “New database”, saving you many, many days of installing, debugging, monitoring and so on. You still need to do some monitoring and management, obviously, but nothing like you need to do if you’re rolling your own database.

This is referred to¹ as SEP theory - by using managed services, you’re making more things Somebody Else’s Problem.

We will be using the managed service called “Azure App Service” which we’re using essentially as “running a Docker container as a service”. (App Service also supports uploading your code only and selecting a runtime.)

Azure terminology¶

Tenant: This identifies the Azure “directory” that you are in. A single account can log into multiple directories and each directory is entirely separately from each other. Typically, your company will have one (or more) directories dedicated to it. The tenant is the highest level container within Azure - everything in Azure from accounts to subscriptions to resource groups to individual resources are all contained within a tenant.

Subscription: A tenant has one or more subscriptions within in. Each subscription is essentially a “costing centre” - it’s how Azure ties your payment details (e.g. your credit card) into Azure. Costing is done at the subscription level and all resource groups and individual resources live within a subscription. It’s essentially the layer below the tenant.

Resource group: Individual resources like databases and apps cannot live freely but live within a particular resource group. A resource group is a way of grouping together different resources that relate to a particular product or feature. For instance, you might want to have a “web portal” resource group that contains the API, frontend and database for your web portal, while another resource group called “data pipeline” contains all the resources for your data processing pipeline.

Resource: A resource is the individual instance of a service that you are using in Azure. An app in Azure App Service is a resource. An instance of Azure SQL is a resource, and so on.

Getting set up with the Azure CLI¶

We’re going to be using the Azure CLI to create our app. You can use the portal too, but we’re using the CLI.

First things first, make sure the Azure CLI is installed - if you’re using the GitHub codespace, this should come pre-installed. You can check this with:

az --version

If it’s not installed, follow the instructions here to install it: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt#installation-options

You should’ve received or have access to a tenant, subscription, username and password to login to Azure. Let’s use that to login to the Azure CLI:

tenant_id="our tenant ID"
subscription_id="our subscription ID"
sp_username="your username"
sp_password="your password"
resource_group="hncdi-explain-supercharge"

az login --service-principal --tenant "$tenant_id" -u "$sp_username" -p "$sp_password"

To verify that you’re successfully logged in, run:

az account show

To make sure you’re in the right subscription, you can run:

az account set --subscription "$subscription_id"

Creating the Azure App Service app¶

Now that we’re logged into Azure, let’s create our app.

We are putting our app into a resource group that we’ve created especially called hncdi-explain-supercharge. It has to be in this resource group because the service principal we’re using has its permissions restricted to that resource group.

Before we can create our app, we need to create a service plan for this app to be associated with. A service plan is how Azure decides what resources to allocate to your app. You can scale up your service plan after you’ve created it, so you can dynamically scale your resource based on your requirements.

This is great because it means that you can start off on a cheap (or even free) plan, then only scale up to more expensive plans when you need to!

Let’s create our app service plan in the “B2” tier using Linux, which means that we get 2 vCPUs, 3.5 GB of RAM and 10 GB of storage. That should be plenty for our purposes.

my_app_name="put something unique to you in here, like your name or random string"

az appservice plan create --name "$my_app_name-plan" --resource-group "$resource_group" --sku B2 --is-linux

Now we’re ready to create our actual app:

my_github_username="put your GitHub username here"

az webapp create --name "$my_app_name-app" \
    --resource-group "$resource_group" \
    --plan "$my_app_name-plan" \
    --deployment-container-image-name "ghcr.io/$my_github_username/docker-and-gh-actions-for-ml:latest"

# Tell Azure App Service that our API is listening on port 8000.
az webapp config appsettings set \
    --resource-group "$resource_group" \
    --name "$my_app_name-app" \
    --settings WEBSITES_PORT=8000

az webapp deployment container config --enable-cd=true -g hncdi-explain-supercharge -n $my_app_name-app

This might take a couple of while to complete. This is because our Docker image is about 1.5 GB, mostly of which is pytorch runtime dependency files. Be patient with it and if you think it’s actually stuck, just let us know and we can debug it together over Zoom. With our permissions we can see all the logging from the app which we can use to figure out what’s going on.

Once it finishes, you should be able to access your API using the URL https://$my_app_name-app.azurewebsites.net where you replace $my_app_name with the name you decided to use for your application.

Try visiting https://$my_app_name-app.azurewebsites.net/Here’s%20a%20story%20about and see what it shows. If you can see some successfully generated text, that means you’ve successfully deployed your app to Azure. Good job!

Calling to Azure App Service webhook in deploy Action¶

There’s only one thing left to do, and that is update our GitHub Action to tell our Azure App Service app that we have deployed a new image and that they should go get it.

This is pretty simple - App Service provides a URL that you can send a POST request to that’ll do exactly this. All we have to do is retrieve this special URL and put it into the GitHub Action.

az webapp deployment container show-cd-url -g hncdi-explain-supercharge -n $my_app_name-app

This should show a little JSON object with a key called CI_CD_URL - this is the URL we need. Go ahead and copy that value. It should look something like https://$my-lovely-api-app:{long password}@my-lovely-api-app.scm.azurewebsites.net/docker/hook.

Let’s update our GitHub Action workflow to add this in:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

name: Deploy

on: [push]

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs: 
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Log in to the Container registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=raw,value=latest

      - name: Build and push Docker image
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

    - name: Call Azure App Service webhook
      run: curl -v -X POST '${{ secrets.AZURE_APP_WEBHOOK_URL }}'

To add this secret to our GitHub repository for the Actions to see, we need to:

Step 1: Go to our repository page on GitHub and click on “Settings”.

Step 2: Click on “Secrets and variables” and then “Actions”.

Step 3: Click on “New repository secret”.

Step 4: Put AZURE_APP_WEBHOOK_URL under “Name” and the full URL of your webhook from the Azure CLI command under “Secret”, then click on “Add secret”.

You should now see your secret appear under the “Repository secrets” section of this page.

Trying out the continuous deployment¶

Now that we’ve got it all up and working, let’s test it out!

To try it out, let’s make a minor change to our root endpoint. This will add the current application version to the health endpoint result:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

import logging
from functools import cache
from importlib.metadata import version

from fastapi import FastAPI, Depends

from .text_generation import TextGenerator

app = FastAPI()


@cache
def get_model() -> TextGenerator:
    logging.error("Loading DistilGPT2 model")
    return TextGenerator()


@app.on_event("startup")
async def on_startup():
    get_model()


@app.get("/")
async def health():
    return {"health": "ok", "version": version(__name__.split(".", maxsplit=1)[0])}


@app.get("/{prompt}")
async def generate_text(
    prompt: str,
    max_new_tokens: int = 50,
    num_return_sequences: int = 1,
    model: TextGenerator = Depends(get_model),
) -> dict:
    sequences = model.generate(
        prompt, max_new_tokens=max_new_tokens, num_return_sequences=num_return_sequences
    )
    return {"generated_sequences": sequences}

The __name__.split(".", maxsplit=1)[0] just gets us the name of the Python package we’re in, and version() gives us the corresponding package version.

Commit this and push it up, then take a look at the “Actions” tab on your GitHub repository to see it running.

Once that’s completed and you’ve waited a couple of minutes for the update to propagate into Azure, you should be able to hit your Azure App Service app endpoint and see the updated result.