For AI agents: the complete documentation index is available at https://docs.us.ovhcloud.com/en/llms.txt, the full documentation bundle is available at https://docs.us.ovhcloud.com/en/llms-full.txt, and this page is available as Markdown at https://docs.us.ovhcloud.com/en/public-cloud/ai-machine-learning/ai-deploy-capabilities.md.
  • English

AI Deploy - Features, capabilities, and limitations

View as Markdown

Learn about the technical capabilities and limitations of the Public Cloud AI Deploy offer.

Info

AI Deploy is covered by OVHcloud Public Cloud Special Conditions.

 

Features

Available features

AI Deploy allows you to deploy your AI applications and models with compute resources (CPUs, GPUs) and high availability.

FeatureDetails
AI environments
Easy customizationAI Deploy allows the installation of almost any Conda or Pip packages. You can easily customize your environment to suit your needs. It also allows you to use your own environment by specifying your Docker image.
Management
Multiple ways to manage your deployed appsYou can manage your AI Deploy apps through the OVHcloud Control Panel, API or CLI. Depending on your needs, you can easily automate their creation and deletion as well.
Easy start and StopYou can start and stop an app in one click.
Check the logs of your appYou can also check the logs of your deployed app through the OVHcloud Control Panel, API or CLI.
Compute resources
Guaranteed compute resourcesSelect the number of CPUs or GPUs required during the creation of the AI Deploy app. Once launched, you will keep these resources as long as your app is running.
Background executionYour tasks can be executed in the background, meaning that closing your web browser will not affect your work.
No maximum runtimeYour tasks can last as long as your app is running.
Monitoring toolsEach AI Deploy app comes with a native Grafana dashboard, allowing you to track and monitor your CPU, GPU, RAM, and storage resources.
Storage
Fast and flexible storageEach AI Deploy service comes with local storage but also the ability to attach remote storage from Object Storage. From a few GiB to multiple TiB, we push your data near our compute power on fast SSD storage for better performances.
Git repositories importationDuring the creation of your AI Deploy app, you can specify one or multiple Git repositories to download inside your app environment.
Security
Open or restricted authenticationDuring the creation of your AI Deploy app, you can either select Public Access, allowing anyone to access your application, or Restricted Access, allowing access via security tokens or credentials.
Billing
Easy billingYou only pay for what you consume, billed per minute with per-second granularity.

 

Logs and Monitoring tools

Command line interface (CLI)

AI Deploy is compliant with the OVHcloud AI CLI. Discover how to install the OVHcloud AI CLI.

Logs

To check the logs of your app, you can do it via the ovhai CLI using the following command:

ovhai app logs <app-id>
Monitoring tools

To see information about your app, you can do so with the ovhai CLI using this command:

ovhai app get <app-id>

You can then access your metrics through the Monitoring Url.

You are also able to check it from the OVHcloud Control Panel in your app information by clicking the Go to Graph Dashboard button.

 

API management

API endpoints to manage your AI Deploy apps can be found here:

GET/cloud/project/{serviceName}/ai/app

 

Planned features

We continuously improve our offers. You can follow, vote, and submit ideas to add to our roadmap.

 

Capabilities and limitations

Supported regions

AI Notebooks can be used from any country in the world, as long as you have an OVHcloud account. Our AI & Machine Learning services are based in the US-EAST-VA (Vint Hill, Virginia) region.

 

Attached resources

Compute resources

Replica type

You can either choose the number of GPUs or CPUs for an AI Deploy app, not both. By default, an app uses one CPU instance. The memory resource is not customizable.

Available replica ranges

If you choose GPU:

  • CPU, memory, and local storage resources are not customizable but scaled linearly with each additional GPU.

If you choose CPU:

  • Memory and local storage resources are not customizable but scaled linearly with each additional CPU.

The maximum number of CPUs/GPUs, memory per CPU/GPU, and local storage is available on the OVHcloud website, Control Panel and the ovhai CLI.

ovhai capabilities flavor list

For your information, the current limits are:

  • CPU: 12 per app.
  • GPU: 4 per app.
Available hardware for AI Deploy

Currently, we provide:

  • ai1-1-cpu
  • L4-1-gpu
  • L40s-1-gpu
Scaling

AI Deploy offers the possibility to choose between two scaling strategies: static or automatic.

If you choose static scaling:

  • You can go from 1 to 10 replicas.

If you choose autoscaling:

  • You can choose both the minimum number of replicas (1 by default) and the maximum number of replicas.
  • You can define the metric monitored that will act as a trigger for autoscaling: CPU or RAM.
  • You can choose the threshold for the percentage of average use: integer between 1 and 100%.

 

Available storage

Local storage

Each AI Deploy app comes with a local storage space, which is ephemeral. When you delete your app, this storage space is also deleted. This storage space depends on the selected instances during AI Deploy app deployment. Please refer to the compute resources section for more information.

Info

Local storage is limited and not the recommended way to handle data, see the OVHcloud documentation on data for more information.

Attached storage

You can attach data volumes from Public Cloud Object Storage. For best performance, we recommend keeping the Object Storage bucket in the same region as your AI Deploy app. Mounting a bucket from a different region also works, but adds cross-region network latency. Attached storage allows you to work on several TB of data, while being persistent when you delete your AI Deploy app.

 

Maximum execution time

There is no duration limitation on AI Deploy apps execution.

 

Environment customization & Permitted deployment

Each environment can be customized directly with PIP or CONDA (we support almost any package and library).

AI Deploy authorizes the deployment of your own Docker images and applications from the OVHcloud portfolio.

Docker images can be hosted in a public or private registry.

The use of docker-compose is not possible.

Please be aware that images need to be built with an AMD architecture. For more information, see this guide.

 

High-availability

Your application is deployed simultaneously on the amount of selected instances. To benefit from high availability, a minimum of two instances is required. In case of instance failure, a new one is automatically created.

 

Rolling upgrade

You can update the Docker image of your application to provide an updated version of your service. Updates are incremental and will not cause any downtime. Your current configuration will also be preserved, such as your HTTP endpoint and deployment policy. There is no need to stop and restart the application for the image update to take effect.

 

Network

  • Public networking can be used for all the AI Tools.
  • Private networking (vRack) is not supported.
  • Ports: you can map your AI Deploy app to only one port. Default port is 8080.
  • Ingress and Egress traffic are included in the service plans and not metered.

 

Quotas per Public Cloud project

Each Public Cloud project grants a customer by default a maximum of 4 GPUs used simultaneously. Reach out to our support if you need to increase this limitation.

 

Go further

Browse all the AI & Machine Learning guides.

Browse the rest of the OVHcloud US documentation for guides on our other products and services.

Learn more about AI & Machine Learning on the OVHcloud US website.

If you need training or technical assistance to implement our solutions, contact your sales representative or click on this link to get a quote and ask our Professional Services experts for a custom analysis of your project.