My Cloud

​

Introduction

My Cloud is a Bring Your Own Cloud (BYOC) implementation that allows you to bring your own cloud to Tune Studio. This feature provides you with the benefits of cost savings, hardware availability, region freedom, and access to all studio features like monitoring, logging, API access, and resource recovery in case of failure, just like Kubernetes.

​

Why My Cloud?

My Cloud offers a seamless one-to-one mapping of all the features and capabilities you enjoy on Tune Studio, but on your own cloud infrastructure. This section highlights the benefits of using My Cloud, and we’ll dive deeper into the specifics of deploying models and fine-tune jobs on your cloud.

• Cost Savings and Optimization: Leverage your existing cloud credits, negotiate better rates with your cloud provider, and avoid additional costs for using BYOC, resulting in significant cost savings.
• Increased Flexibility and Hardware Availability:Utilize your existing hardware or access hardware from any region supported by your cloud provider, giving you more control over your infrastructure and the ability to deploy resources in any region.
• Unified Management and Studio Features: Get access to all Tune Studio features, including monitoring, logging, API access, and resource recovery in case of failure, just like Kubernetes, for streamlined management of your resources from a single interface.
• Enhanced Security: Benefit from mTLS-secured connections and firewall rules that restrict traffic to specific ports, ensuring the security of your resources.
• Simplified Scalability: Easily scale your resources up or down as needed, without worrying about the underlying infrastructure, and take advantage of the scalability offered by your cloud provider.
• Single Pane of Glass: Manage all your resources, including models and fine-tune jobs, from a single interface, making it easier to track and optimize your resource utilization.

​

Getting Started

Before we begin, make sure to check out the Guide on Integrating Your Cloud Provider to get started with integrating your cloud provider with Tune Studio.

​

Models

With My Cloud, you can deploy models on your cloud infrastructure, which are fully managed by Studio. This means you can start, stop, and delete models as needed, and the underlying infrastructure is taken care of by Studio. The process of creating a model on your cloud is identical to deploying a normal model on Tune Studio, with the only difference being that you select an integration, region, and machine to deploy on.

​

Benefits of Deploying Models on Your Cloud

• Full control: You have complete control over the infrastructure and resources used by your models.
• Cost-effective: You can utilize your existing cloud credits or negotiate better rates with your cloud provider.
• Region freedom: You can deploy models in any region supported by your cloud provider.

​

Fine-tune Jobs

My Cloud also allows you to deploy adapter fine-tune jobs on your cloud, which are managed by Studio. The lifecycle of a job remains the same, and the machine is handled by Studio. However, there’s a slight difference in how these jobs work compared to those deployed on Tune Studio.

When you deploy a fine-tune job on your cloud, the base models are still running on Studio. To enable auto-deployment of a model when the job finishes, we need access to the adapter weights. To achieve this, the job running on your cloud uploads the weights to Studio, from where we can make the adapter fine-tuned model available for usage right away.

​

Benefits of Deploying Fine-tune Jobs on Your Cloud

• Faster deployment: With My Cloud, you can deploy fine-tune jobs on your cloud infrastructure, which can lead to faster deployment times.
• Increased control: You have more control over the infrastructure and resources used by your fine-tune jobs.
• Improved collaboration: By uploading the adapter weights to Studio, you can easily share and collaborate on fine-tuned models with your team.

By using My Cloud, you can enjoy the benefits of Tune Studio while maintaining control over your cloud infrastructure. Get started today by integrating your cloud provider and deploying models and fine-tune jobs on your cloud!

​

How it Works

To integrate your cloud with Tune Studio, follow the respective guide for your cloud provider. Once integrated, deploying a resource through Studio on your cloud works similarly to other cloud providers.

Here’s an overview of the process:

1. Create a VPC and Firewall Rule: When you deploy your first resource (model/finetune job) through Studio on your cloud, we create a VPC and a firewall rule to allow Studio to talk to the end resource for orchestration or API calls in case of models.
2. Provision a VM: Deploying a Studio resource works by provisioning a VM on your cloud, which is managed in accordance with your studio resource.
3. External IPV4 Address: Every VM created has an external IPV4 address used to communicate with the resource, and the connection is mTLS secured. Firewall rules are in place to only allow traffic on certain ports.

​

Machine Lifespan

The lifespan of a machine created by Studio depends on the type of resource it is associated with. Currently, you cannot interact with a machine directly, but understanding its lifespan can help you manage your resources more effectively.

​

Models

• Creation: A machine (VM) is created when a model is first created.
• Stopping: The machine is stopped when you stop the model.
• Starting: The same machine is started when you start the model.
• Deletion: The machine is deleted when you delete the model.

​

Fine-tune Jobs

• Creation: A machine (VM) is created when a fine-tune job is first created.
• Completed: The machine is stopped when the job completes successfully.
• Failed: The machine is stopped when a job fails.
• Retry: The same machine is started and used for retries.
• Deletion: The machine is deleted when you delete the job.

​

Supported Cloud Providers

We currently support the following cloud providers for BYOC:

1. GCP
2. AWS - Coming Soon
3. Azure - Coming Soon

​

Guide on Integrating Your Cloud Provider

​

GCP

To integrate your GCP account with Tune Studio, follow these steps:

1

Make sure compute engine API is enabled

Before proceeding, visit the Compute Engine page on the GCP Console. If prompted, enable the Compute Engine API. If you’re not prompted, the API is already enabled.

2

Create a Service Account

Create a service account on GCP by following the guide on creating a service account, click on “Create and Continue”. You can also assign the basic editor role to this service account during creation.

3

Give Basic Editor Permissions

If you’ve already assigned basic editor permissions to the service account, skip this step. Otherwise, assign the basic editor permissions to this principal. Alternatively, you can create a role with granular permissions by following the Granular Permissions with Roles guide and assign the role to the service account.

4

Create a Key

Create a key for the service account by going to the service account page, selecting the principal, and clicking on the “Create key” button.

5

Connect to GCP

Go to the Integrations Tab on Studio, connect to GCP, and paste the contents of the key JSON file.

​

Granular Permissions with roles

This section is recommended if you want to create a granular role for the service account but not necessary for basic integration.

Copy the contents of the yaml file below and create a yaml file you can use to create a new role.

title: "Tune studio role"
description: "Permissions to use Tune studio My cloud feature"
stage: "ALPHA"
includedPermissions:
  - compute.disks.create
  - compute.disks.get
  - compute.disks.use
  - compute.firewalls.create
  - compute.firewalls.get
  - compute.globalOperations.get
  - compute.images.useReadOnly
  - compute.instances.create
  - compute.instances.delete
  - compute.instances.get
  - compute.instances.setMetadata
  - compute.instances.setServiceAccount
  - compute.instances.setTags
  - compute.instances.start
  - compute.instances.stop
  - compute.networks.create
  - compute.networks.get
  - compute.networks.updatePolicy
  - compute.subnetworks.use
  - compute.subnetworks.useExternalIp
  - compute.zoneOperations.get
  - compute.zones.get
  - iam.serviceAccounts.actAs
  - resourcemanager.projects.get

After creating the yaml file create a new role using the following command and assign it to the service account.

gcloud iam roles create ROLE_ID --project=PROJECT_ID --file=YAML_FILE_PATH

Feel free to follow along the GCP guide to create a custom role if you don’t have gcloud available.

​

Pricing

With BYOC, you only pay for the resources you use on your cloud provider. We don’t charge any additional fees for using BYOC. However, please note that the limits and quotas for BYOC are determined by your Tune Studio subscription plan. For more information on our pricing plans and limits, please visit our pricing page.