How to setup a fullstack Computer Vision AI pipeline on Google GKE with CVAT within 5 minutes

This video walks you through how to install Onepanel Community Edition with a complete CVAT computer vision pipeline on Google's GKE environment.

For docs and to download Onepanel CE go to our Github repo:

https://docs.onepanel.ai/
​

Timestamps:

Intro (0:00)
Install CLI (0:15)
Create Kubernetes cluster (0:19​)
Get Kubernetes config (0:29)
Install Onepanel CLI (0:37)
Generate configuration files (0:45​)
Update params.yaml file (0:56)
Add GPU nodes (1:03)
Get node labels (1:13​)
Setup object storage (1:28​)
Database setup (1:43)
Deploy Onepanel (1:51)
Set up DNS (1:08)
Get auth token (2:14)
Create CVAT workspace (2:28​)
CVAT create task (2:45)
Filesyncer (2:52)
CVAT annotation (3:26​)
Train inference models from annotated data with Workflows (3:40)
Logs and Tensorboard (4:00​)
Create models on CVAT (4:11)
Run automatic annotation with trained model (4:53​)

Transcript:

In this video we'll walk through deploying Onepanel to GKE,
labelling your data using CVAT,
training a semantic segmentation model on the labelled data
and automatically annotating new data using your newly trained model.
Make sure you have the latest version of Google Cloud SDK (gcloud)
Run the command to create your GKE cluster
The command will automatically retrieve your cluster's access credentials but you can also get them by running
and confirm access by running a `kubectl` command
Next, download the latest `opctl` for your operating system
and run the following command to initialize a `params.yaml` template
Refer to "Configuration files" article in our documentation for more information about each section
Open params file and then add a namespace, domain, and fully qualified domain for your deployment
you can optionally add GPU nodes by following the steps in our documentation
Next, update your node pools params, you can get your node pool label and values
by running the following `kubectl` command
Next, set up the default object storage for your artifacts, which is used to store logs,
models and data for your pipelines
Next, set your database credentials by updating username and password in the `database` section.
Finally, run the following command to deploy Onepanel to your cluster
Once deployed, the CLI will display the host name and wildcard domain you need to use to set up your DNS
You can also get this information again by running the `opctl status` command
Next, add the `A` record under your DNS provider and save
Run the following command to get your admin auth token for logging into Onepanel
Open your app URL with your preferred browser and then type in your credentials
Let's now launch a CVAT Workspace so we can annotate our data
Click Create Workspace and select the CVAT template, enter a name and select a node pool
Once your CVAT Workspace is running, click View
In CVAT, click Create new task
Enter a name for your task and then under Constructor, add your labels
You can then sync your data by clicking the Onepanel icon in bottom right corner of your screen
In Workspace Path, enter the path you want to sync your data into
then click Browse in Object storage location field and navigate to your image or video data
Finally, click Sync to Workspace
Once syncing is complete, click Refresh in CVAT and select your files
click Submit to create your annotation task
Click Tasks
Click Job 1 to go into CVAT to start annotating your data
You can now train object detection or semantic segmentation models directly from CVAT
Click on Actions for a task you want to train a model and click on Execute training Workflow
Select a training Workflow Template and update your hyperparameters by following our documentation
Click Execute Workflow to start the training Workflow
Click Open Workflow Details to see your training Workflow progress
You can click Logs to view real-time logs and click TensorBoard to see real-time training metrics
Once training is complete, navigate to the model folder and copy its path
Navigate back to CVAT and click Models and Create new model and enter a model name
Click the Onepanel icon in bottom right corner, and paste the path you copied earlier
into Object storage location field and enter the destination in Workspace path
Click Sync to Workspace; you should see a log of data being synced into CVAT
Navigate to Connected file share in CVAT and click Refresh to see newly synced files
Expand the file tree to navigate to the directory you synced your model into,
select the directory and click Submit
Click Tasks and then click Automatic annotation under Actions menu for the task you want to annotate.
Select the model you created earlier and make sure the class/label mappings are correct then click Submit
Wait for process to complete and then click Open to open the task
Finally, click Job # to view annotations