Deploying an LLM
Last updated
Was this helpful?
Last updated
Was this helpful?
Certain Gretel Models can utilize an online LLM (Large Language Model) to improve functionality. This guide will walk you through the steps to deploy an LLM in your Gretel Hybrid environment.
Ensure you have completed the general prerequisites for deploying Gretel Hybrid, found in the guide.
You'll need to have already installed Gretel Hybrid.
This guide will utlize helm
to install a chart within your Kubernetes cluster.
helm
chartThe Gretel Inference LLM chart is available in the Gretel Helm repository.
To add repository to your local helm
installation, run the following command:
Create a values.yml
file:
Ensure your kubectl
context is set to the correct cluster where you're already running Gretel Hybrid.
Apply the chart to your Kubernetes cluster:
After giving the pod a few minutes to spin up, ensure that the pod is in a healthy state:
Transform v2 can utilize the Gretel Inference LLM service for classification. For an example of how to configure a hybrid Transform v2 job to use classification, see the guide.