-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
68a2600
commit 6164de9
Showing
7 changed files
with
140 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
<p align="center"> | ||
<img src="https://github.com/intel/terraform-intel-gcp-vm/blob/main/images/logo-classicblue-800px.png?raw=true" alt="Intel Logo" width="250"/> | ||
</p> | ||
|
||
# Intel® Optimized Cloud Modules for Terraform | ||
|
||
© Copyright 2024, Intel Corporation | ||
|
||
## GCP C4 Instance with 5th Generation Intel® Xeon® Scalable Processor (Emerald Rapids) & Open Platform for Enterprise AI (OPEA) ChatQnA Example | ||
|
||
This demo will showcase Retrieval Augmented Generation (RAG) CPU inference using 5th Gen Xeon Scalable Processors on GCP using the OPEA ChatQnA Example. For more information about OPEA, go [here](https://opea.dev/). For more information on this specific example, go [here](https://github.com/opea-project/GenAIExamples/tree/main/ChatQnA). | ||
|
||
## Usage | ||
|
||
## After cloning the repo, modify /examples/gen-ai-xeon-opea-chatqna/variables.tf to add your Huggingface Token | ||
|
||
Some Models required a Token. Modify the Huggingface Token variable to your specific Huggingface Token, for information on creating a Huggingface token go [here](https://huggingface.co/docs/hub/en/security-tokens) | ||
|
||
```hcl | ||
variable "huggingface_token" { | ||
description = "Huggingface Token" | ||
default = " <YOUR HUGGINGFACE TOKEN> " | ||
type = string | ||
} | ||
``` | ||
|
||
**If needed, also modify values on /examples/gen-ai-xeon-opea-chatqna/main.tf to match your needs** | ||
|
||
## GCP Cloud Shell Usage | ||
|
||
1. Log on to GCP Portal | ||
2. Enter the GCP Cloud Shell (terminal button on top right of page) | ||
3. Run the following commands in order: | ||
|
||
|
||
```bash | ||
git clone https://github.com/intel/terraform-intel-gcp-vm.git | ||
cd terraform-intel-gcp-vm/examples/gen-ai-xeon-opea-chatqna | ||
# ADD TOKEN TO variables.tf (SEE ABOVE) | ||
terraform init | ||
terraform plan | ||
terraform apply | ||
|
||
# (enter your GCP project ID and "yes" to confirm) | ||
|
||
``` | ||
|
||
After the Terraform module successfully creates the GCP VM instance, **wait ~15 minutes** for the module to launch the containers and download the LLMs before continuing. | ||
|
||
## Accessing the Demo | ||
|
||
You can access the demos using the following: | ||
|
||
- OPEA ChatQnA: `http://yourpublicip:5174` | ||
|
||
|
||
## Deleting the Demo | ||
|
||
To delete the demo, run `terraform destroy` to delete all resources created. | ||
|
||
## Pre-requisites for running on a local Workstation (disregard if using GCP Cloud Shell) | ||
|
||
1. Google Cloud CLI: https://cloud.google.com/sdk/docs/install | ||
2. CGP account access configured: https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference.html#running-terraform-on-your-workstation | ||
3. Terraform: https://learn.hashicorp.com/tutorials/terraform/install-cli | ||
4. Git: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
#cloud-config | ||
package_update: true | ||
package_upgrade: true | ||
|
||
package: | ||
- git | ||
|
||
runcmd: | ||
- apt install ansible -y | ||
- git clone https://github.com/intel/optimized-cloud-recipes.git /tmp/optimized-cloud-recipes | ||
- cd /tmp/optimized-cloud-recipes/recipes/ai-opea-chatqna-xeon | ||
- cp opea.sh /etc/profile.d/opea.sh | ||
- echo 'export HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN}' | sudo tee -a /etc/profile.d/opea.sh | ||
- chmod +x /etc/profile.d/opea.sh | ||
- source /etc/profile.d/opea.sh | ||
- ansible-playbook recipe.yml | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
#Random ID to minimize the chances of name conflicts | ||
resource "random_id" "rid" { | ||
byte_length = 3 | ||
} | ||
|
||
#GCP Linux VM with Intel OPEA ChatQnA | ||
module "linux_vm" { | ||
source = "../.." #intel/gcp-vm/intel | ||
project = var.project | ||
boot_image_project = "ubuntu-os-cloud" | ||
boot_image_family = "ubuntu-2204-lts" | ||
name = "lmelo-ai-opea-chatqna-${random_id.rid.dec}" | ||
zone = "us-east4-a" #"us-central1-a" | ||
machine_type = "c4-highcpu-48" | ||
allow_stopping_for_update = true | ||
tags = ["lmelo-ai-opea-chatqna-${random_id.rid.dec}"] | ||
user_data = templatefile("./cloud_init.yml", { HUGGINGFACEHUB_API_TOKEN = var.huggingface_token }) | ||
access_config = [{ | ||
nat_ip = null | ||
public_ptr_domain_name = null | ||
network_tier = "PREMIUM" | ||
}, ] | ||
} | ||
|
||
#Required firewall rules | ||
resource "google_compute_firewall" "rules" { | ||
project = var.project | ||
name = "lmelo-ai-opea-chatqna-${random_id.rid.dec}" | ||
network = "default" | ||
description = "Allows access to OPEA AI ChatQnA" | ||
|
||
allow { | ||
protocol = "tcp" | ||
ports = ["22","80", "443", "6379", "8001", "6006", "6007", "6000", "7000", "8808", "8000", "8888", "5173", "5174", "9009", "9000"] | ||
} | ||
source_ranges = ["0.0.0.0/0"] | ||
target_tags = ["lmelo-ai-opea-chatqna-${random_id.rid.dec}"] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
variable "project" { | ||
type = string | ||
default = "551221341017" | ||
description = "Enter GCP Project ID" | ||
} | ||
|
||
# Variable for Huggingface Token | ||
variable "huggingface_token" { | ||
description = "Enter a Huggingface Token to be used to download the models" | ||
default = "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" | ||
type = string | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters