Prepare for the “Intro to LLM Red Teaming Using Garak” Workshop

Prepare for the “Intro to LLM Red Teaming Using Garak” Workshop

Before attending the Intro to LLM Red Teaming Using Garak workshop, please download the student workbook and one of the preconfigured virtual-machine images listed below. The virtual machines include Garak and a local instance of Microsoft’s Phi-3 language model, allowing you to complete the hands-on activities without relying on an external AI service.

Because the virtual-machine files are large, begin downloading them well before the workshop. You need only the image that matches your virtualization software—either VirtualBox or VMware.

Required Course Materials
1. Student Workbook

Download and review the workbook before class:

Download the Intro to LLM Red Teaming Student Workbook

File: Intro_to_LLM_Red_Teaming_Student_Workbook.docx
Approximate size: 84.92 KB

The workbook contains workshop concepts, guided exercises, knowledge checks, and space for recording observations and assessment findings.

2. Preconfigured Virtual Machine or Windows Install

Choose one of the following images.

VirtualBox - credentials for image (secone/secone123)

Download the VirtualBox Garak Image

File: Garak.ova
Approximate size: 14.08 GB

Import the .ova file using the appliance-import feature in Oracle VirtualBox.

VMware - credentials for image (secone/secone123)

Download the VMware Garak Image

File: Lubuntu Garak.zip
Approximate size: 15.32 GB

Extract the ZIP archive before opening the virtual machine in VMware Workstation, VMware Fusion, or another compatible VMware product.

or

You can install directly on a Windows computers:

GARAK Setup and Download:
Start up PowerShell as Admin
#Install python version 3.13
winget install Python.Python.3.13
#Create Project Folder
mkdir garak-project
#Go to folder
cd garak-project
#Create Environment
py -m venv .venv
#Allow powershell scripts
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
#Activate Environment
.\.venv\Scripts\Activate.ps1
#Upgrades and Installs Pip
py -m pip install --upgrade pip
#Install Garak
py -m pip install -U garak
#Ollama Windows Download
irm https://ollama.com/install.ps1 | iex
#Download Phi3 Model
Ollama pull phi3
#Scan Phi3 with autoDANCached Probe
py -m garak --model_type ollama --model_name phi3 --probes dan.AutoDANCached

Before the Workshop

Please complete the following steps before class:

Install or update your preferred virtualization platform.
Download the student workbook.
Download either the VirtualBox image or the VMware image.
Import or open the virtual machine.
Start the virtual machine and confirm that the Lubuntu desktop loads successfully.
Verify that Garak and the local Phi-3 model are available.
Bring the workstation’s administrator credentials in case a local configuration change is needed.
Ensure that your computer has enough free disk space for the download, extracted image, snapshots, and workshop output files. At least 35–40 GB of available space is recommended.

Depending on your virtualization software, you may be asked whether the virtual machine was moved or copied. Selecting “I copied it” is generally appropriate because it creates new identifiers for your local copy.

What We Will Cover

This workshop introduces the principles and practice of red teaming applications powered by large language models. We will discuss why traditional software testing is not sufficient by itself for generative AI systems and examine how adversarial testing can reveal unsafe, unreliable, biased, or policy-violating behavior.

Topics will include:

The purpose, value, and limitations of LLM red teaming
Common risks associated with generative AI applications
Threat modeling and defining the scope of an assessment
Understanding Garak probes, generators, detectors, and reports
Configuring Garak to test a locally hosted language model
Running focused and grouped probe evaluations
Interpreting results without treating automated findings as definitive
Recognizing prompt-injection, jailbreak, content-safety, and misuse risks
Documenting evidence and distinguishing true findings from false positives
Developing practical recommendations for model and application developers
Applying age-appropriate safety criteria to applications intended for children
Hands-On Activities

During the workshop, you will use Garak to evaluate the local Phi-3 model. Activities will include executing selected probes, reviewing model responses, locating Garak reports, and assessing whether observed behavior represents an acceptable risk.

A major exercise will place you in the role of a red-team assessor reviewing a generative-AI application designed to communicate with middle-school students. You will investigate areas such as jailbreak resistance, slur generation, bullying, sexualized content, and attempts to conceal unsafe instructions. Based on the evidence, you will decide whether the application should be approved, conditionally approved, or rejected and provide recommendations to its developers.

Important Testing Guidance

All workshop testing must remain inside the authorized lab environment. Do not direct probes at public systems, production applications, or third-party services without explicit written authorization.

Some test prompts and model responses may contain offensive, discriminatory, sexualized, or otherwise disturbing material. These examples are included strictly for controlled security testing and educational analysis. Handle generated reports responsibly, avoid unnecessarily reproducing harmful content, and follow the instructor’s guidance throughout the exercises.

Need Help Before Class?

If the download, extraction, or virtual-machine import fails, record the error message and contact the instructor before the workshop. Resolving setup issues in advance will allow us to spend class time on red-teaming techniques and hands-on analysis.

Please arrive with the workbook downloaded and the virtual machine successfully started. We look forward to exploring how structured adversarial testing can help make generative-AI systems safer, more reliable, and better suited to their intended audiences.

＼ Get the latest news ／