LEARNEZY Guides
DATE: 2025.11.15 03:00 CLEARANCE: PUBLIC

Guided Project 2: Optimizing LLM Prompts with TOON

Project TOON LLMs Optimization Prompt Engineering Cost Saving

// table of contents

▼

Guided Project 2: Optimizing LLM Prompts with TOON

In this project, you will experience firsthand the token efficiency of TOON by refactoring a prompt that uses a verbose JSON input into a more compact TOON equivalent. You will measure the token savings and understand how this translates to cost reduction and potentially improved LLM performance.

Project Objective: Optimize an LLM prompt for a sales AI agent by converting its data input from JSON to TOON, focusing on token count reduction.

Technologies Used:

  • Python or Node.js
  • JSON
  • TOON (python-toon or @toon-format/toon)
  • tiktoken (for OpenAI model token counting, or equivalent)
  • An LLM API (e.g., OpenAI, Anthropic, Gemini - you’ll need an API key for your chosen LLM)

Project Setup

Ensure your development environment is ready, including:

  • Python with pip install openai tiktoken python-toon
  • OR Node.js with npm install openai tiktoken @toon-format/toon
  • Your LLM API key set as an environment variable.

Create a new directory for this project:

mkdir toon-optimization-project
cd toon-optimization-project
# For Python: touch optimize_prompt.py
# For Node.js: touch optimize_prompt.js

Step 1: Define the Problem and Sample Data (JSON)

Imagine you’re building a sales AI assistant that needs to analyze a customer’s recent orders and generate a personalized upsell recommendation. The customer’s order history will be provided as input.

Let’s define a sample JSON dataset for a customer’s orders. This data is “tabular-eligible,” meaning it’s an array of objects with uniform fields, which is TOON’s sweet spot.

{
  "customerId": "CUST9876",
  "customerName": "Maria Rodriguez",
  "recentOrders": [
    {
      "orderId": "ORD1001",
      "productName": "Bluetooth Speaker X",
      "quantity": 1,
      "priceUSD": 79.99,
      "orderDate": "2025-10-01",
      "status": "delivered"
    },
    {
      "orderId": "ORD1002",
      "productName": "Smart LED Bulbs (4-pack)",
      "quantity": 2,
      "priceUSD": 49.99,
      "orderDate": "2025-10-05",
      "status": "delivered"
    },
    {
      "orderId": "ORD1003",
      "productName": "Portable Charger 10000mAh",
      "quantity": 1,
      "priceUSD": 29.99,
      "orderDate": "2025-10-10",
      "status": "shipped"
    },
    {
      "orderId": "ORD1004",
      "productName": "Noise Cancelling Headphones",
      "quantity": 1,
      "priceUSD": 199.99,
      "orderDate": "2025-11-01",
      "status": "processing"
    },
    {
      "orderId": "ORD1005",
      "productName": "Smart Home Hub 2.0",
      "quantity": 1,
      "priceUSD": 149.99,
      "orderDate": "2025-11-05",
      "status": "delivered"
    }
  ]
}

Step 2: Create the Baseline (JSON) Prompt and Measure Tokens

First, we’ll create the prompt using the standard JSON data and measure its token count.

Choose your language:

Python (optimize_prompt.py)

import os
import json
import tiktoken
from openai import OpenAI
from toon import encode as toon_encode, decode as toon_decode, compare_formats

# Function to count tokens (from Chapter 6)
def count_tokens(text: str, model_name: str = "gpt-4o-mini") -> int:
    try:
        encoding = tiktoken.get_encoding("cl100k_base")
        return len(encoding.encode(text))
    except Exception as e:
        print(f"Error counting tokens: {e}")
        return -1

# Sample JSON data (from Step 1)
customer_data = {
  "customerId": "CUST9876",
  "customerName": "Maria Rodriguez",
  "recentOrders": [
    {
      "orderId": "ORD1001",
      "productName": "Bluetooth Speaker X",
      "quantity": 1,
      "priceUSD": 79.99,
      "orderDate": "2025-10-01",
      "status": "delivered"
    },
    {
      "orderId": "ORD1002",
      "productName": "Smart LED Bulbs (4-pack)",
      "quantity": 2,
      "priceUSD": 49.99,
      "orderDate": "2025-10-05",
      "status": "delivered"
    },
    {
      "orderId": "ORD1003",
      "productName": "Portable Charger 10000mAh",
      "quantity": 1,
      "priceUSD": 29.99,
      "orderDate": "2025-10-10",
      "status": "shipped"
    },
    {
      "orderId": "ORD1004",
      "productName": "Noise Cancelling Headphones",
      "quantity": 1,
      "priceUSD": 199.99,
      "orderDate": "2025-11-01",
      "status": "processing"
    },
    {
      "orderId": "ORD1005",
      "productName": "Smart Home Hub 2.0",
      "quantity": 1,
      "priceUSD": 149.99,
      "orderDate": "2025-11-05",
      "status": "delivered"
    }
  ]
}

# --- BASELINE: JSON Prompt ---
json_data_str = json.dumps(customer_data, indent=2) # Pretty print for prompt readability

json_prompt = f"""
You are a sales assistant for our e-commerce platform.
The customer's recent order history is provided below in JSON format.

Your task is to:
1. Summarize Maria Rodriguez's recent purchases.
2. Based on her purchases, recommend ONE relevant upsell product.
   Consider items that complement her existing products (e.g., smart home accessories for a smart home hub,
   cases for speakers, etc.).
3. Your recommendation should include the product name and a short reason (1-2 sentences).

Customer Data (JSON):
`json
{json_data_str}
`

Please provide your summary and recommendation in the following JSON format:
`json
{{
  "summary": "...",
  ""recommendation": {{
    "productName": "...",
    "reason": "..."
  }}
}}
`
"""

json_prompt_tokens = count_tokens(json_prompt)
print("=== JSON Prompt (Baseline) ===")
print(json_prompt)
print(f"Token count for JSON prompt: {json_prompt_tokens}")

# (Optional) Call LLM with JSON prompt
# client = OpenAI()
# try:
#     response = client.chat.completions.create(
#         model="gpt-4o-mini",
#         messages=[
#             {"role": "user", "content": json_prompt}
#         ],
#         response_format={"type": "json_object"},
#         temperature=0.7
#     )
#     print("\n--- LLM Response (JSON Prompt) ---")
#     print(response.choices[0].message.content)
# except Exception as e:
#     print(f"Error calling LLM with JSON prompt: {e}")

Node.js (optimize_prompt.js)

import fs from 'fs';
import OpenAI from 'openai';
import tiktoken from 'tiktoken';
import { encode as toonEncode, decode as toonDecode } from '@toon-format/toon';

// Function to count tokens
function countTokens(text, modelName = "gpt-4o-mini") {
    try {
        const encoding = tiktoken.get_encoding("cl100k_base");
        return encoding.encode(text).length;
    } catch (e) {
        console.error(`Error counting tokens: ${e}`);
        return -1;
    }
}

// Sample JSON data (from Step 1)
const customerData = {
  customerId: "CUST9876",
  customerName: "Maria Rodriguez",
  recentOrders: [
    {
      orderId: "ORD1001",
      productName: "Bluetooth Speaker X",
      quantity: 1,
      priceUSD: 79.99,
      orderDate: "2025-10-01",
      status: "delivered",
    },
    {
      orderId: "ORD1002",
      productName: "Smart LED Bulbs (4-pack)",
      quantity: 2,
      priceUSD: 49.99,
      orderDate: "2025-10-05",
      status: "delivered",
    },
    {
      orderId: "ORD1003",
      productName: "Portable Charger 10000mAh",
      quantity: 1,
      priceUSD: 29.99,
      orderDate: "2025-10-10",
      status: "shipped",
    },
    {
      orderId: "ORD1004",
      productName: "Noise Cancelling Headphones",
      quantity: 1,
      priceUSD: 199.99,
      orderDate: "2025-11-01",
      status: "processing",
    },
    {
      orderId: "ORD1005",
      productName: "Smart Home Hub 2.0",
      quantity: 1,
      priceUSD: 149.99,
      orderDate: "2025-11-05",
      status: "delivered",
    },
  ],
};

// --- BASELINE: JSON Prompt ---
const jsonDataStr = JSON.stringify(customerData, null, 2); // Pretty print for prompt readability

const jsonPrompt = `
You are a sales assistant for our e-commerce platform.
The customer's recent order history is provided below in JSON format.

Your task is to:
1. Summarize Maria Rodriguez's recent purchases.
2. Based on her purchases, recommend ONE relevant upsell product.
   Consider items that complement her existing products (e.g., smart home accessories for a smart home hub,
   cases for speakers, etc.).
3. Your recommendation should include the product name and a short reason (1-2 sentences).

Customer Data (JSON):
\`\`\`json
${jsonDataStr}
\`\`\`

Please provide your summary and recommendation in the following JSON format:
\`\`\`json
{
  "summary": "...",
  "recommendation": {
    "productName": "...",
    "reason": "..."
  }
}
\`\`\`
`;

const jsonPromptTokens = countTokens(jsonPrompt);
console.log("=== JSON Prompt (Baseline) ===");
console.log(jsonPrompt);
console.log(`Token count for JSON prompt: ${jsonPromptTokens}`);

// (Optional) Call LLM with JSON prompt
// const openai = new OpenAI();
// async function callLlmWithJson() {
//   try {
//     const response = await openai.chat.completions.create({
//       model: "gpt-4o-mini",
//       messages: [{ role: "user", content: jsonPrompt }],
//       response_format: { type: "json_object" },
//       temperature: 0.7,
//     });
//     console.log("\n--- LLM Response (JSON Prompt) ---");
//     console.log(response.choices[0].message.content);
//   } catch (e) {
//     console.error(`Error calling LLM with JSON prompt: ${e}`);
//   }
// }
// callLlmWithJson();

Run this part of the script and note the token count.

Step 3: Convert to TOON and Measure Tokens

Now, let’s convert the recentOrders array within customer_data to its TOON tabular representation. The customerId and customerName can remain as simple TOON key-value pairs.

Python (optimize_prompt.py - continue appending)

# --- OPTIMIZED: TOON Prompt ---

# Extract tabular part for TOON conversion
orders_toon_data = customer_data["recentOrders"]
customer_info_toon = {
    "customerId": customer_data["customerId"],
    "customerName": customer_data["customerName"]
}

# Encode customer info and orders to TOON
customer_info_toon_str = toon_encode(customer_info_toon, indent=2)
orders_toon_str = toon_encode({"recentOrders": orders_toon_data}, indent=2)

# Combine into a single TOON block for the prompt
# We explicitly show the structure of the customer info and then the recent orders
full_toon_data_str = f"""
{customer_info_toon_str.strip()}
{orders_toon_str.strip()}
"""

toon_prompt = f"""
You are a sales assistant for our e-commerce platform.
The customer's recent order history is provided below in TOON format.
Pay attention to the structure, especially the 'recentOrders[N]{{...}}' block which defines the table.

Your task is to:
1. Summarize Maria Rodriguez's recent purchases.
2. Based on her purchases, recommend ONE relevant upsell product.
   Consider items that complement her existing products (e.g., smart home accessories for a smart home hub,
   cases for speakers, etc.).
3. Your recommendation should include the product name and a short reason (1-2 sentences).

Customer Data (TOON):
`
{full_toon_data_str}
`

Please provide your summary and recommendation in the following TOON format:
`
summary: "..."
recommendation:
  productName: "..."
  reason: "..."
`
"""

toon_prompt_tokens = count_tokens(toon_prompt)
print("\n=== TOON Prompt (Optimized) ===")
print(toon_prompt)
print(f"Token count for TOON prompt: {toon_prompt_tokens}")

print("\n=== Token Comparison Summary ===")
print(f"JSON Prompt Tokens: {json_prompt_tokens}")
print(f"TOON Prompt Tokens: {toon_prompt_tokens}")
if json_prompt_tokens > 0:
    savings_percent = ((json_prompt_tokens - toon_prompt_tokens) / json_prompt_tokens) * 100
    print(f"TOON achieved {savings_percent:.2f}% token savings!")

# (Optional) Call LLM with TOON prompt
# client = OpenAI()
# try:
#     response = client.chat.completions.create(
#         model="gpt-4o-mini",
#         messages=[
#             {"role": "user", "content": toon_prompt}
#         ],
#         # We expect TOON output, not JSON, so don't specify response_format={"type": "json_object"}
#         # You might need to add a system message to explicitly guide LLM for TOON output:
#         # {"role": "system", "content": "You are a helpful assistant that provides structured TOON output."},
#         temperature=0.7
#     )
#     print("\n--- LLM Response (TOON Prompt) ---")
#     print(response.choices[0].message.content)
#     # You can then decode this TOON response back to a Python object:
#     # decoded_response = toon_decode(response.choices[0].message.content)
#     # print("\n--- Decoded TOON Response ---")
#     # print(decoded_response)
# except Exception as e:
#     print(f"Error calling LLM with TOON prompt: {e}")

Node.js (optimize_prompt.js - continue appending)

// --- OPTIMIZED: TOON Prompt ---

// Extract tabular part for TOON conversion
const ordersToonData = customerData.recentOrders;
const customerInfoToon = {
  customerId: customerData.customerId,
  customerName: customerData.customerName,
};

// Encode customer info and orders to TOON
const customerInfoToonStr = toonEncode(customerInfoToon, { indent: 2 });
const ordersToonStr = toonEncode({ recentOrders: ordersToonData }, { indent: 2 });

// Combine into a single TOON block for the prompt
const fullToonDataStr = `
${customerInfoToonStr.trim()}
${ordersToonStr.trim()}
`;

const toonPrompt = `
You are a sales assistant for our e-commerce platform.
The customer's recent order history is provided below in TOON format.
Pay attention to the structure, especially the 'recentOrders[N]{...}' block which defines the table.

Your task is to:
1. Summarize Maria Rodriguez's recent purchases.
2. Based on her purchases, recommend ONE relevant upsell product.
   Consider items that complement her existing products (e.g., smart home accessories for a smart home hub,
   cases for speakers, etc.).
3. Your recommendation should include the product name and a short reason (1-2 sentences).

Customer Data (TOON):
\`\`\`
${fullToonDataStr}
\`\`\`

Please provide your summary and recommendation in the following TOON format:
\`\`\`
summary: "..."
recommendation:
  productName: "..."
  reason: "..."
\`\`\`
`;

const toonPromptTokens = countTokens(toonPrompt);
console.log("\n=== TOON Prompt (Optimized) ===");
console.log(toonPrompt);
console.log(`Token count for TOON prompt: ${toonPromptTokens}`);

console.log("\n=== Token Comparison Summary ===");
console.log(`JSON Prompt Tokens: ${jsonPromptTokens}`);
console.log(`TOON Prompt Tokens: ${toonPromptTokens}`);
if (jsonPromptTokens > 0) {
  const savingsPercent =
    ((jsonPromptTokens - toonPromptTokens) / jsonPromptTokens) * 100;
  console.log(`TOON achieved ${savingsPercent.toFixed(2)}% token savings!`);
}

// (Optional) Call LLM with TOON prompt
// const openai = new OpenAI();
// async function callLlmWithToon() {
//   try {
//     const response = await openai.chat.completions.create({
//       model: "gpt-4o-mini",
//       messages: [
//         { role: "user", content: toonPrompt }
//       ],
//       // For TOON output, you typically do NOT use response_format: {"type": "json_object"}
//       // You might need a system message like:
//       // {"role": "system", "content": "You are a helpful assistant that provides structured TOON output."},
//       temperature: 0.7,
//     });
//     console.log("\n--- LLM Response (TOON Prompt) ---");
//     console.log(response.choices[0].message.content);
//     // You can then decode this TOON response back to a JavaScript object:
//     // const decodedResponse = toonDecode(response.choices[0].message.content);
//     // console.log("\n--- Decoded TOON Response ---");
//     // console.log(decodedResponse);
//   } catch (e) {
//     console.error(`Error calling LLM with TOON prompt: ${e}`);
//   }
// }
// callLlmWithToon();

Run this second part of the script and compare the token counts. You should see a noticeable reduction in token usage with the TOON prompt.

Step 4: Analyze and Discuss the Results

After running the script, reflect on the following:

  1. Token Savings: What was the percentage reduction in token count from the JSON prompt to the TOON prompt?
  2. Readability: How does the TOON-formatted data look compared to the JSON? Is it more or less readable for a human at a glance in this specific context?
  3. LLM Impact (Hypothetical/Optional):
    • If you uncommented the LLM calls, compare the quality of responses. Did the LLM seem to handle the TOON input well?
    • Consider how repeated calls with this optimized prompt would affect your LLM API costs over time.

Step 5: Mini-Challenges and Further Optimization

Challenge 1: Experiment with Delimiters

Modify the TOON encoding for recentOrders to use a different delimiter (e.g., delimiter='\t' for Python or { delimiter: '\t' } for Node.js). Rerun the token count and see if it makes a difference for this dataset and tokenizer. Explain why or why not.

Challenge 2: Implement Output Decoding and Validation

  1. Extend the optional LLM call for the TOON prompt.
  2. After receiving the TOON output from the LLM, use toon_decode() (Python) or toonDecode() (Node.js) to convert it back into a native Python dictionary or JavaScript object.
  3. Define a simple JSON Schema for the expected output structure (summary and recommendation).
  4. Validate the decoded LLM output against this schema. Print the decoded and validated output, or any validation errors.

Challenge 3: Edge Case Data

Modify the customer_data to include an orderDate with an unusual string (e.g., "Not available" or "2025-11-XX"). How does TOON handle this? Does it get quoted? How does the LLM react if it expects a date format?

This project provides a practical demonstration of how TOON can be a valuable tool in your AI engineering toolkit, helping you to create more efficient and cost-effective LLM-powered applications.