To use Studio through LlamaIndex, here are the follow:

  1. Install dependencies
pip install llama-index
pip install llama-index-llms-openrouter
  1. Export studio key as OpenRouter key
export OPENROUTER_API_KEY="<Tune Studio API Key>"
  1. Init llm using OpenRouter
from llama_index.llms.openrouter import OpenRouter

llm = OpenRouter(
    api_base="<https://proxy.tune.app>",
    max_tokens=256,
    model="rohan/mixtral-8x7b-inst-v0-1-32k",
)

response = llm.complete("Paul Graham is")
print(response)

Follow along with the LlamaIndex tutorial here

Was this page helpful?

Web assistant with function calling using Next JSYour own chat app with threads API and Next JS