Models & Multimodal
17 cloud providers, Sign in with ChatGPT, local GGUF models, OpenAI-compatible endpoints, and multimodal file inputs.
Configure providers during sandwalk setup, in Dashboard → Models (/dashboard/models), or with sandwalk config. Keys stay on your machine in ~/.sandwalk/config.json.
Cloud Model Providers
Run GPT-5, o3/o4-mini, and Codex models via OPENAI_API_KEY or sign in directly with your ChatGPT Plus / Pro subscription.
Claude Opus and Sonnet models with extended thinking effort controls (Low, Medium, High) and prompt-prefix caching.
Gemini Pro and Flash with 1M+ token context windows, native image/PDF/audio/video multimodal support, and Gemini embeddings.
Grok reasoning and multimodal models for fast code synthesis, scientific Q&A, and long-context analysis.
DeepSeek-R1 and DeepSeek-V3 reasoning and coding models with transparent chain-of-thought traces.
Kimi K2 / K3 long-context agentic models optimized for multi-step tool calling and deep literature review.
Unified gateway to 200+ frontier and open-weight models plus OpenRouter embedding endpoints for Omnigraph.
Enterprise AWS Bedrock Converse API using your local boto3 / AWS SSO credentials with zero static API keys.
NVIDIA-hosted Llama, Nemotron, and BioNeMo foundation models accelerated on TensorRT-LLM.
Wafer-scale ultra-low-latency inference (>2,000 tokens/s) for instant Fast-mode agent loops and distillation.
LPU-accelerated open-weight models for sub-second interactive tool calling and rapid sub-agent fan-out.
First-class adapters for Fireworks AI, Together AI, Mistral AI, Qwen Cloud, MiniMax, and OpenCode Zen.
Local & Custom OpenAI-Compatible Runtimes
Local endpoints are auto-discovered from /v1/models or Ollama's /api/tags. Add custom endpoints (such as vLLM, SGLang, TGI, LiteLLM, or an HPC cluster) with Add Local LLM in Dashboard → Models, or in the CLI with /models:
Download, load, and eject quantized GGUF models and local embeddings directly from Sandwalk's top toolbar with GPU-aware background deferral.
Automatically discovers installed Ollama models and context window sizes via /api/tags and /api/show.
High-throughput Apple Silicon MLX inference server utilizing unified memory on Mac Studio, Mac mini, and MacBook Pro.
Connect to LM Studio's local OpenAI-compatible server for desktop GGUF and MLX model experimentation.
Run custom domain-tuned Unsloth and DS4 checkpoints locally or on dedicated GPU workstations.
Add any custom base URL (/v1/chat/completions & /v1/models) on your LAN, Tailscale tailnet, or university/corporate HPC cluster.
Multimodal & File Inputs
Attach files to any message using @ in the Composer, drag-and-drop, or Cmd+V / Ctrl+V clipboard paste. In Dashboard → Models, each model row displays badges for the modalities it accepts natively:
| Modality | Extensions | Native Model Support & Handling |
|---|---|---|
| Images & Figures | .png, .jpg, .webp, .gif, .svg | Sent natively to vision-capable models (OpenAI GPT-4o/5, Claude, Gemini, Grok, Qwen-VL, local VLMs). Clipboard paste (Cmd+V) saves to uploads/ and inserts an [Image N] chip. |
| Documents & PDFs | .pdf, .md, .csv, .tsv, .docx, .pptx | Native PDF vision on Gemini, Claude, and OpenAI; office documents can also be converted to Markdown via /convert-documents-to-markdown or analyzed by Python sandbox tools. |
| Audio & Video | .mp3, .wav, .m4a, .mp4, .mov, .webm | Supported natively by Google Gemini models and playable inline in the Artifacts panel. |
| 3D Structures & Sequences | .pdb, .cif, .sdf, .mol2, .fasta | Automatically passed to scientific tools (rdkit, biopython, boltz, fold) and rendered interactively in the 3D Structure Viewer. |
Sign in with ChatGPT
OpenAI models can run on your existing ChatGPT Plus or Pro subscription without an API key. In the app, click Continue with ChatGPT under Dashboard → Models, or run in a terminal:
sandwalk login chatgpt # opens browser; stores tokens in ~/.sandwalk/chatgpt-auth.json
sandwalk login status
sandwalk logout chatgptModels then appear under ChatGPT plan in the model picker.
Amazon Bedrock (IAM)
Bedrock uses the same boto3 credential chain as the AWS CLI and LangChain ChatBedrockConverse. Sandwalk defaults to IAM / your AWS profile (AWS_PROFILE, ~/.aws/credentials, or instance role):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream",
"bedrock:ListFoundationModels",
"bedrock:ListInferenceProfiles",
"bedrock:GetFoundationModel"
],
"Resource": "*"
}
]
}Verify locally with aws sts get-caller-identity, then click Models → Amazon Bedrock → Test in the Dashboard.
Last updated on