VynOps
v0.1.0  ·  Open Source  ·  MIT License

VynAI — AI-Powered Ollama Server Platform

Deploy, manage, and scale Ollama LLM server fleets with intelligent routing, GPU monitoring, usage analytics, and a self-hosted OpenAI-compatible API gateway.

Built on Next.js 15 · Groq Llama-4 · Ollama · NVIDIA GPU metrics · OpenAI-compatible API

Get Started Free View SourcePricing
Fleet management
Model routing
Usage analytics
GPU monitoring
Model lifecycle
API gateway
API key auth
Rate limiting
Request logging
OpenAI-compatible

Features

Run your own AI infrastructure

Ollama Server Fleet Management

Deploy and manage multiple Ollama server instances across your infrastructure from a unified dashboard — with health monitoring, auto-restart, and resource tracking.

  • Add unlimited Ollama servers with a single API call or via the UI
  • Real-time health monitoring: model load status, GPU memory, VRAM usage
  • Automatic restart on crash with configurable retry policy
  • Per-server model inventory: which models are loaded, their size and status
  • Remote model pull: download models across all servers from the UI

Intelligent Model Routing

A smart gateway that routes inference requests to the right server and model based on load, capability, and latency — transparently to clients.

  • OpenAI-compatible API gateway endpoint — drop-in replacement
  • Least-loaded routing: automatically send requests to the least busy server
  • Capability routing: route code requests to Codestral, chat to Llama, etc.
  • Fallback routing: if primary server is busy, route to secondary automatically
  • Request queuing with configurable timeout and priority levels

Usage Analytics & Cost Attribution

Track every inference request — tokens generated, latency, model used, and which application or user triggered it.

  • Per-request logging: model, tokens in/out, latency, server, timestamp
  • Usage dashboards by model, server, API key, and team
  • Token cost estimation: map usage to equivalent cloud LLM API costs
  • Rate limiting per API key: prevent any single app from hogging capacity
  • Export usage data as CSV or via API for finance showback

GPU Resource Monitoring

Deep GPU visibility across all servers — VRAM utilisation, temperature, utilisation %, and per-model memory footprint.

  • NVIDIA GPU metrics via nvidia-smi and NVML
  • Per-GPU VRAM usage, temperature, power draw, and utilisation
  • Model memory footprint: how much VRAM each loaded model consumes
  • GPU saturation alerts: notify when VRAM >90% or temperature >85°C
  • Multi-GPU server support with per-card breakdown

Model Lifecycle Management

Pull, update, unload, and retire models across your entire fleet from one place — with version tracking and rollback.

  • Bulk model operations: pull llama3.2 on all servers in one click
  • Model version pinning: pin a server to a specific model digest
  • Automated model updates: check for new versions on a schedule
  • Staged rollout: update one server first, validate, then roll out to fleet
  • Model A/B testing: split traffic between two model versions and compare quality

API Gateway & Access Control

A secure, authenticated gateway in front of your Ollama servers — with API keys, rate limits, and audit logging.

  • API key management: create, revoke, and scope keys per application or team
  • Per-key rate limiting: tokens per minute, requests per hour
  • IP allowlisting and CORS configuration
  • Full request audit log: every inference call logged with key and application
  • OpenAI SDK compatible: works with LangChain, LlamaIndex, and any OpenAI client

Installation

Quick Start

From zero to a managed Ollama fleet in under 10 minutes.

01

Prerequisites

# Install Ollama on at least one server: curl -fsSL https://ollama.com/install.sh | sh ollama serve # starts on http://localhost:11434 # Then clone VynAI: git clone https://github.com/vynops/VynAI cd vynai cp .env.local.example .env.local
02

Configure and start

# .env.local: VYNAI_SECRET=<openssl rand -base64 32> # Optional: Groq API key for AI-assisted analytics GROQ_API_KEY=<from console.groq.com> npm install npm run dev # Dashboard: http://localhost:3010 # Gateway endpoint: http://localhost:3010/v1 (OpenAI-compatible)
03

Register your Ollama servers

curl -X POST http://localhost:3010/api/servers \ -H 'Content-Type: application/json' \ -d '{ "name": "gpu-server-01", "url": "http://192.168.1.20:11434", "gpuCount": 2, "vramGiB": 48 }'
04

Use the OpenAI-compatible gateway

# Point your existing OpenAI SDK at VynAI: from openai import OpenAI client = OpenAI( base_url="http://localhost:3010/v1", api_key="vynai-your-api-key" ) response = client.chat.completions.create( model="llama3.2", messages=[{"role": "user", "content": "Hello!"}] )

Environment Variables

Full reference for .env.local.

VariableDescriptionRequired
VYNAI_SECRETRandom 32-byte secret for session and API key signingrequired
GROQ_API_KEYOptional: Groq key for AI-assisted usage analysis and summariesoptional
DEFAULT_OLLAMA_URLURL of default Ollama server to register on first startupoptional
GATEWAY_RATE_LIMIT_RPMGlobal requests-per-minute limit across all API keys (default: 1000)optional
GATEWAY_RATE_LIMIT_TPMGlobal tokens-per-minute limit across all API keys (default: unlimited)optional
SLACK_WEBHOOK_URLSlack webhook for GPU saturation and server down alertsoptional
MODEL_AUTO_UPDATESet to "true" to automatically check for model updates dailyoptional
LOG_RETENTION_DAYSNumber of days to retain request logs (default: 30)optional
PORTHTTP port (default: 3010)optional

Supported Models

Any model available in the Ollama model library is supported. Popular choices:

General Purpose

  • llama3.2
  • llama3.1:70b
  • mistral
  • gemma3
  • qwen2.5

Code Generation

  • codestral
  • deepseek-coder-v2
  • qwen2.5-coder
  • starcoder2

Embedding & Vision

  • nomic-embed-text
  • mxbai-embed-large
  • llava
  • moondream
Free · Open Source · Self-hosted

Own your AI infrastructure

No OpenAI dependency. No per-token cloud costs. Run models on your own hardware, your own terms.

Get Started View PricingTalk to us