LLM Providers

AINexLayer supports 50+ Large Language Model (LLM) providers, giving you the flexibility to choose the best model for your specific use case, budget, and requirements.

Overview

AINexLayer is model-agnostic, meaning you can use any supported LLM provider without changing your workflow. This flexibility allows you to:

• Optimize Costs: Choose cost-effective models for different tasks

• Ensure Privacy: Use local models for sensitive data

• Maximize Performance: Select the best model for each use case

• Avoid Vendor Lock-in: Switch providers as needed

Cloud-Based Providers

OpenAI

Best for: General-purpose tasks, code generation, creative writing

Available Models

• GPT-3.5 Turbo: Fast, cost-effective for most tasks

• GPT-4: Advanced reasoning and complex analysis

• GPT-4o: Multimodal with vision capabilities

• GPT-4 Turbo: Enhanced performance with larger context

• GPT-4-32k: Extended context length for long documents

Configuration

{
  "provider": "openai",
  "apiKey": "your-openai-api-key",
  "model": "gpt-4",
  "temperature": 0.7,
  "maxTokens": 2000
}

Pricing (Approximate)

• GPT-3.5 Turbo: $0.002/1K tokens

• GPT-4: $0.03/1K tokens

• GPT-4o: $0.005/1K tokens

Anthropic

Best for: Analysis, reasoning, safety-critical applications

Available Models

• Claude 2: Advanced reasoning and analysis

• Claude 3 Haiku: Fast, lightweight model

• Claude 3 Sonnet: Balanced performance and speed

• Claude 3 Opus: Most capable model for complex tasks

Configuration

{
  "provider": "anthropic",
  "apiKey": "your-anthropic-api-key",
  "model": "claude-3-sonnet-20240229",
  "temperature": 0.7,
  "maxTokens": 2000
}

Pricing (Approximate)

• Claude 3 Haiku: $0.00025/1K tokens

• Claude 3 Sonnet: $0.003/1K tokens

• Claude 3 Opus: $0.015/1K tokens

Google

Best for: Multimodal tasks, research, analysis

Available Models

• Gemini Pro: Google's advanced language model

• Gemini Ultra: Google's most capable model

• Gemini Pro Vision: Multimodal with image understanding

Configuration

{
  "provider": "google",
  "apiKey": "your-google-api-key",
  "model": "gemini-pro",
  "temperature": 0.7,
  "maxTokens": 2000
}

Pricing (Approximate)

• Gemini Pro: $0.0005/1K tokens

• Gemini Ultra: $0.001/1K tokens

Azure OpenAI

Best for: Enterprise deployments, compliance requirements

Available Models

• GPT-3.5 Turbo: Enterprise-grade GPT-3.5

• GPT-4: Enterprise-grade GPT-4

• GPT-4-32k: Extended context for enterprise use

Configuration

{
  "provider": "azure-openai",
  "apiKey": "your-azure-api-key",
  "endpoint": "https://your-resource.openai.azure.com/",
  "deploymentName": "gpt-4",
  "apiVersion": "2024-02-15-preview"
}

AWS Bedrock

Best for: AWS ecosystem integration, enterprise scale

Available Models

• Claude 3: Anthropic models via AWS

• Llama 2: Meta's open-source models

• Titan: Amazon's proprietary models

• Jurassic-2: AI21 Labs models

Configuration

{
  "provider": "aws-bedrock",
  "region": "us-east-1",
  "model": "anthropic.claude-3-sonnet-20240229-v1:0",
  "accessKeyId": "your-access-key",
  "secretAccessKey": "your-secret-key"
}

Specialized Providers

Mistral AI

Best for: European data residency, cost-effective alternatives

Available Models

• Mistral 7B: Efficient open-source model

• Mixtral 8x7B: Mixture of experts model

• Mistral Large: High-performance model

Configuration

{
  "provider": "mistral",
  "apiKey": "your-mistral-api-key",
  "model": "mistral-large-latest",
  "temperature": 0.7
}

Cohere

Best for: Business applications, command and control

Available Models

• Command: Business-focused language model

• Command-R: Enhanced reasoning capabilities

• Command Light: Faster, lighter version

Configuration

{
  "provider": "cohere",
  "apiKey": "your-cohere-api-key",
  "model": "command-r-plus",
  "temperature": 0.7
}

Groq

Best for: Ultra-fast inference, real-time applications

Available Models

• Llama 2: Fast inference of Llama models

• Mixtral: Fast inference of Mixtral models

• Gemma: Google's efficient models

Configuration

{
  "provider": "groq",
  "apiKey": "your-groq-api-key",
  "model": "llama2-70b-4096",
  "temperature": 0.7
}

DeepSeek

Best for: Advanced reasoning, mathematical problems

Available Models

• DeepSeek Chat: Advanced reasoning model

• DeepSeek Reasoner: Specialized reasoning model

Configuration

{
  "provider": "deepseek",
  "apiKey": "your-deepseek-api-key",
  "model": "deepseek-chat",
  "temperature": 0.7
}

Local Models

Ollama

Best for: Privacy, offline use, cost control

Available Models

• Llama 2: Meta's open-source models

• Llama 3: Latest Llama models

• Mistral: Mistral AI models

• CodeLlama: Code-specific models

• Falcon: TII's open-source models

• Vicuna: Fine-tuned Llama models

Configuration

{
  "provider": "ollama",
  "baseURL": "http://localhost:11434",
  "model": "llama2:7b",
  "temperature": 0.7
}

Installation

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Pull a model
ollama pull llama2:7b

# Start Ollama service
ollama serve

LM Studio

Best for: Local development, model experimentation

Available Models

• GGUF Format: Quantized models for efficiency

• Various Sizes: 7B, 13B, 70B parameter models

• Multiple Providers: Meta, Mistral, Google models

Configuration

{
  "provider": "lmstudio",
  "baseURL": "http://localhost:1234/v1",
  "model": "local-model",
  "temperature": 0.7
}

LocalAI

Best for: Self-hosted inference, custom models

Available Models

• Open Source Models: Various open-source alternatives

• Custom Models: Your own fine-tuned models

• Multiple Formats: GGML, GGUF, ONNX support

Configuration

{
  "provider": "localai",
  "baseURL": "http://localhost:8080/v1",
  "model": "gpt-3.5-turbo",
  "temperature": 0.7
}

Model Selection Guide

By Use Case

General Chat and Q&A

• Recommended: GPT-3.5 Turbo, Claude 3 Haiku

• Why: Cost-effective, fast, good for most tasks

• Use Cases: Customer support, general questions

Complex Analysis

• Recommended: GPT-4, Claude 3 Opus

• Why: Advanced reasoning, better understanding

• Use Cases: Document analysis, research, complex queries

Code Generation

• Recommended: GPT-4, CodeLlama

• Why: Code-specific training, better syntax

• Use Cases: Software development, code review

Creative Writing

• Recommended: GPT-4, Claude 3 Sonnet

• Why: Creative capabilities, style variation

• Use Cases: Content creation, marketing copy

Privacy-Sensitive

• Recommended: Local models (Ollama, LM Studio)

• Why: Data stays on your infrastructure

• Use Cases: Sensitive documents, compliance

By Performance Requirements

Speed Priority

• Fastest: Groq, GPT-3.5 Turbo

• Medium: Claude 3 Sonnet, GPT-4

• Slower: Claude 3 Opus, GPT-4-32k

Quality Priority

• Highest: GPT-4, Claude 3 Opus

• High: Claude 3 Sonnet, GPT-4o

• Good: GPT-3.5 Turbo, Claude 3 Haiku

Cost Priority

• Cheapest: Local models, GPT-3.5 Turbo

• Moderate: Claude 3 Haiku, Gemini Pro

• Expensive: GPT-4, Claude 3 Opus

Configuration Management

Environment Variables

# OpenAI
OPEN_AI_KEY=your-openai-api-key

# Anthropic
ANTHROPIC_API_KEY=your-anthropic-api-key

# Google
GOOGLE_API_KEY=your-google-api-key

# Azure OpenAI
AZURE_OPENAI_API_KEY=your-azure-api-key
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4

# AWS Bedrock
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=us-east-1

Model Configuration

{
  "defaultModel": "gpt-4",
  "fallbackModel": "gpt-3.5-turbo",
  "models": {
    "gpt-4": {
      "provider": "openai",
      "temperature": 0.7,
      "maxTokens": 2000
    },
    "claude-3-sonnet": {
      "provider": "anthropic",
      "temperature": 0.7,
      "maxTokens": 2000
    }
  }
}

Performance Optimization

Response Time Optimization

• Choose Faster Models: GPT-3.5 Turbo, Claude 3 Haiku

• Optimize Prompts: Shorter, more focused prompts

• Use Streaming: Stream responses for better UX

• Cache Responses: Cache frequent responses

Cost Optimization

• Model Selection: Choose appropriate model for task

• Prompt Optimization: Reduce token usage

• Response Limits: Set appropriate max tokens

• Batch Processing: Process multiple requests together

Quality Optimization

• Model Selection: Choose best model for task

• Prompt Engineering: Optimize prompts for better results

• Temperature Tuning: Adjust creativity vs. consistency

• Context Management: Provide relevant context

Troubleshooting

Common Issues

API Key Problems

• Invalid Key: Check API key format and validity

• Expired Key: Renew expired API keys

• Rate Limits: Check API rate limits and quotas

• Permissions: Verify API key permissions

Model Availability

• Model Not Found: Check model name spelling

• Region Restrictions: Verify model availability in your region

• Quota Limits: Check usage quotas and limits

• Service Status: Check provider service status

Performance Issues

• Slow Responses: Check network connectivity

• Timeout Errors: Increase timeout settings

• Memory Issues: Monitor system resources

• Concurrent Limits: Check concurrent request limits

Error Handling

{
  "error": {
    "type": "rate_limit_exceeded",
    "message": "API rate limit exceeded",
    "retryAfter": 60,
    "suggestion": "Wait 60 seconds before retrying"
  }
}

Best Practices

Model Selection

• Start Simple: Begin with GPT-3.5 Turbo or Claude 3 Haiku

• Test Performance: Evaluate models for your specific use case

• Consider Costs: Balance performance with cost

• Plan for Scale: Consider scaling requirements

Configuration Management

• Environment Variables: Use environment variables for API keys

• Model Fallbacks: Configure fallback models

• Monitoring: Monitor model performance and costs

• Documentation: Document model configurations

Security

• API Key Security: Secure API keys and credentials

• Data Privacy: Consider data privacy requirements

• Access Control: Implement proper access controls

• Audit Logging: Log model usage and access

---

🤖 Choose the right LLM provider for your needs. AINexLayer's model-agnostic architecture gives you the flexibility to optimize for cost, performance, privacy, or any combination of these factors.