Features - Portkey Docs

AI Gateway

Connect to 3,000+ AI models using a single consistent API. Set up load balancers, automated fallbacks, caching, conditional routing, and more, seamlessly.

🌐 Universal API

Integrate with multiple AI models through a single API

💾 Caching

Implement simple and semantic caching for improved performance

🔄 Fallbacks

Set up automated fallbacks for enhanced reliability

🔀 Multimodal Capabilities

Handle various data types with multimodal AI capabilities

🔁 Automatic Retries

Implement automatic retries for improved resilience

🔌 Circuit Breaker

Configure per-strategy circuit protection and failure handling

⚖️ Load Balancing

Distribute workload efficiently across multiple models

🔑 Integrations

Manage access to LLMs

⏱️ Request Timeouts

Set and manage request timeouts

🐤 Canary Tests

Implement canary testing for safe deployments

🚦 Conditional Routing

Route requests based on specific conditions

💳 Budget Limits

Set monthly or custom budget limits on LLM usage, at provider or API key level

⏳ Rate Limits

Programmatically set rate limits at API key level and prevent abuse

Observability & Logs

Gain real-time insights, track key metrics, and streamline debugging with our OpenTelemetry-compliant system.

📜 Logs

Access and analyze detailed logs

🕵️ Tracing

Implement distributed tracing for request flows

📊 Analytics

Gain insights through comprehensive analytics

🧪 Filters

Apply filters for targeted analysis

🏷️ Metadata

Manage and utilize metadata effectively

💬 Feedback

Collect and analyze user feedback

Guardrails

Enforce Real-Time LLM Behaviour with 50+ state-of-the-art AI guardrails, so that you can synchronously run Guardrails on your requests and route them with precision.

🛡️ Deterministic Guardrails

Implement rule-based safety checks

🤖 LLM-based Guardrails

Leverage AI for advanced content filtering

🤝 Partner Guardrails

Integrate third-party safety solutions

🎒 Bring Your Own

Customize guardrails to your needs

🏛️ Central Guardrails

Deploy org-wide guardrails like PII redaction and company policy adherence across all AI calls

Agents

Natively integrate the AI Gateway’s routing, guardrails, and observability suite with leading agent frameworks and take them to production.

🤖 Autogen

👥 CrewAI

🔗 Langchain Agents

🦙 Llama Agents

φ PhiData

🔀 ControlFlow

🕸️ LangGraph

Governance & Administration

Organise your teams and entities with the access control, encryption, and audit trails that an enterprise AI platform needs.

📁 Org Management

Organise your teams and entities with more access control in the AI Gateway

🔒 SSO Integrations

Seamless integrations with your preferred Single Sign-On providers like Okta and Microsoft

👤 Access Rules

Team and individual level RBAC for AI services, LLMs, logs, and more

🗂️ Audit Logs

System-wide audit trails for all AI services and all the AI Gateway features described above

🔑 KMS Integration

Bring your own encryption keys to the AI Gateway to encrypt data at rest

🗄️ Data Isolation

Ensure the highest level of data privacy and security with isolated storage infrastructure

📅 Custom Retention Periods

Set custom retention periods for different users to meet your data storage requirements

❄️ Export to Data Lakes

Easily export all your data to your preferred data lake for long-term storage

📝 Executive Reporting

Executive-only dashboards to track AI adoption, ROI, and strategic impact across your organisation

​AI Gateway