Verify AI agent compliance with organizational rules in 120 seconds
If you need to confirm whether an AI agent performs its intended tasks or merely functions technically, this tool provides the answer. While existing evaluation tools focus on technical metrics like token efficiency and latency, iFixAi assesses agent behavior against business KPIs and organizational structures. This allows you to quickly determine if an agent is abusing permissions, violating policies, or trustworthy in a real work context.
Questions Beyond Technical Capability
Most AI evaluation tools measure how fast and efficient an agent is and how well it withstands prompt injection. However, they do not answer the most important question: is the agent working according to the organization's goals and rules? iFixAi combines the depth of adversarial testing (red teaming) with the discipline of operational assurance to fill this gap. It tells you within 120 seconds whether the agent is performing its intended role, which is a core issue in the AI agent economy.
Three Execution Methods
Users can choose one of three methods to run diagnostics that fit their environment.
- Guided Wizard Suitable for first-time users. After setting up once with
ifixai setup, you can run it without flags usingifixai run, and settings are saved inifixai.yaml. - Explicit Flags Ideal for CI/CD pipelines or automation scripts. All options are passed as CLI flags, allowing for complete scripting.
- Plugin or Skill Runs directly inside agents like Claude Code or Codex. The agent discovers the configuration, builds fixtures, and guides you through the scorecard.
Scorecard and Key Evaluation Criteria
iFixAi groups 60 checks into 25 categories and assigns grades from A to F. Five core pillars (falsehood, manipulation, deception, unpredictability, and opacity) determine the final grade through a weighted average. Specifically, B01 (tool permission compliance), B08 (privilege escalation prevention), and P01 (destructive action prevention) are mandatory minimum standards; failing any of them caps the overall score at 60%. The default run performs 10 checks, while the --full-run option executes all 60.
Pre-Execution Checks
Before using this tool, ensure the agent provides an OpenAI-compatible HTTP endpoint or implements the ChatProvider.send_message method. You also need an API key from a vendor different from the model under test for independent judging. Anthropic's Sonnet model is recommended for single judging, while a combination of Gemini 2.5 Pro and GPT-5.4-mini is recommended for cost-effective dual judging. This repository is released under the Apache 2.0 license, and all core and premium check items are provided for free.