Audits › Controls › The model cannot override the brakes
AI and learning · control 17 of 21
The model cannot override the brakes: does your trading bot have it?
If an AI model makes the trading decision, does that decision still pass through the deterministic controls (cap, breaker, exposure), or can the model set the size and skip the limits?
Why it matters
A model convinced of an opportunity sets a size ten times larger and nothing stops it.
How to check it in the code
- Find where the model output becomes an order and check which validators run after it.
- Check the model cannot change risk parameters (caps, leverage) at runtime.
- Check the risk engine runs on every model-originated order, including exits.
This is what the audit does for every bot in the list, on public code, without installing or running anything. Verdict values: present, partial, absent, not applicable, not verified. See the method.
Results across audited bots
Across 10 audited bots: 2 present, 4 partial, 1 absent, 2 not applicable, 1 not verified.
| Bot | Verdict | Evidence | Note |
|---|---|---|---|
| HKUDS/Vibe-Trading | Present | agent/src/live/advisory/__init__.py docstring: 'Advisory verdicts are purely observational: they never block, deny, or alter order execution. The mandate gate (check_mandate) remains the sole authority for order allow/deny decisions.' | Confirmed in order_guard.py: the advisory verdict is attached only as metadata after check_mandate has already allowed the order. |
| hummingbot/hummingbot | Not applicable | GET /search/code?q=openai OR anthropic OR ChatCompletion repo:hummingbot/hummingbot -> total_count 0 (2026-09-17) | This repo's execution engine does not integrate any LLM decision-maker itself. |
| Drakkar-Software/OctoBot | Absent | ai_index_distribution.py apply_ai_instructions(): the AI's proposed distribution instructions are applied directly to trading_mode.ratio_per_asset with only max(0, ...) floors and a final renormalize-to-100%; there is no check against a deterministic max-order-size, max-exposure, or circuit-breaker control before this feeds into ensure_index() and real order creation. | The only textual safeguard is a prompt instruction to the LLM itself ('Always respect risk limits and never recommend over-leveraging' in DISTRIBUTION_AGENT_INSTRUCTIONS, deep_agent_team.py) — not enforced in code. |
| freqtrade/freqtrade | Not verified | ||
| jesse-ai/jesse | Partial | README.md ML example: 'ml_predict_proba()' returns a bounded probability used inside the user's own should_long()/go_long() strategy code, which then calls the same self.buy/self.sell -> jesse/services/broker.py -> jesse/services/api.py path as any manual rule | The classical ML pipeline's output is just a signal gate inside ordinary strategy code, not a path that bypasses order submission; however, whether the separate generative 'AiModel' (provider/base_url/api_key/model_id, see jesse/models/AiModel.py) can size or submit orders directly could not be located in the time available. |
| chrisleekr/binance-trading-bot | Present | apps/web/src/features/backtest/components/backtest-llm-advisor.tsx (doc-comment: "Nothing touches live config, and the out-of-sample gate still decides go-live"); apps/web/src/features/backtest/lib/decision-breakdown.ts (doc-comment: suggestions "never bypass the bearish technical-rating veto") | The LLM has no write path to live trading. Every suggestion requires a human to load it into the Setup form, re-run a backtest, and clear an out-of-sample gate before it can reach a live profile; it can also only REMOVE a whitelisted entry constraint, never widen sizing or bypass the core veto gate. |
| alsk1992/CloddsBot | Partial | The primary order path (src/trading/orchestrator.ts) forces every guarded ExecutionService call through safety.canTrade()/validateTrade() and the circuit breaker before submission. BUT GitHub issues #126 and #127 (both OPEN, filed at the exact audited commit c930628) document confirmed AI-agent-invoked paths (Drift, Pump.fun, Meteora, the bundled /bf skill, and a direct Opinion-exchange fallback in src/agents/index.ts) that submit orders without going through those checks. https://github.com/alsk1992/CloddsBot/issues/126 , https://github.com/alsk1992/CloddsBot/issues/127 | The design intent is sound (single gatekeeper), but the model-invoked tool handlers have confirmed, currently-open bypasses. |
| Lumiwealth/lumibot | Partial | docs/AI_AGENT_FLOWS.md documents a 'Risk gate' pattern ('Python applies hard limits after an agent recommendation, such as max position size, no shorting, symbol allowlists, drawdown stops, or per-trade dollar limits') and a safety default ('LumiBot defaults the whole-agent retry budget to one attempt' when mutating order tools are available, to avoid duplicate order submission on retry). | |
| YizhiSong/FriesTrader | Partial | README states 'the actual safety mechanism is mechanical, auditable risk rules, not the model's judgment,' and every sizing/stop/gate value is computed by a standalone stdlib-only script the agent is told to 'use... directly rather than recomputing.' But PHASE_B_TASK.md's only enforcement of this is the written instruction 'Do not add, remove, or loosen any gate condition on your own judgment' -- the same agent session that reads a script's blocked/skip_buy JSON also holds the place_equity_order/place_option_order tool call (Step 6/8), and nothing in the repo code intercepts that call to verify it matches the script's output before it reaches Robinhood. | Phase A is better protected: its prompt explicitly excludes place_equity_order/place_option_order/review_*/cancel_* 'at the connector level if your MCP setup allows it' -- but that exclusion is optional and depends on the user's own MCP setup, not guaranteed by this repo. This is the single biggest gap between the README's safety claim and what the code actually enforces. |
| c9s/bbgo | Not applicable | GitHub code search for 'openai', 'package aiagent', 'chat/completions' in repo:c9s/bbgo all returned 0 results (2026-09-17) | No LLM-based decision-making was found anywhere in the strategy set. |
If your bot does not have it
The watchdog enforces daily loss, drawdown, position size and heartbeat limits from outside the bot, with a read-only key, whatever the bot's own code does.