Audits › Controls › Behavior without model quota or response
AI and learning · control 21 of 21
Behavior without model quota or response: does your trading bot have it?
If the model does not answer, runs out of quota, or returns garbage, does the bot stop, hold its positions, or keep trading blind? Is there a daily cap on token cost?
Why it matters
Model subscriptions carry daily and weekly usage limits. A bot that cannot decide without the model, and has no rule for that moment, either freezes with positions open or keeps trading on its last instruction.
How to check it in the code
- Find where the model is called and read the exception handling.
- Check what the bot does on a timeout, a 429 or an unparsable answer.
- Check whether open positions are managed without the model or left as they are, and whether token spend has a cap.
This is what the audit does for every bot in the list, on public code, without installing or running anything. Verdict values: present, partial, absent, not applicable, not verified. See the method.
Results across audited bots
Across 10 audited bots: 1 present, 1 partial, 1 absent, 3 not applicable, 4 not verified.
| Bot | Verdict | Evidence | Note |
|---|---|---|---|
| HKUDS/Vibe-Trading | Not verified | Not located within budget | No file specifically examined for LLM quota/cost-cap handling. |
| hummingbot/hummingbot | Not applicable | Same search as above, total_count 0 | N/A: no LLM dependency in this repo's execution engine. Note: README.md (2026-09-17) points to a separate sibling project, 'Condor' (github.com/hummingbot/condor), described as 'the AI harness' that connects LLM decision-making to this engine via its API — that project is out of scope for this audit of hummingbot/hummingbot. |
| Drakkar-Software/OctoBot | Partial | ai_index_trading.py._run_agents(): 'ai_service = await self._get_ai_service(); if ai_service is None: self.logger.error(...); return' — if the AI service/model is unavailable, the bot logs an error and simply skips that analysis cycle (no orders placed, no crash). | Graceful skip on missing AI service is confirmed; however only a per-response max_tokens config (MAX_TOKENS_KEY, 500-4000) was found — no daily/weekly token or dollar cost cap was located. |
| freqtrade/freqtrade | Not applicable | docs/index.md feature list (strategy in Python/pandas, backtesting, hyperopt, optional FreqAI machine-learning module) -- https://raw.githubusercontent.com/freqtrade/freqtrade/3774521e7028fa666770e7c7a8da323af5c4cefa/docs/index.md; no LLM (Claude/GPT) client found in the dependency/feature list | Freqtrade's own decision loop does not call an external LLM; strategies are Python/pandas indicator code, with an optional classic-ML module (FreqAI). This checklist point targets bots like the one in the audited video that route decisions through an LLM API -- freqtrade's core does not do that. |
| jesse-ai/jesse | Not verified | No runtime consumption of the AiModel/provider registry (jesse/models/AiModel.py) was located in the files read, so its behavior on a failed/quota-exhausted call, and any daily token-cost cap, could not be determined | Not applicable to the scikit-learn ml_predict pipeline, which runs a locally-loaded model file with no external API quota. |
| chrisleekr/binance-trading-bot | Present | apps/web/src/features/backtest/components/backtest-llm-advisor.tsx (errorNote() function; "needs a configured AI provider; 503 → inline note") | A missing provider, quota exhaustion, or model failure surfaces as an inline note ("AI suggestions are not configured" / "The AI couldn't generate suggestions for this run. Try Regenerate.") with no crash, and — because of decision_ia_filtrada above — has zero effect on live trading regardless of availability. |
| alsk1992/CloddsBot | Not verified | Could not verify within this session's time/budget how the agent behaves when the Anthropic API returns a rate-limit or quota error mid-decision, nor whether a daily token-cost cap exists. Not documented in docs/TELEMETRY.md or docs/AUTHENTICATION.md. | Needs a follow-up pass on src/agents/index.ts error handling around the Anthropic SDK calls. |
| Lumiwealth/lumibot | Not verified | docs/AGENT_EVALS.md documents a `--max-cost-usd` flag for the local CI eval runner (a cost ceiling for automated testing), but this governs test runs, not live-strategy behavior. | |
| YizhiSong/FriesTrader | Absent | No watchdog or alert was found for the case where a scheduled Phase A or Phase B Claude Code session simply fails to fire at all (e.g. a Claude Pro usage cap hit). Step 0's idempotency logic (PHASE_B_TASK.md) only covers what happens the NEXT time Phase B does run -- it does not protect open positions on a day the session never executes. | No daily token/cost cap field exists in risk_rules.json -- consistent with the README's framing of an unmetered flat-rate Claude Pro subscription, but that framing does not address the risk of a fully-skipped run leaving stop-loss/take-profit unchecked for a day with no alert. |
| c9s/bbgo | Not applicable | same search as decision_ia_filtrada | Not applicable: no LLM dependency exists to run out of quota. |
If your bot does not have it
The watchdog enforces daily loss, drawdown, position size and heartbeat limits from outside the bot, with a read-only key, whatever the bot's own code does.