Finance Illustrated

AI Trading Claims: How to Check a Bot or Signal Before You Trust It

An AI label proves nothing on its own. Learn the questions that show what a trading claim really measures, and how to check the business before you pay anything.

Topic contributors: Axel OreUpdated 19 October 20265 min read

In this article
A magnifying glass examines a separate test report beside an AI-labelled laptop chart.

What does an AI trading claim really measure?

An AI label does not show that a trading tool is accurate, profitable or legal. A joint investor alert from the SEC, NASAA and FINRA warns about frauds that use AI as a selling point. It also warns that AI-generated information may be wrong. Its main lesson: check claims yourself, using sources you trust.

Turn each claim into a question
The claimThe question to ask
'The model predicted the market'Was each prediction recorded with a date before the outcome? Were all of them kept, including the misses?
'A profitable strategy'Is it a live result, a or a simulation? Are costs and losses included?
'90% accuracy'What counted as correct? How big were the losses?
'Regulated company'Which company exactly, in which country, and where is its entry in the official register?
'A celebrity uses it'Can you confirm that through the celebrity's own official channels?

Write the claim down exactly as it was made, without making it sound better. Then work through the questions one by one. A polished video or a confident chatbot answer is not evidence.

Five checks connect a claim to its original source, date, units and country, a reproduced calculation and remaining unknowns.
For each AI claim, work back to the original source, date, units and country. Reproduce what you can and leave unanswered questions visible.

Why can a great backtest be luck?

A backtest runs trading rules over past prices to see how they would have done. It is easy to test hundreds of rules and show only the best one. With enough tries, one will look brilliant by luck alone.

So ask when the record was made. Were the rules fixed first, then tested on prices that were not used to build them? Or were they adjusted until the past looked good? Our guide on how to backtest without fooling yourself explains the traps.

A , kept with dates while the trades happened, is much stronger evidence than any backtest. Even then, ask whether every account and every losing month is included.

Try it yourself

Find the 'genius' strategy, then test it

Press 'Generate 20 strategies' and find the best one. Then press 'Run it on new data' and see how it does next.

20 made-up strategies each win 50% of trades, like a coin flip, and risk 1% per trade over 100 trades. By chance, one ends ahead of the rest. On new data, every strategy still wins only 50% of the time, so the past winner has no edge.

What would you expect?

What this shows: All 20 strategies were coin flips. The best past record came from luck, so it says nothing about the next 100 trades.

Some sellers of AI bots show only the best of many tests. Ask for a live, dated record made before the rules were chosen, with every losing period included.

Can a '90% accurate' trading bot still lose money?

Yes. A ignores how big the wins and losses are. Compare two made-up demos. Each has 9 winning trades, 1 losing trade and a $0.20 cost on each of the 10 trades:

Same win rate, different results (made-up numbers)
MeasureDemo ADemo B
9 winning trades9 × $1 = $99 × $3 = $27
1 losing trade−$20−$5
Costs10 × $0.20 = $210 × $0.20 = $2
Net result$9 − $20 − $2 = −$13$27 − $5 − $2 = $20

Both demos are '90% accurate'. Demo A loses money. And even Demo B proves little: ten chosen trades say nothing about the losses that could come next.

Before you trust a headline number, ask for:

  • the full, dated record of every trade;
  • how a trade and a 'win' are defined;
  • how open trades are counted;
  • where the cost figures come from.

If you only get the winners, write 'not proven' instead of guessing that the rest were good.

How do you check the company behind an AI trading tool?

  1. Write down the exact claim, word for word.
  2. Note the company's legal name, website and country.
  3. Look it up yourself in the official register for that country. In the US, use FINRA BrokerCheck for brokers and advisers, and NFA BASIC for futures and forex firms. In Canada, follow the CSA's registration guidance. In the UK, use the FCA's firm checks.
  4. Find the original performance report, and label it live, simulated or backtested.
  5. List what is missing: fees, the worst fall, losing periods and withdrawals. Treat anything unknown as unknown, not as good news.

A register entry is not an endorsement. It also may not cover everything the firm offers, so check what it is allowed to do. The maths, the evidence and the register are three separate checks. Passing one does not answer the others.

What are the red flags in an AI trading offer?

  • Promised high returns with little or no risk.
  • A secret method that nobody can check.
  • Pressure to pay quickly, often in crypto.
  • A 'fee' or 'tax' you must pay before you can withdraw your profits.
  • Contact details that only the seller gave you.
A request for a release fee should trigger your own check of the company, its permissions and the payment demand. Contact details supplied by the sender cannot verify that sender.
Check through a route you found yourself. A rising balance on a dashboard cannot prove a payment demand is real.

Never use contact details from an unexpected message to check that same message. Find the company's details yourself. If you have already sent money, stop sending more and report it to your country's regulator.

The same patterns appear in crypto scams and in trading guru offers.

How can you use AI tools safely?

AI can still help you learn. You can ask it to draft study questions, explain a term or organise your trading journal. Check every important fact against an original source, and ask what data the tool can see and how old it is.

That is very different from letting software place real orders, or trusting a predicted price. The same rule applies to AI chart analysis in practice apps, including our own Trading Game. Treat it as a second opinion to question, not a forecast.

To build skills without risking money, start with paper trading. A simulation can teach you the process, while leaving any profit claim unproven.

Questions beginners ask

Are AI trading bots legit?

Some are real software, but the label proves nothing. Regulators warn that fraudsters use AI as a selling point. Check the company in the official register, ask for a full live record and never pay a fee to withdraw profits.

What does '90% accuracy' mean for a trading bot?

Usually that 9 in 10 trades or predictions were counted as correct. It says nothing about the size of the losses, the costs or how the trades were chosen. A bot can be right 90% of the time and still lose money.

Is a backtest proof that a strategy works?

No. A backtest shows how fixed rules would have done on past prices. If many rules were tried and only the best is shown, its result can be pure luck. A live, dated record is stronger evidence.

Can I use an AI chatbot to research stocks?

It can help you understand terms and draft questions. But it can be wrong or out of date, and it may sound sure when it is not. Check every fact in an original source before you rely on it.

Keep learning