Assumptions, Costs, and Overfitting

The 6:42 That Rohan Let Go

The platform at Andheri was doing what it always did at 6:30 in the evening — too many people, one working fan, and a voice overhead announcing that the 6:42 local was running fifteen minutes late.

Rohan leaned against a pillar and opened his phone. His colleague had shared a PDF that afternoon with just one line: "Bro, check this strategy. Insane numbers."

He checked it now.

The chart on page two made him sit up a little straighter. A single line, climbing almost without pause, left to right, ten years of it. No real dips. No ugly stretches. Just a smooth, confident staircase to a number that made his stomach do a small, hopeful flip. Eighteen percent a year. Every year. For a decade.

He scrolled back up to check he hadn't misread it. He hadn't.

Rohan had seen enough charts by now — jagged ones, humbling ones — to know that markets didn't usually behave like well-behaved students. This one did. And some part of him, the part that was already imagining what eighteen percent a year could do to his SIP money, wanted very badly to just believe it.

The announcement crackled again. Ten minutes now.

He typed a message to Kavya, who'd been trading longer than him and had a habit of asking annoying questions at exactly the wrong moment. Look at this backtest. Looks unreal. Should I just start following this rule?

Three dots appeared almost instantly.

Before you get excited — does it say anywhere what happens when you actually buy or sell? Like, at what price?

Rohan frowned and scrolled down. The methodology section was short. "Buy when signal triggers, sell at target." Nothing about brokerage. Nothing about the price sometimes moving between the moment the rule said "buy" and the moment an order actually went through.

Doesn't say, he typed.

And how many different settings does the rule use? Like specific numbers — days, percentages?

He scrolled again. There they were, buried in a footnote: a 23-day average, a 2.7% threshold, three other adjustable numbers. Five in total. Each one oddly specific, like a lock with five separate combinations, all somehow set to the exact numbers that made this exact ten years look flawless.

The train's headlight appeared far down the track. People around him began shuffling forward, bags swinging onto shoulders.

Rohan didn't move yet.

He looked at that smooth, staircase line again — and for the first time, it didn't look like proof. It looked like a question he hadn't asked yet.

The 6:42 pulled in, doors opening, the crowd surging past him. Rohan let it go.

He had a few more questions for Kavya before he'd let this chart anywhere near his money.

• Explain why ignoring real trading costs and slippage makes a backtest's returns look better than they would be in real trading.
• Explain what overfitting is and why a rule tuned tightly to past data can fail on new data even when the past-data math is completely correct.
• Given a backtest result, identify at least two hidden assumptions or fitting choices worth questioning before trusting the result.

A backtest is only useful if you trust it, and you should only trust it after you've checked what it's hiding. That's the real reason this matters: people don't lose money because the math in a backtest is wrong. They lose money because the math is right, but built on a story that leaves things out.

Here's the truth underneath it: a backtest's clean-looking results can be quietly inflated in two separate, invisible ways — and both can happen even when every calculation is technically correct.

The first way is ignoring real trading friction. Most simple backtests assume every trade fills exactly at the target price, with no fees. But real trading isn't that tidy. There's slippage — the gap between the price you expected and the price you actually got, because time passes and prices move between your decision and your order filling. And there are transaction costs — brokerage, taxes, fees — that quietly take a bite out of every single trade. Strip both of these out of a simulation, and even an average strategy starts looking excellent. Put them back in, and the same strategy can look ordinary, or worse.

The cleaner a backtest looks, the more it deserves a second look, not more trust.

The second way is overfitting. A trading rule usually has adjustable settings — how many days to average, what percentage counts as a signal, and so on. If someone (even with good intentions) keeps adjusting those settings until the rule's returns look as good as possible on one specific stretch of past data, the rule isn't finding a real pattern anymore. It's memorizing that one stretch — noise included. It will explain that past perfectly, because it was shaped to fit it. That's exactly why it often falls apart on new data it hasn't seen.

Notice what these two problems have in common: neither shows up as an error. The arithmetic can be flawless. The code can run perfectly. And the backtest can still be misleading, because the problem isn't in the calculation — it's in the assumptions and the fitting choices behind the calculation.

So the cleaner a backtest looks, the more it deserves a second look, not more trust.

That's the shift this concept asks of you: stop treating a smooth, impressive result as proof, and start treating it as an invitation to ask two specific questions. First: did this account for real trading costs and slippage? Second: how many settings were tuned to make this exact historical period look good — and would the same rule still work on a different period? You don't need to run any calculations to ask these questions. You just need to remember that a backtest is a map of old terrain, and a map can look beautiful while still leaving out exactly the parts that would have mattered.

Rohan let the 6:42 pull away without him. He stayed against the pillar, phone to his ear now, Kavya's voice coming through as the platform slowly emptied.

"Okay, read me the methodology section again," she said.

He scrolled to it. "Buy when signal triggers, sell at target. That's basically it."

"No mention of brokerage anywhere? No slippage assumption?"

"Nothing. It just says 'trades executed at signal price.'"

"So every single trade in ten years filled at exactly the number the rule wanted, with zero cost to you. That's not how your broker works, right?"

Rohan laughed a little, despite himself. "No. Definitely not."

"Now scroll to the footnote with the settings. Read me the numbers."

He found it — small text at the bottom of page four. "23-day average. 2.7% threshold. Three more after that. Five total."

"Five specific, tuned numbers," Kavya said. "Ask yourself — why 23 days and not 20, or 30? Why 2.7%, not 3%? Those aren't round numbers. Someone tried a bunch of combinations until this exact set made this exact ten years look best."

Rohan stared at the smooth staircase chart again. It hadn't changed. But he had. "So it's not that the numbers are fake. It's that... this might just be the one combination that happened to fit this one stretch of history."

"Right. And a different ten years — or the next one — might not agree with it at all."

He didn't message his colleague back yet. Instead he saved two lines in his notes: Did this include real costs and slippage? How many settings were tuned to fit this exact period? The eighteen-percent number hadn't moved. But it had stopped being an answer. It was just a question now, waiting to be checked.

Point: Two separate, invisible problems — ignored real-world costs/slippage and overfitting to one historical stretch — can both quietly inflate a backtest's headline number even when its arithmetic is completely correct.

• Believing a smooth, rising backtest line proves the strategy will make money in the future. — Watching a strategy 'work' consistently across years of real historical data feels like objective proof, since the data is genuine and the math checks out. Fix: Remember a backtest shows how a rule performed under specific assumptions on one particular history — treat it as informative about the past, never as a guarantee about the future.
• Assuming that if a backtest's code and math are error-free, its results must be trustworthy. — Learners equate 'no calculation mistakes' with 'no hidden problems,' so a bug-free report feels automatically reliable. Fix: Separate two different questions: 'Is the arithmetic correct?' and 'Were the assumptions and fitting choices honest?' A backtest can pass the first and still fail the second.
• Thinking overfitting only happens when someone is careless or dishonest. — The word sounds like a mistake or a trick, so it feels like it requires bad intent or sloppy work. Fix: Know that overfitting can happen even with careful, well-meaning tuning — the more adjustable settings a rule has, the easier it is to accidentally fit noise instead of a real pattern.

• A backtest can be arithmetically perfect and still be misleading — the problem hides in assumptions and fitting choices, not in calculation errors.
• Ignoring real trading costs and slippage makes almost any strategy look better than it would perform in real trading.
• Overfitting is tuning a rule so tightly to one stretch of past data that it memorizes noise instead of finding a real pattern.
• Before trusting any backtest, ask two questions: did it include real costs and slippage, and how many settings were tuned to fit this exact period?
• The cleaner and more impressive a historical result looks, the more it deserves scrutiny, not blind trust.

Q1. In a backtest, what does 'slippage' mean?

• The gap between the price you expected and the price you actually got when your order filled
• The total profit a strategy makes over ten years
• A software bug in the backtesting code
• The number of settings a trading rule uses Answer: The gap between the price you expected and the price you actually got when your order filled — Slippage happens because time passes and prices move between when a signal triggers and when the order actually fills. Simple backtests often ignore this gap entirely.

Q2. True or False: A backtest can have completely correct math and still give misleading results. Answer: True — The arithmetic can be flawless while the assumptions (ignoring costs/slippage) or the fitting process (overfitting) still quietly distort what the results actually mean.

Q3. A trading rule has eight different adjustable settings, and every one of them was fine-tuned until the strategy's return looked as high as possible on one specific ten-year period. What does this suggest?

• The rule may be overfit, meaning it could be memorizing that period's noise rather than a pattern that transfers to new data
• The rule is definitely fraudulent and was built to deceive investors
• The rule is more reliable because it was tested for a long time
• The rule's brokerage costs are automatically included since it was carefully tuned Answer: The rule may be overfit, meaning it could be memorizing that period's noise rather than a pattern that transfers to new data — Having many settings tuned specifically to fit one exact stretch of data is a classic sign of overfitting — the rule explains that past perfectly because it was shaped to fit it, which is why it often fails on new data.

Q4. You're shown a backtest report claiming a smooth, steady 20% annual return over 12 years, with no mention of brokerage, taxes, or price slippage anywhere in the methodology. Before considering using this strategy with real money, what should you do first?

• Ask whether real trading costs and slippage were included, and check how many settings were tuned to fit that exact historical period
• Trust the result immediately since the return line looks smooth and consistent
• Assume the report is fraudulent and discard it without further thought
• Focus only on checking that the percentage math adds up correctly Answer: Ask whether real trading costs and slippage were included, and check how many settings were tuned to fit that exact historical period — A clean, impressive result is an invitation to investigate, not a reason to trust more. Checking for hidden costs/slippage and overfitting are the two specific questions worth asking before risking real money.

Q5. A backtest shows a strategy returning 25% a year with zero mention of brokerage, taxes, or slippage. Someone says: "The numbers are the numbers, real trading should match this closely." Should real trading match a backtest that ignores costs? Reveal: Weak: yes, if the math checks out, real results should be close. Strong: a backtest that skips brokerage, taxes, and execution slippage is measuring an idealized world — real trading always involves some cost and delay, so actual returns will run lower than the simulated ones.