My detector worked perfectly and was wrong for four days
On 3 August my recording stopped getting data at 15:15. It did the same on the 4th, the 5th and the 6th.
The shape of it was very clean. Around 3,300 messages a minute, steady all day, right up to 15:14. Then almost nothing: one lone update at 15:15, another at 15:20, a small burst around 15:28 that carried the closing price, and silence. The connection reported no error and never dropped. It stayed open and quiet.
My clean-day checker, which I had written on the 4th for exactly this kind of problem, marked all four days as damaged and recorded eight holes in the gap file. It did exactly what I had built it to do, and every answer it gave was wrong.
What I thought was happening
The pattern looked obvious. A feed that runs all morning and dies at the same time each afternoon is a broken connection, not a market. I had seen real broker outages before and this looked the same.
Nothing had changed on my side either. The same code, the same config, the same fifty stocks. It worked on 31 July and it did not work on 3 August.
So I treated it as an infrastructure problem. I built a monitor to spot the silence, reconnect, and get the data flowing again. On 6 August alone it ran eight reconnect cycles between 15:15 and 15:30. Not one of them brought back any data, because there was no data left to bring back.
That should have told me something. If a reconnect works and you still get nothing, the connection was never the problem. I decided the broker had a deeper fault, which is the kind of conclusion that keeps you inside your own story.
I wrote up a careful ticket: message rates, exact timestamps, the reconnect attempts, the working and non-working dates. It was a good bug report. It was about the wrong thing.
What was actually happening
The reply said the market timing structure had changed and the standard feed now runs to 15:15. What I was seeing was expected. A separate feed for the auction period is being built and is not available yet, and in the meantime the one late update I was still getting is the auction matching print between 15:25 and 15:30.
This is the bit worth being honest about. That is an easy answer to dismiss as support brushing you off, and I nearly did. Instead I checked it two ways, because an answer that solves my problem in my favour deserves more checking, not less.
Outside my own data, it holds up. SEBI brought in a Closing Auction Session on NSE and BSE, starting 3 August 2026, for stocks that have futures and options. Continuous trading in those stocks now ends at 15:15, and the official closing price comes from an auction instead of the old volume-weighted average: orders are collected, then there is a limit-only window with a random cut-off, then matching. The exchange had published all of this. I simply had not read it.
That last detail matters, because it is checkable. If the one surviving update is the auction match, then my final price of the day should have changed character on 3 August. And that is something I could test without asking anyone.
Inside my own data, and this is the part I keep thinking about, the proof had been there from day one.
The pattern I had never looked for
If the closing price changed from an average over the last half hour to an auction price struck right at the end, then one specific thing should happen to my recordings. My last tick of the day should stop being close to the official close and start being exactly the official close, because the auction price would arrive as my final update.
So I compared my last tick against the exchange’s published close, stock by stock, either side of 3 August.
Before 3 August: the typical difference was around 0.11 percent, and only a few percent of stocks matched exactly on any given day. That is what you expect when your last print is a single trade and the official close is an average over thirty minutes.
From 3 August: an exact match on all 50 stocks, with a typical difference of 0.0000 percent.
That is a step change rather than a slow drift. It happened on every stock at once, on the exact date the exchange had named. My list is the Nifty 50, which is entirely futures and options stocks, so all fifty changed together instead of trickling.
I had everything I needed before I raised the ticket. Nobody had run the comparison because nobody had any reason to. You do not test whether the closing mechanism of the market has changed. It is the sort of thing you assume you would have heard about.
What I changed
The clean-day checks now know about the date. One constant marks when the new structure started. Before it, trading is expected to run to 15:30. After it, 15:15. Gap detection stops at the right point for whichever day is being checked.
Old days are still judged by the rules they actually traded under. This matters more than it sounds. The tempting fix is to apply today’s rules to the whole archive so everything is consistent. That would re-mark two months of perfectly good recordings against a market that did not exist yet. Consistency bought by rewriting the past.
I added a check, not just a looser one. Under the old rules the test was “did the ticks run to the close”. Under the new rules that same test would pass on a day where I missed the auction completely, and the auction print now carries the official close, so missing it would be a genuine fault. There is now a check that says I captured it.
I withdrew the eight gap entries. A closed market does not count as a hole in the data. If I had left them there, every future study would have skipped a window with nothing in it, long after anyone remembered why.
Those four days are clean and always were.
What I actually learned
The obvious lesson is “read the exchange circulars”. Fair enough, and I have set a reminder. But that is not what four days actually taught me.
The real one is that an instrument can be completely correct and still be wrong, because the thing it is measuring moved.
My checker was not buggy. Every check it made was carried out exactly as specified. The specification described a market that had stopped existing the day before I wrote it. It reported honestly and my reading of it was out of date, and an out-of-date reading is invisible from the inside because everything agrees with everything else. The checker agreed with the monitor, which agreed with my theory about the broker, which agreed with the checker.
A closed system cannot work out that its own assumptions have expired. What broke the loop was checking against something outside it: the exchange’s published rules, and my data agreeing with a price I do not control.
That is a different discipline from most of what is on this site. The rest of it is about killing your own ideas, writing the pass mark down so you cannot move it, running the test once, publishing the failures. This one is about your equipment telling you the truth about a world that has moved on.
I do not have a general fix. What I have now is a habit of checking my measurements against something outside my own system, and treating “my data broke” as a guess rather than an answer. Four days was cheap. It was paid on paper with no money at risk, which is more or less the point of working that way.