Tradery

[vovay45]

How Tradery Database handles Yahoo data errors?
Like this:
BBBY

[koolaidluke]

Hi VY - At first I thought this must be the flash crash, but that was May 6. I checked QuoteMedia and Google. They both had the same numbers as Yahoo.

I've noticed that some trading algorithms are very much affected by the flash crash, which is ironic because, trading algorithms seem to be what caused the flash crash in the first place. (It might be a good idea to write an API object thingie that disallows trading on May 6, 2010, since that day was such an outlier.)

Anyway, I didn't check MSN, but if Yahoo, Google and QM all say the same thing I'd tend to think it probably wasn't a mistake.

Here's an example: VSH has 3 bars of data!

But to answer your question of how Tradery handles Yahoo mistakes, I'd say it doesn't. At least not until Yahoo fixes them itself and the Tradery database is refreshed. I know Adrian will say that there are 70,000 tickers that have to updated for this site, so error checking is not so easy a thing.

[vovay45]

Do you think this type of data provider error acceptable for Tadery and should not be filtered in database ?



VY

[koolaidluke]

Hi VY - I'm not disagreeing with you. I'm not saying that database error correction wouldn't be a good thing or that Yahoo doesn't make mistakes. I'm just saying that for the particular case you've chosen, I can find no independent corroboration that a mistake actually occurred. It looks screwy, I'll grant you that. But that doesn't prove it's actually wrong. Here's the same chart from StockCharts.com. StockCharts usually does a good job at checking their data and you can see their chart has the same characteristic.

But there are other examples (like VSH) where an error has undeniably occurred. If you just fix the simple ones, like missing bars, zero volume or volume that is misadjusted for a split, that would be a huge improvement. But I'm sure even that wouldn't be so easy to do.

[vovay45]

If we will not fix data errors than systems designed to catch such data errors will be the leaders in backtesting.
Look at 1 min bar chart I posted before.
Was this trades possible in real or virtual trading?


VY

[koolaidluke]

Hi VY - This is the sorta thing I do when I don't like the data:

[koolaidluke]

As luck would have it, just as I was defending the Tradery database, somehow, tonight it's became corrupted. Many symbol histories have been severely truncated, leaving just a few weeks of data. This is the type of error that I can't compensate for with cute little programming tricks. I hope this is only temporary.

[adrian]

All data should now be available. Let me know if you see any other problems.

Thanks,

Adrian