emmajerome
New Member
There are numerous financial data providers and unfortunately very few offer many details on their process for collecting and processing the data. As such, it can be very difficult to assess the quality of the data from the provider’s website. There are however several principles an analyst can use in determining the quality of the end of day data and intraday data.
1. Is the data intraday data or end of day data?
End-of-day data can easily be found for free from a variety of providers or direct from the exchanges. End-of-day data is usually very reliable and consistent. However, note that for indices, the volumes can differ between providers. This is because most indices are specified on a single price metric (such as a composite of 500 share prices as with the S&P500) but there is no specification for volume and so this is manually calculated by each provider and can vary due to the different methods of aggregating the individual stock volumes.
Intraday data, however, is very different and varies greatly in consistency and quality across different providers. The remainder of this article is focused on examining the quality of the intraday data providers.
2. Is the data direct from the exchange or via a third-party provider?
The most reliable source of intraday data is direct from the exchanges. However, this is often not practical for smaller institutions or individual traders since the cost is usually very high and some exchanges will only offer large institutional investors (such as brokerages) for their data feeds.
Thus, many traders and hedge-funds will use a third-party data provider for their intraday data. Unfortunately these providers vary greatly in quality. In general we found two key differentiators - do the providers provide samples and do they disclose where they source the data from (always prioritize providers that source the data directly from the exchanges as opposed to those who do not disclose the source of their data or that source from other third-party providers).
For example, at FirstRate Data we source all our intraday data directly from the exchanges. Index datasets such as S&P 500 Intraday are sourced direct from two authorized compilers of the price index and tested against eachother.
3. How much information does the data provider give on their data-cleaning?
All intraday data will need to be cleaned and tested before use. Good data providers should always have a process for cleaning and testing their data. Typical issues such as testing for gaps in the time-series or outliers (possible bad traders) should be checked. In addition, the provider should disclose how stock-splits and dividends are dealt with since both a stock split or a dividend payment should not impact the time-series.
4. What post-sales support do the data providers offer?
We found that strong customer support is a good indicator of the provider's data quality. Prior to purchasing any intraday data we would recommend contacting the provider to request a sample (or a sample for a different time frame, if a sample is already available). Also enquire as to the data cleaning protocols and the source of the data. In general we exclude any provider that has a response time of longer than 48 hours.
1. Is the data intraday data or end of day data?
End-of-day data can easily be found for free from a variety of providers or direct from the exchanges. End-of-day data is usually very reliable and consistent. However, note that for indices, the volumes can differ between providers. This is because most indices are specified on a single price metric (such as a composite of 500 share prices as with the S&P500) but there is no specification for volume and so this is manually calculated by each provider and can vary due to the different methods of aggregating the individual stock volumes.
Intraday data, however, is very different and varies greatly in consistency and quality across different providers. The remainder of this article is focused on examining the quality of the intraday data providers.
2. Is the data direct from the exchange or via a third-party provider?
The most reliable source of intraday data is direct from the exchanges. However, this is often not practical for smaller institutions or individual traders since the cost is usually very high and some exchanges will only offer large institutional investors (such as brokerages) for their data feeds.
Thus, many traders and hedge-funds will use a third-party data provider for their intraday data. Unfortunately these providers vary greatly in quality. In general we found two key differentiators - do the providers provide samples and do they disclose where they source the data from (always prioritize providers that source the data directly from the exchanges as opposed to those who do not disclose the source of their data or that source from other third-party providers).
For example, at FirstRate Data we source all our intraday data directly from the exchanges. Index datasets such as S&P 500 Intraday are sourced direct from two authorized compilers of the price index and tested against eachother.
3. How much information does the data provider give on their data-cleaning?
All intraday data will need to be cleaned and tested before use. Good data providers should always have a process for cleaning and testing their data. Typical issues such as testing for gaps in the time-series or outliers (possible bad traders) should be checked. In addition, the provider should disclose how stock-splits and dividends are dealt with since both a stock split or a dividend payment should not impact the time-series.
4. What post-sales support do the data providers offer?
We found that strong customer support is a good indicator of the provider's data quality. Prior to purchasing any intraday data we would recommend contacting the provider to request a sample (or a sample for a different time frame, if a sample is already available). Also enquire as to the data cleaning protocols and the source of the data. In general we exclude any provider that has a response time of longer than 48 hours.
