Alternative Data Sources Every Hedge Fund Should Be Monitoring
There was a time when getting an informational edge meant having access to research that everyone else didn’t. Today, the bigger challenge is often working out what to pay attention to when there is an extraordinary amount of information sitting in public view.
Think about a retailer heading into an important quarter. Long before its results are published, thousands of small changes may already be happening online. Products are moving in and out of stock, prices are being adjusted, promotions are appearing, customer reviews are accumulating, and competitors are making changes of their own. None of those signals tells you exactly what the company’s earnings will look like, but followed over time and viewed together, they can start to reveal how the business and the market around it are moving.
That’s a big part of the appeal of alternative data for hedge funds. Traditional financial information still matters enormously, but investors no longer have to wait for an earnings call or quarterly filing to start building a picture of what’s happening. Public web data can provide signals between those reporting periods, giving research teams another way to test an investment thesis against what they’re seeing in the market.
Of course, having more data doesn’t automatically give you an edge. There are millions of possible signals on the public web, and plenty of them will turn out to be noise. The interesting work lies in finding sources that have a meaningful relationship with the question you’re trying to answer, collecting them consistently enough to see changes over time, and knowing when the pattern deserves a closer look.
Build Better Alternative Data Pipelines
Collect reliable public web data across markets, websites, and regions for investment research.
What Makes Alternative Data Useful?
Alternative data is a broad term, which is partly why conversations about it can become vague very quickly. For an investment team, the more useful question is usually what a particular dataset can tell you that you couldn’t easily see through traditional financial reporting.
Timing is one obvious advantage. Company filings give investors detailed information, but they describe a period that has already happened. Public web data is constantly changing, which means it can sometimes provide a much more current view of the things happening around a company. If you’re following online pricing, product availability, hiring activity, or customer interest, you’re observing signals as they develop rather than several weeks or months later.
The real value tends to emerge once you have enough history to know what normal looks like. Seeing that a retailer has 8,000 products in stock today doesn’t tell you very much on its own. If you’ve tracked its assortment for two years and usually see inventory building at this point in the season, a sudden contraction becomes considerably more interesting. You still need to understand why it’s happening, but now you have something worth investigating.
That’s an important distinction because alternative data rarely hands you an investment decision neatly packaged and ready to use. It gives analysts another lens through which to examine a company, sector, or wider market, and its usefulness depends heavily on the questions they ask of it.
Online Pricing Can Reveal More Than Who’s Cheapest
Pricing data is one of the more obvious places to start because it’s public, constantly changing, and closely connected to how businesses compete. Tracking those movements across retailers can give investment teams a view of market behavior that would be difficult to build manually.
Imagine following the same group of products across several major retailers for a year. Over time, you begin to see which businesses tend to discount first, how competitors respond, and whether promotional periods are becoming longer or more aggressive. If a company that historically holds its prices suddenly starts discounting heavily across an important category, that’s a much more interesting signal than discovering that one laptop happens to be $50 cheaper on a Tuesday morning.
The same data can help teams look at pricing power. A business that repeatedly raises prices without competitors undercutting it may be operating in a very different environment from one that appears locked into constant discounting. Combine that with availability and assortment data and the picture becomes richer again, because a higher price on a product that’s almost impossible to find may tell a different story from the same increase in a heavily stocked category.
None of these observations should be treated as proof of financial performance on their own. They’re signals that can be compared with the investment thesis and investigated alongside other information. That’s where alternative data tends to be at its most useful: helping analysts notice changes earlier and giving them another set of questions to ask.
Availability and Assortment Can Show How a Market Is Moving
Prices get plenty of attention, but what companies are selling and whether customers can get hold of it can be just as revealing.
Tracking product availability over time can help analysts see where inventory conditions are changing. A product disappearing from several retailers might reflect strong demand, a supply problem, or a planned transition to a newer model. You won’t know which explanation is correct from the stock status alone, but a sustained change across a category gives the research team somewhere useful to dig.
Assortment data can tell a different story. Watching the size and composition of a retailer’s catalog over time can reveal where it’s expanding, pulling back, or placing new bets. If a business gradually adds hundreds of products to a particular category while reducing its presence elsewhere, that shift is visible online well before anyone necessarily discusses it in a financial presentation.
Competitor activity makes those changes even more useful. You can see whether one retailer is moving alone or whether the whole market is heading in the same direction, which helps put company-specific signals into context. Over time, you’re building a record of how the competitive landscape changes rather than relying on occasional snapshots of what happened to be on the website when someone checked.
Search Data Can Offer a Window Into Demand
Sometimes the interesting signal isn’t what a company is doing but what people appear to be looking for.
Search behavior can help investment teams follow changes in attention around products, brands, and categories. That might mean tracking how search engine results change, looking at the terms appearing around a particular market, or following rankings to understand which businesses are gaining or losing visibility.
The important part is the trend rather than any individual search result. Rankings move for all sorts of reasons, and a sudden spike in interest can disappear just as quickly as it arrived. When the same movement continues over weeks or months, however, it can become another useful piece of evidence about how demand or competitive visibility may be changing.
There’s also value in comparing search signals with what’s happening elsewhere. Growing interest in a product becomes more interesting if retailers are simultaneously expanding their assortments or struggling to keep it in stock. A brand losing search visibility while competitors increase their presence may deserve further investigation, particularly if the pattern starts appearing across several markets.
This is where combining alternative datasets starts to become much more powerful than monitoring each source in isolation. No single signal has to carry the entire investment thesis. Instead, analysts can look for places where several independent observations begin pointing in the same direction.
Reviews and Ratings Can Show What Customers Are Experiencing
Customer reviews are messy, emotional, and occasionally written by someone furious that a toaster didn’t arrive before their holiday. That doesn’t make them useless. At sufficient scale, reviews can provide a surprisingly detailed view of how customers are responding to a product or business.
The interesting part isn’t whether one customer leaves a five-star review or another gives a company one star. It’s what happens across thousands of reviews over time. Analysts can follow changes in review volume, average ratings, and the subjects customers repeatedly mention, then look at whether those patterns line up with other developments around the business.
Imagine a consumer brand launches a new version of one of its biggest products. Reviews start appearing quickly, but complaints about a particular feature keep coming up and the average rating begins drifting downward. A few negative comments wouldn’t mean much, but if the pattern continues while competing products are receiving stronger feedback, there’s something there for an analyst to investigate.
Review volume can be interesting for similar reasons. A sudden increase might reflect stronger sales, a successful launch, or a promotional push, although it needs to be interpreted carefully because the relationship between purchases and reviews varies enormously between platforms. Used alongside pricing, availability, and other signals, customer feedback can add another perspective on how a product is performing in the market.
Hiring Activity Can Tell You Where a Company Is Investing
A company’s careers page can reveal quite a lot about where it’s putting its attention long before those plans show up elsewhere.
One job posting won’t tell you much, but patterns in hiring can. If a company starts building out an engineering team in a new country, recruiting heavily for a particular product area, or opening dozens of roles connected to a new capability, those changes can offer clues about where investment is heading. Following the same company over time makes it easier to distinguish a genuine shift from the normal churn of vacancies being opened and filled.
The details within those postings can be useful too. Job descriptions often mention technologies, teams, locations, and responsibilities that provide more context about what the company is building. When similar roles begin appearing repeatedly, analysts can compare that activity with management commentary and other public information to see whether the company’s actions line up with the strategy it’s describing.
There are plenty of reasons to be cautious with the interpretation. A vacancy doesn’t guarantee someone will be hired, and a disappearing job posting doesn’t necessarily mean the role has been filled. What matters is having enough historical data to recognize meaningful changes in hiring behavior rather than drawing conclusions from a handful of listings.
Marketplaces Offer a View of the Competitive Landscape
Marketplaces are particularly rich sources of alternative data because so much competitive activity happens in one place. Search for a popular product and you may find established retailers competing alongside smaller merchants, different versions of the same item, changing prices, varying delivery promises, and sellers moving in and out of prominent positions.
Following that activity over time can help analysts understand how competitive a market is becoming. A brand that once dominated a category might gradually find itself surrounded by cheaper alternatives, while another could begin appearing more frequently across popular searches and categories. Seller counts can change as well, giving analysts another way to watch how crowded particular products or markets are becoming.
The same information can help when researching the businesses behind those marketplaces. Changes in seller activity, product selection, availability, and pricing can provide some indication of what’s happening across the ecosystem rather than relying solely on figures published by the platform itself.
Marketplaces are also a good example of why collection frequency matters. What you see on Monday morning may look very different by Friday afternoon, particularly in fast-moving categories. Building a historical record makes those movements much easier to interpret because analysts can see whether something represents a genuine shift or another variation in a market that changes constantly anyway.
Build a Stronger Research Dataset
Support investment analysis with consistent public web data collected across global markets.
Travel Data Can Reveal Changes in Demand
Travel websites provide another enormous pool of public information, particularly for funds researching airlines, hotels, tourism businesses, or the wider economies that depend on them.
Consider what happens when someone searches for a flight or hotel. They’re shown current prices and availability for a particular destination and date, and those results continue changing as demand develops. Collect the same routes, destinations, and travel periods consistently and you can begin building a picture of how conditions are moving.
Hotel availability tightening ahead of a particular period could suggest stronger demand, while changes in flight pricing across frequently monitored routes may be worth comparing with capacity or wider travel trends. The important thing is maintaining enough consistency in the searches to make the results comparable. A flight price collected six months before departure isn’t directly comparable with one collected the night before, so the collection methodology matters just as much as the number you eventually analyze.
This is another area where geographic context can become important. Search results and prices aren’t always identical for users in different markets, which means an investment team trying to understand what a particular group of customers is seeing may need to collect the data from the relevant locations.
Real Estate Listings Can Show Changes Before Official Data Catches Up
Property markets generate plenty of traditional statistics, but public listings can give analysts a much closer view of what’s happening while properties are still being marketed.
Rather than waiting for completed transactions to appear in official figures, teams can follow asking prices, the number of available properties, how long listings remain active, and whether sellers begin making reductions. Over time, those movements can help build a more current picture of conditions in particular cities or regions.
The history of individual listings can be especially revealing. If properties that previously sold quickly begin sitting on the market for longer and going through several price reductions, the change may be worth investigating before it becomes obvious in broader market statistics. The opposite pattern can be useful too, particularly when inventory is shrinking and listings are disappearing quickly.
For a hedge fund, the relevance can extend beyond property companies themselves. Housing activity can affect businesses exposed to mortgages, construction, home improvement, furniture, and plenty of other areas of consumer spending. Alternative data becomes particularly useful when analysts can connect a change in one market with the companies likely to feel its effects.
The Best Signals Usually Become More Useful Together
By this point, you could be forgiven for thinking the answer is simply to collect everything on the internet. That would certainly keep the data engineering team busy, but it probably wouldn’t make the investment team much happier.
The challenge is deciding which sources have a meaningful connection to the company, sector, or thesis you’re researching. If you’re looking at a retailer, pricing and inventory might deserve close attention, but search visibility and customer feedback could provide useful supporting context. For a travel business, availability and pricing across booking platforms may tell you far more than monitoring thousands of unrelated web signals ever could.
Combining those sources can also help analysts avoid putting too much weight on one unusual movement. A jump in search interest might be interesting on its own, but it becomes more compelling when product availability is tightening and review activity is rising at the same time. Equally, contradictory signals can be useful because they give the research team a reason to question the story they were expecting to find.
The aim is to build a body of evidence that can be followed over time. Once analysts understand how those signals normally behave and how they relate to one another, changes become much easier to put into context.
Good Alternative Data Depends on Consistent Collection
Finding an interesting signal is only the beginning. If an investment team wants to compare what happened this week with what happened six months ago, the underlying data needs to have been collected in a reasonably consistent way throughout that period.
That can be harder than it sounds when the source is the public web. Websites change their layouts, products disappear, search results move around, and information that used to sit directly in the HTML may suddenly start loading through JavaScript. A retailer can redesign its site without warning, leaving a collection pipeline happily visiting pages while an important field has stopped coming through correctly.
Those gaps matter when analysts are looking for relatively small changes in a much larger dataset. If the number of products available from a retailer suddenly falls by 20 percent, that could be a useful market signal, or it could mean the website changed and your scraper missed part of the catalog. Without monitoring the collection process and understanding what a normal dataset looks like, the two can be surprisingly difficult to tell apart.
Historical analysis makes consistency even more important. Once a dataset has been collected for several years, you can’t simply go back and recreate everything if you discover that the methodology changed halfway through. Keeping track of how sources are collected, validating the output, and spotting problems early protects the history that makes alternative data valuable in the first place.
Geography Can Change the Picture
Public web data can also look different depending on where it’s collected from. That’s easy to overlook when you’re sitting in one office looking at one version of a website, but it becomes important when you’re researching businesses that operate across multiple markets.
Retail prices are an obvious example. The same company may run different promotions in different countries, while product availability and delivery options can vary much more locally. Travel platforms can return different results depending on the market they’re serving, and search engines regularly adjust what they show according to location.
For an investment team, those differences can be part of the signal. If you’re researching how a retailer is performing in the U.S., collecting the version of its website intended for customers somewhere else may give you a distorted picture of its pricing or assortment. Teams following several markets may want to compare those differences directly, particularly when they’re trying to understand whether a change is happening across the business or is concentrated in one region.
This is where proxy infrastructure starts to become particularly relevant to alternative data collection. It allows teams to collect public information from the locations they’re interested in, rather than assuming one view of a website represents what customers everywhere are seeing.
Scale Changes the Engineering Behind the Dataset
An analyst can learn plenty by manually checking a few websites. The engineering challenge begins when that useful experiment turns into something the fund wants updated every day across thousands or millions of pages.
Collection volumes tend to grow quickly once a dataset proves valuable. Someone wants another competitor included, the research team decides to follow the same signal in five more countries, or an analyst asks whether the data can be refreshed more frequently around earnings season. Before long, a scraper that started as a relatively small research project is feeding a dataset that people expect to be available whenever they need it.
At that point, reliability matters just as much as coverage. Collection jobs need to keep running when individual requests fail, websites change, or traffic volumes increase, and teams need enough visibility to notice when the resulting data starts behaving differently. If an analyst is making decisions from a historical series, unexplained gaps or changes in collection methodology can undermine months of work.
The web sources themselves won’t necessarily cooperate with your schedule either. Some are simple enough to collect through standard requests, while others rely heavily on JavaScript or behave differently depending on the browser environment. Building a large alternative data operation usually means being able to handle that variety rather than expecting every source to fit neatly into the same collection method.
Alternative Data Also Comes With Responsibility
The fact that information can be found online doesn’t mean investment teams should collect anything they happen to come across. Alternative data programs need clear boundaries around what they’re collecting, where it comes from, and how it’s going to be used.
For public web data, teams should pay particular attention to the difference between genuinely public business information and personal or otherwise sensitive information. A retailer publishing product prices is very different from collecting information about individual people, even if both can technically be encountered online. Legal requirements, website terms, licensing arrangements, and internal compliance policies can all affect what makes sense for a particular project.
That makes governance part of the data strategy from the beginning. Funds need to understand where datasets came from and how they were collected, particularly when data is being purchased from an external provider rather than gathered internally. A fascinating signal becomes considerably less useful if nobody can confidently explain its provenance when compliance starts asking questions.
For engineering teams, having those boundaries established early also makes life easier. They know which sources are appropriate to collect and what information should be excluded, rather than trying to make those decisions after years of historical data have already accumulated.
Building Alternative Data Pipelines With Rayobyte
At Rayobyte, we work with teams collecting public web data at the kind of scale where a promising alternative dataset can quickly become a serious infrastructure project. Once you’re following large numbers of products, search results, listings, or other public sources across multiple markets, keeping collection reliable becomes a significant part of maintaining the dataset.
Rayobyte’s residential, datacenter, ISP, and mobile proxy networks give teams options for different collection workloads and geographic requirements. If a research team needs to understand what a public website is showing in different markets, that geographic coverage can help them build datasets that better represent the locations they’re studying.
Some of those sources will also require more than a straightforward HTTP request. Modern ecommerce sites, marketplaces, travel platforms, and other dynamic websites often rely heavily on JavaScript, which is where browser infrastructure can become part of the collection stack. rayobrowse is designed for those workloads and gives engineering teams a browser layer that works with Playwright without requiring them to maintain all of the underlying browser infrastructure themselves.
The right setup depends on the signal you’re following and how frequently you need to collect it. What matters is having enough flexibility to keep useful datasets running as the research expands, rather than discovering that the infrastructure becomes the limiting factor just as the signal starts getting interesting.
Speak to a member of our team today to find out more about what we can do for your business.
Scale Alternative Data Collection
Build reliable datasets across millions of public web pages without making infrastructure the bottleneck.
