What Is Threat Hunting With Proxies

Companies are spending top dollar to take a proactive approach to cyber threats. But, as they say, prevention is better than cure, so many businesses are keen on threat hunting. But what is threat hunting exactly?

Many common cyber threats can be detected early on with the right tools and strategies, from ransomware to phishing. Web traffic can be one place to look for clues because external attacks often emerge through the web. This is where threat hunting with proxies comes in.

In this article, you’ll learn what proactive threat hunting is, what role proxies play, and how ethical users can use proxies to avoid getting flagged by these threat hunting tools.

What Is Cyber Threat Hunting?

Threat hunting is a proactive form of cyber defense that involves searching for and identifying abnormal behavior or weaknesses in the system. It helps reduce the time between intrusion and discovery of the threat, allowing security experts to nip danger in the bud before it causes any harm.

While most threats can’t get past security measures, some are more sophisticated and advanced. Attackers can hide in the system and files for weeks, gradually moving through the network and collecting more information. This process can take weeks or even months. Without proactive hunting, it can easily escape the sight of security tools and personnel.

A data breach, on average, costs a business $4 million. Unfortunately, many data breaches begin silently behind doors without any security software or expert knowledge.

There are many ways for security managers and experts to hunt threats before they get big. One of the most frequently used elements for threat hunting is the use of a web proxy.

Threat hunting with web proxy logs

Web proxy logs provide ample information for threat hunters. These proxies are channels between a device sending request and the device or server receiving it. Web proxies produce a standard set of information that can be used to detect abnormal or suspicious behavior.

Threat hunters use different protocols and techniques to analyze these logs and detect any bad actors in the mix. Typically, web proxy logs provide the following information:

• Destination URL (Hostname)
• Destination IP
• Destination Port
• User Agent
• Request Method
• Device Action
• Domain Category
• Requested File Name
• Duration
• HTTP Status
• Protocol
• Bytes In
• Bytes Out

The use of web proxies for threat hunting isn’t limited to ongoing attacks in their early stages. The purpose is also to enlighten, learn, and empower the security operations center (SOC).

For instance, a threat hunter in an organization conducts a web proxy log threat analysis and notices unusual requests from user agents like cURL interacting with SharePoint sites. They highlight the issue and learn that the requests are, in fact, authentic and coming from the DevOps teams.

However, this information allows them to enact a protocol that restricts cURL usage to specific hosts. That way, another cURL request cannot successfully go to another site within the organization.

How does it work?

Now that you understand threat hunting let’s learn how web proxy logs help these hunters. Since web proxy logs contain many data points, analysts have to use different techniques to identify vulnerabilities and malicious parties interacting with the network.

• Reviewing blocked traffic: While the user in the organization may not have been able to visit a particular website because it’s blocked, it’s worth checking what prompted them to do so. It could be a sign that their device has been compromised.
• URLs with IP requests: This filtration can identify logs that use hard coded IP addresses to circumvent DNS security controls.
• URLs with file extensions: This filtration exposes URLs with file extensions like .doc, .pdf., .exe. that may be malicious. Often, attackers use macro-enabled doc or pdf files to introduce malware into a system or network.
• Known referrer URL with uncommon URL: Filtering out logs with common referrer domains and unique URLs may help identify phishing links. Here are some of the techniques they utilize when threat hunting with web proxy logs:
  • Uncommon user agents: Unknown or rare user agents may be malicious. If logs show such user agents, especially those associated with well-known malware, hunters can immediately block access. This is a common practice to outsmart email security measures.
  • Threat intel lists: Any traffic to known Indicators of Compromise (IoCs) is a major red flag about device compromise. Any host within the network connecting with such URLs or IPs may be compromised.

Here’s an example of how threat hunters use web proxy logs:

The duration of requests from a proxy (IP address) can indicate if there’s malicious malware at the end. For instance, malware typically asks for commands periodically. Usually, these requests are not of the same duration and may use different durations to make requests more random. However, malware may also keep the connection open.

The threat hunter can sum the total request time over a 12-hour or 24-hour period to identify malicious users sending requests. This is also how a threat hunter may identify web scrapers.

Challenges with threat hunting with proxy logs

Threat hunting has its limitations, especially when using web proxy logs. So it’s not a foolproof way to detect threats early on. But addressing these challenges can improve efforts. Here are the main challenges:

• Web proxy logs are enormous, with tens of thousands to millions of log entries depending on the size of the company or business.
• Analyzing web proxy logs is a time and resource-consuming endeavor that many companies cannot afford.
• Given the sheer amount of logs to investigate, it may not be quick enough for fast-moving threats.
• The learned knowledge then has to be implemented on a broader level which takes additional time.

Proactive Threat Hunting and the Dilemma for Proxy Users

Threat hunting is a legitimate and feasible practice for companies to protect themselves from unforeseen attacks. But when using proxy logs, it can also flag users that may be scraping the websites. Such a scenario is problematic for users simply trying to do legitimate tasks.

Users can prevent threat hunters from detecting their activity by using multiple proxies, especially ones that help mask their actual IP address. In addition, since there’s no single IP address for all their activity, their logs don’t raise a flag for these hunters.

You’ll need quality proxies for this purpose that look legitimate to threat hunting software. If you’re wondering what threat hunting software is, it’s simply a tool that carries out threat hunting protocols and analysis.

Rayobyte proxies

Rayobyte’s residential proxies are the best solution to circumvent threat hunters for users simply trying to do their research and not cause any harm. These are proxies based in residential areas, with local internet service providers assigned IP addresses. In other words, these proxies look like actual users and not just user agents. This makes it hard for threat hunting software to see them as malicious.

More importantly, these residential proxies can be rotated, which means you can use the small set of proxies without sending too many requests in a specific time frame. That’s a surefire way to avoid getting trapped into the filters threat hunters use with proxy logs.

You can get proxies from virtually any region globally as per your business needs. That’s why Rayobyte also offers its premium Data Center and ISP proxies from around the world. Using proxies based in the website’s area can help immensely. A threat hunter may become suspicious of requests coming from a country they don’t operate in or don’t usually receive traffic from.

Wrap Up

So what is threat hunting? It’s a proactive way to detect threats to a company’s digital security. This can be achieved in many ways; analyzing web proxy logs is a common technique used by threat hunters and threat hunter software.

Rotating proxies provide a viable solution for those who don’t want to get mistaken as malicious parties. By using different proxies, users won’t appear suspicious to threat hunters, who would otherwise think of them as attackers.

Proxies are also advised for companies’ threat hunting because proxies can safeguard their own devices and users. However, if one device is compromised, it can expose the whole network and company to attacks. Reach out to Rayobyte’s team and get started today.

The information contained within this article, including information posted by official staff, guest-submitted material, message board postings, or other third-party material is presented solely for the purposes of education and furtherance of the knowledge of the reader. All trademarks used in this publication are hereby acknowledged as the property of their respective owners.