-
Gustaw Krista changed their photo a year ago
-
Gustaw Krista became a registered member a year ago
-
Aarthi Basheer replied to the discussion Is web scraping dead with the rise of anti-bot technologies? in the forum General Web Scraping a year ago
Reply to Is web scraping dead with the rise of anti-bot technologies?
The rise of APIs has made it easier to get data legally without scraping.
-
Aarthi Basheer replied to the discussion How can I scrape data from a site that uses AJAX to load content? in the forum General Web Scraping a year ago
Reply to How can I scrape data from a site that uses AJAX to load content?
Use Selenium or Playwright to render the page and trigger the AJAX requests dynamically.
-
Aarthi Basheer started the discussion What are the top 5 tools for scraping structured and unstructured data? in the forum General Web Scraping a year ago
What are the top 5 tools for scraping structured and unstructured data?
BeautifulSoup is great for scraping static web pages and handling unstructured HTML content.
-
Aarthi Basheer changed their photo a year ago
-
Aarthi Basheer became a registered member a year ago
-
Emmanuele Sagit replied to the discussion What’s the best tool for scraping JavaScript-heavy websites? in the forum General Web Scraping a year ago
Reply to What’s the best tool for scraping JavaScript-heavy websites?
If you don’t need a full browser, requests-html can render JavaScript within a Python environment.
-
Emmanuele Sagit replied to the discussion How to avoid IP bans while scraping heavily trafficked sites? in the forum General Web Scraping a year ago
Reply to How to avoid IP bans while scraping heavily trafficked sites?
Use residential proxies instead of data center proxies, as these are less likely to get flagged by sites.
-
Emmanuele Sagit started the discussion How do I handle large-scale web scraping projects efficiently? in the forum General Web Scraping a year ago
How do I handle large-scale web scraping projects efficiently?
Use Scrapy or Golang for larger projects that require more efficiency and scalability.
- Load More
