-
Nora Rhys replied to the discussion Best techniques to rotate proxies when scraping websites. in the forum General Web Scraping a year ago
Reply to Best techniques to rotate proxies when scraping websites.
Combining rotating proxies with random delays between requests can minimize detection.
-
Nora Rhys started the discussion What are the benefits of using Selenium for web scraping vs. API scraping? in the forum General Web Scraping a year ago
What are the benefits of using Selenium for web scraping vs. API scraping?
Selenium is useful when there’s no API available, or the data is hidden behind JavaScript.
-
Nora Rhys changed their photo a year ago
-
Nora Rhys became a registered member a year ago
-
Sasha Voski replied to the discussion How to scrape websites that require login/authentication? in the forum General Web Scraping a year ago
Reply to How to scrape websites that require login/authentication?
If the login is API-based, try capturing the request and mimicking it in your code using libraries like requests.
-
Sasha Voski started the discussion How do you scrape dynamic tables that use JavaScript on a webpage? in the forum General Web Scraping a year ago
How do you scrape dynamic tables that use JavaScript on a webpage?
Use Selenium or Playwright to render the page and then scrape the content.
-
Sasha Voski changed their photo a year ago
-
Sasha Voski became a registered member a year ago
-
Margarid Ruzanna replied to the discussion How to avoid IP bans while scraping heavily trafficked sites? in the forum General Web Scraping a year ago
Reply to How to avoid IP bans while scraping heavily trafficked sites?
Implement rate-limiting in your scraper to mimic real user activity.
-
Margarid Ruzanna started the discussion Is web scraping dead with the rise of anti-bot technologies? in the forum General Web Scraping a year ago
Is web scraping dead with the rise of anti-bot technologies?
Web scraping is not dead, but it’s becoming more challenging with modern anti-bot tools like CAPTCHAs.
- Load More
