-
Kanchana Lalita became a registered member a year ago
-
Nora Rhys replied to the discussion How do you handle pagination when scraping websites? in the forum General Web Scraping a year ago
Reply to How do you handle pagination when scraping websites?
For sites using AJAX to load more content, monitor the network tab for API calls and replicate them.
-
Nora Rhys replied to the discussion Best techniques to rotate proxies when scraping websites. in the forum General Web Scraping a year ago
Reply to Best techniques to rotate proxies when scraping websites.
Combining rotating proxies with random delays between requests can minimize detection.
-
Nora Rhys started the discussion What are the benefits of using Selenium for web scraping vs. API scraping? in the forum General Web Scraping a year ago
What are the benefits of using Selenium for web scraping vs. API scraping?
Selenium is useful when there’s no API available, or the data is hidden behind JavaScript.
-
Nora Rhys changed their photo a year ago
-
Nora Rhys became a registered member a year ago
-
Sasha Voski replied to the discussion How to scrape websites that require login/authentication? in the forum General Web Scraping a year ago
Reply to How to scrape websites that require login/authentication?
If the login is API-based, try capturing the request and mimicking it in your code using libraries like requests.
-
Sasha Voski started the discussion How do you scrape dynamic tables that use JavaScript on a webpage? in the forum General Web Scraping a year ago
How do you scrape dynamic tables that use JavaScript on a webpage?
Use Selenium or Playwright to render the page and then scrape the content.
-
Sasha Voski changed their photo a year ago
-
Sasha Voski became a registered member a year ago
- Load More
