Raja Lakeshia
-
Raja Lakeshia replied to the discussion How to scrape data from a website protected by Cloudflare? in the forum General Web Scraping a year ago
How to scrape data from a website protected by Cloudflare?
Rotating proxies and changing user agents can sometimes help you get past Cloudflare.
-
Raja Lakeshia replied to the discussion How do you handle pagination when scraping websites? in the forum General Web Scraping a year ago
How do you handle pagination when scraping websites?
For infinite scrolling, I use Selenium to simulate scrolling, waiting for new data to load each time. You can control the scroll rate and set timeouts to ensure all items are loaded.
-
Raja Lakeshia started the discussion How to scrape data from a website with rate limits and timeouts? in the forum General Web Scraping a year ago
How to scrape data from a website with rate limits and timeouts?
Add delays between requests to prevent reaching rate limits too quickly.
-
Raja Lakeshia changed their photo a year ago
-
Raja Lakeshia updated their profile a year ago
-
Raja Lakeshia became a registered member a year ago