Raja Lakeshia
-
Raja Lakeshia replied to the discussion How to scrape data from a website protected by Cloudflare? in the forum General Web Scraping 11 months ago
How to scrape data from a website protected by Cloudflare?
Rotating proxies and changing user agents can sometimes help you get past Cloudflare.
-
Raja Lakeshia replied to the discussion How do you handle pagination when scraping websites? in the forum General Web Scraping 11 months ago
How do you handle pagination when scraping websites?
For infinite scrolling, I use Selenium to simulate scrolling, waiting for new data to load each time. You can control the scroll rate and set timeouts to ensure all items are loaded.
-
Raja Lakeshia started the discussion How to scrape data from a website with rate limits and timeouts? in the forum General Web Scraping 11 months ago
How to scrape data from a website with rate limits and timeouts?
Add delays between requests to prevent reaching rate limits too quickly.
-
Raja Lakeshia changed their photo 11 months ago
-
Raja Lakeshia updated their profile 11 months ago
-
Raja Lakeshia became a registered member 11 months ago