-
How can I scrape websites with infinite scroll without losing data?
I usually rely on Puppeteer for this. You can set it to scroll a specific number of pixels or until a specific element is visible. Combine this with a wait time to make sure data is fully loaded each time.
Log in to reply.