Khloe Walther
-
Khloe Walther replied to the discussion How do I identify hidden APIs that might be easier to scrape? in the forum General Web Scraping a year ago
Reply to How do I identify hidden APIs that might be easier to scrape?
Many sites use GraphQL APIs, so I look for POST requests with query bodies in the network tab.
-
Khloe Walther replied to the discussion What are the best methods for scraping data from dynamically-loaded websites? in the forum General Web Scraping a year ago
Reply to What are the best methods for scraping data from dynamically-loaded websites?
Scrapy Splash is another option for Python users. It can render JavaScript within Scrapy pipelines, allowing you to handle dynamic content without switching libraries.
-
Khloe Walther started the discussion What’s the best way to scrape e-commerce sites for product specifications? in the forum General Web Scraping a year ago
What’s the best way to scrape e-commerce sites for product specifications?
I set up scripts to navigate product categories first, as this reduces redundant scraping of main product listings.
-
Khloe Walther changed their photo a year ago
-
Khloe Walther became a registered member a year ago
