-
Bronislawa Mirela replied to the discussion What’s the best approach to handling large datasets while scraping? in the forum General Web Scraping a year ago
Reply to What’s the best approach to handling large datasets while scraping?
Real-time monitoring of data quality helps me catch errors or missing entries early, especially when scraping high-volume sites.
-
Bronislawa Mirela started the discussion What’s the best way to scrape restaurant reviews and ratings on GrabFood? in the forum General Web Scraping a year ago
What’s the best way to scrape restaurant reviews and ratings on GrabFood?
Grab’s app has a structured review section, which I scrape with tools like Playwright to gather reviews and ratings on specific restaurants.
-
Bronislawa Mirela changed their photo a year ago
-
Bronislawa Mirela became a registered member a year ago
-
Joline Abdastartus replied to the discussion How do I handle scraping for real-time data that updates frequently? in the forum General Web Scraping a year ago
Reply to How do I handle scraping for real-time data that updates frequently?
I cache recent data locally and compare it with new scrapes to identify meaningful changes instead of saving duplicate data.
-
Joline Abdastartus replied to the discussion What strategies can I use to scrape websites with limited search functionality? in the forum General Web Scraping a year ago
Reply to What strategies can I use to scrape websites with limited search functionality?
Monitoring the site’s network traffic can reveal API calls with hidden parameters. Often, tweaking these parameters yields more search results.
-
Joline Abdastartus replied to the discussion How can I handle pagination when scraping JavaScript-heavy sites? in the forum General Web Scraping a year ago
Reply to How can I handle pagination when scraping JavaScript-heavy sites?
Logging each URL as I progress through pages ensures I don’t revisit pages accidentally. This is crucial for scraping sites with complex pagination structures.
-
Joline Abdastartus replied to the discussion What’s the best approach to handling large datasets while scraping? in the forum General Web Scraping a year ago
Reply to What’s the best approach to handling large datasets while scraping?
A distributed scraping setup, where multiple scrapers work on different parts of the site simultaneously, helps speed up data collection without overloading any one scraper.
-
Joline Abdastartus started the discussion How can I scrape promotional offers and discounts on Grab? in the forum General Web Scraping a year ago
How can I scrape promotional offers and discounts on Grab?
Grab’s app and website often feature promotional banners with special codes. Using a headless browser like Puppeteer, I capture these banners and associated offers.
-
Joline Abdastartus changed their photo a year ago
- Load More
