-
Keith Marwin changed their photo a year ago
-
Keith Marwin became a registered member a year ago
-
Suhaila Kiyoshi replied to the discussion How do I deal with scraped data that has inconsistent formatting? in the forum General Web Scraping a year ago
Reply to How do I deal with scraped data that has inconsistent formatting?
Pandas is incredibly useful for normalizing scraped data by filling in missing values and aligning data types.
-
Suhaila Kiyoshi replied to the discussion How do I scrape data from sites using custom fonts or icons? in the forum General Web Scraping a year ago
Reply to How do I scrape data from sites using custom fonts or icons?
Font Awesome icons and similar libraries can sometimes be scraped as their CSS classes directly without image recognition.
-
Suhaila Kiyoshi replied to the discussion How do I handle scraping for real-time data that updates frequently? in the forum General Web Scraping a year ago
Reply to How do I handle scraping for real-time data that updates frequently?
For high-frequency scrapes, rotating IPs and adding randomized delays are critical to avoid detection. Real-time scrapers can get flagged quickly.
-
Suhaila Kiyoshi replied to the discussion How can I handle pagination when scraping JavaScript-heavy sites? in the forum General Web Scraping a year ago
Reply to How can I handle pagination when scraping JavaScript-heavy sites?
Adding timeouts between page loads helps reduce detection and gives the page time to load all content, preventing skipped data.
-
Suhaila Kiyoshi started the discussion What’s the best way to scrape product listings and prices from Shopee? in the forum General Web Scraping a year ago
What’s the best way to scrape product listings and prices from Shopee?
Shopee provides public APIs in some regions, allowing for reliable and structured access to product listings and prices.
-
Suhaila Kiyoshi changed their photo a year ago
-
Suhaila Kiyoshi became a registered member a year ago
-
Lana Sneferu replied to the discussion How can I scrape multi-step verification processes? in the forum General Web Scraping a year ago
Reply to How can I scrape multi-step verification processes?
Using headless browsers like Playwright or Puppeteer, I store session cookies to skip the multi-step process for subsequent requests.
- Load More
