-
Manoj Fikreta replied to the discussion How can I handle data extraction from websites with region-specific restriction? in the forum General Web Scraping a year ago
Reply to How can I handle data extraction from websites with region-specific restriction?
Automating interactions through a headless browser with geolocation settings often bypasses restrictions, especially for location-specific data.
-
Manoj Fikreta started the discussion How to gather trending product data from Shopee and Lazada in Southeast Asia? in the forum General Web Scraping a year ago
How to gather trending product data from Shopee and Lazada in Southeast Asia?
I track top-selling categories on both Shopee and Lazada to identify which items are currently trending in Southeast Asia.
-
Manoj Fikreta changed their photo a year ago
-
Manoj Fikreta became a registered member a year ago
-
Herakles Urias replied to the discussion How can I scrape embedded data from audio or video content? in the forum General Web Scraping a year ago
Reply to How can I scrape embedded data from audio or video content?
I use the open-source tool pydub to split longer audio into smaller clips, which can then be transcribed more easily.
-
Herakles Urias replied to the discussion What techniques can I use to scrape real-time web chats or comment sections? in the forum General Web Scraping a year ago
Reply to What techniques can I use to scrape real-time web chats or comment sections?
I save timestamps for each chat message, which helps organize the data chronologically and maintain the original conversation flow.
-
Herakles Urias replied to the discussion How do I approach scraping data that appears only on hover or in tooltips? in the forum General Web Scraping a year ago
Reply to How do I approach scraping data that appears only on hover or in tooltips?
If hovering over items manually is impractical, I program the script to hover over each element sequentially and capture the data.
-
Herakles Urias replied to the discussion How can I scrape structured data from sites without standard HTML tags? in the forum General Web Scraping a year ago
Reply to How can I scrape structured data from sites without standard HTML tags?
Inspecting CSS classes and id attributes for unique identifiers helps locate data within unusual or proprietary tags.
-
Herakles Urias replied to the discussion How do I handle sites that block based on unusual request patterns? in the forum General Web Scraping a year ago
Reply to How do I handle sites that block based on unusual request patterns?
I use a mix of user-agents and device emulations to simulate requests from various browsers, which can help bypass pattern detection.
-
Herakles Urias replied to the discussion How can I handle data extraction from websites with region-specific restriction? in the forum General Web Scraping a year ago
Reply to How can I handle data extraction from websites with region-specific restriction?
Tor can provide IPs from various countries, but it’s slower and may not work on sites with strong anti-bot measures.
- Load More
