-
Iraida Anicetus changed their photo a year ago
-
Iraida Anicetus became a registered member a year ago
-
Jaana Lorn replied to the discussion What are the best practices for scraping financial data from news or stock site? in the forum General Web Scraping a year ago
Reply to What are the best practices for scraping financial data from news or stock site?
For news data, I prioritize only the key fields, like headlines and timestamps, to keep requests lightweight and avoid bans.
-
Jaana Lorn replied to the discussion How do I handle sites that block based on unusual request patterns? in the forum General Web Scraping a year ago
Reply to How do I handle sites that block based on unusual request patterns?
Setting the script to take breaks at random intervals mimics real user behavior and helps avoid blocks on monitored sites.
-
Jaana Lorn started the discussion How to track e-commerce growth in Asia using Lazada, Shopee, and Tokopedia? in the forum General Web Scraping a year ago
How to track e-commerce growth in Asia using Lazada, Shopee, and Tokopedia?
By monitoring new listings and category growth on each platform, I can see which areas, like electronics or home goods, are expanding most quickly.
-
Jaana Lorn changed their photo a year ago
-
Jaana Lorn became a registered member a year ago
-
Claudius Rebeka replied to the discussion What techniques can I use to scrape real-time web chats or comment sections? in the forum General Web Scraping a year ago
Reply to What techniques can I use to scrape real-time web chats or comment sections?
Monitoring XHR requests while interacting with the chat often reveals direct API calls that can be scraped more efficiently.
-
Claudius Rebeka replied to the discussion How do I handle sites that block based on unusual request patterns? in the forum General Web Scraping a year ago
Reply to How do I handle sites that block based on unusual request patterns?
Introducing human-like behaviors, like small pauses between scrolling and clicking, makes the bot behavior less predictable.
rayobyte.com
How do I handle sites that block based on unusual request patterns? - Rayobyte Community
Adding randomized delays between requests and avoiding repetitive patterns makes the scraper seem more like a human user.
-
Claudius Rebeka replied to the discussion How do I extract text from images or infographics? in the forum General Web Scraping a year ago
Reply to How do I extract text from images or infographics?
Some sites offer high-resolution downloads, so I scrape these versions to improve the OCR output compared to low-quality images.
- Load More
