-
Diindiisi Aygerim replied to the discussion How do I scrape real-time data from stock market websites? in the forum General Web Scraping a year ago
Reply to How do I scrape real-time data from stock market websites?
If you must scrape, use Selenium or Playwright to grab dynamic stock data, but beware of rate limits and legal restrictions.
-
Diindiisi Aygerim replied to the discussion Best practices for web scraping to avoid getting blocked by websites. in the forum General Web Scraping a year ago
Reply to Best practices for web scraping to avoid getting blocked by websites.
Respect the robots.txt file and don’t scrape more than what’s allowed.
-
Diindiisi Aygerim started the discussion How can I automate data scraping from e-commerce sites like Amazon? in the forum General Web Scraping a year ago
How can I automate data scraping from e-commerce sites like Amazon?
Use Scrapy with a rotating proxy service to scrape product details without getting blocked.
-
Diindiisi Aygerim changed their photo a year ago
-
Diindiisi Aygerim updated their profile a year ago
-
Diindiisi Aygerim became a registered member a year ago
-
Sulaiman Filipina replied to the discussion How to scrape data from a website protected by Cloudflare? in the forum General Web Scraping a year ago
Reply to How to scrape data from a website protected by Cloudflare?
If the website’s protection is advanced, you may need to use Playwright or Puppeteer with a headless browser to mimic real user behavior.
-
Sulaiman Filipina replied to the discussion How to bypass CAPTCHA when scraping websites? in the forum General Web Scraping a year ago
Reply to How to bypass CAPTCHA when scraping websites?
In some cases, using a headless browser like Playwright with human-like behavior can reduce CAPTCHA triggers.
-
Sulaiman Filipina started the discussion How does Go handle concurrency compared to Python? in the forum General Web Scraping a year ago
How does Go handle concurrency compared to Python?
Go has goroutines, which are extremely lightweight compared to Python’s threading and multiprocessing.
-
Sulaiman Filipina changed their photo a year ago
- Load More
