-
Gustaw Krista replied to the discussion How to scrape data from a website protected by Cloudflare? in the forum General Web Scraping a year ago
Reply to How to scrape data from a website protected by Cloudflare?
Honestly, scraping sites protected by Cloudflare is hard, and it’s often better to try and find an API to use instead.
-
Gustaw Krista started the discussion How do I scrape financial data from websites like Yahoo Finance? in the forum General Web Scraping a year ago
How do I scrape financial data from websites like Yahoo Finance?
Yahoo Finance has a free API you can use to pull data directly instead of scraping.
-
Gustaw Krista changed their photo a year ago
-
Gustaw Krista became a registered member a year ago
-
Aarthi Basheer replied to the discussion How can I scrape data from a site that uses AJAX to load content? in the forum General Web Scraping a year ago
Reply to How can I scrape data from a site that uses AJAX to load content?
Use Selenium or Playwright to render the page and trigger the AJAX requests dynamically.
-
Aarthi Basheer started the discussion What are the top 5 tools for scraping structured and unstructured data? in the forum General Web Scraping a year ago
What are the top 5 tools for scraping structured and unstructured data?
BeautifulSoup is great for scraping static web pages and handling unstructured HTML content.
-
Aarthi Basheer changed their photo a year ago
-
Aarthi Basheer became a registered member a year ago
-
Emmanuele Sagit replied to the discussion What’s the best tool for scraping JavaScript-heavy websites? in the forum General Web Scraping a year ago
Reply to What’s the best tool for scraping JavaScript-heavy websites?
If you don’t need a full browser, requests-html can render JavaScript within a Python environment.
-
Emmanuele Sagit replied to the discussion How to avoid IP bans while scraping heavily trafficked sites? in the forum General Web Scraping a year ago
Reply to How to avoid IP bans while scraping heavily trafficked sites?
Use residential proxies instead of data center proxies, as these are less likely to get flagged by sites.
- Load More
