-
Aeolus Afrodita started the discussion How can I scrape data from infinite scrolling pages (e.g., social media)? in the forum General Web Scraping a year ago
How can I scrape data from infinite scrolling pages (e.g., social media)?
Use Selenium to simulate scrolling down and loading more content dynamically.
-
Aeolus Afrodita changed their photo a year ago
-
Aeolus Afrodita became a registered member a year ago
-
Zeus Praxidike replied to the discussion What’s the best Python library for web scraping in 2024? in the forum General Web Scraping a year ago
Reply to What’s the best Python library for web scraping in 2024?
Scrapy is better for large-scale scraping projects because it has built-in crawling capabilities.
-
Zeus Praxidike started the discussion How to scrape websites that require login/authentication? in the forum General Web Scraping a year ago
How to scrape websites that require login/authentication?
Use Selenium or Playwright to automate logging in before scraping.
-
Zeus Praxidike changed their photo a year ago
-
Zeus Praxidike became a registered member a year ago
-
Aggie Suki started the discussion How do I extract data from a PDF using web scraping tools? in the forum General Web Scraping a year ago
How do I extract data from a PDF using web scraping tools?
Use PyPDF2 or pdfplumber in Python to extract text from PDFs.
-
Aggie Suki replied to the discussion How to bypass CAPTCHA when scraping websites? in the forum General Web Scraping a year ago
Reply to How to bypass CAPTCHA when scraping websites?
Another way is to use Selenium to render the page and solve the CAPTCHA manually when required.
-
Aggie Suki changed their photo a year ago
- Load More
