-
Baltassar Igor replied to the discussion How can I scrape JavaScript-based content without headless browsers? in the forum General Web Scraping a year ago
Reply to How can I scrape JavaScript-based content without headless browsers?
Some sites allow access to backend APIs without requiring the front-end JavaScript interactions. Finding these endpoints often eliminates the need for rendering.
-
Baltassar Igor replied to the discussion What’s the best way to scrape map-based data from websites? in the forum General Web Scraping a year ago
Reply to What’s the best way to scrape map-based data from websites?
API services, like Google Maps API, are the easiest and most accurate method, though they may require payment for high usage.
-
Baltassar Igor replied to the discussion How can I detect and manage duplicate data in my scraped results? in the forum General Web Scraping a year ago
Reply to How can I detect and manage duplicate data in my scraped results?
Storing unique identifiers in a database helps prevent duplication by checking for existing entries before inserting new data.
-
Baltassar Igor replied to the discussion How do I scrape data from sites using custom fonts or icons? in the forum General Web Scraping a year ago
Reply to How do I scrape data from sites using custom fonts or icons?
Custom-built dictionaries for each icon or font help translate these into readable text, especially for sites using unique icons.
-
Baltassar Igor started the discussion How can I use Java to scrape product information on Carrefour’s European site? in the forum General Web Scraping a year ago
How can I use Java to scrape product information on Carrefour’s European site?
JSoup, a Java library, can extract static HTML content from Carrefour’s site, including product names, descriptions, and prices.
-
Baltassar Igor changed their photo a year ago
-
Baltassar Igor became a registered member a year ago
-
Rhouth Vilma replied to the discussion How do I deal with scraped data that has inconsistent formatting? in the forum General Web Scraping a year ago
Reply to How do I deal with scraped data that has inconsistent formatting?
If I expect certain formats, like currency or dates, I parse those fields specifically to convert them into standardized formats.
-
Rhouth Vilma replied to the discussion How do I scrape data from sites using custom fonts or icons? in the forum General Web Scraping a year ago
Reply to How do I scrape data from sites using custom fonts or icons?
Extracting the text using Playwright and checking for any special CSS or JavaScript manipulations reveals additional data.
-
Rhouth Vilma replied to the discussion What strategies can I use to scrape websites with limited search functionality? in the forum General Web Scraping a year ago
Reply to What strategies can I use to scrape websites with limited search functionality?
I also check the site’s sitemap, as it often contains URLs that don’t show up in the internal search but are publicly accessible.
- Load More
