News Feed Forums General Web Scraping How do you scrape product ratings from an e-commerce website?

  • How do you scrape product ratings from an e-commerce website?

    Posted by Senka Leontios on 12/17/2024 at 10:34 am

    Scraping product ratings is a common task in e-commerce data collection. How do you do it effectively? The first step is to identify where the ratings are located in the HTML. They’re often found near the product title or price and may be represented by stars, text, or numbers. Using Python’s requests and BeautifulSoup libraries, you can fetch and parse this data. However, if the ratings are rendered dynamically using JavaScript, tools like Selenium or Puppeteer are required.
    Here’s an example of scraping ratings with BeautifulSoup:

    import requests
    from bs4 import BeautifulSoup
    url = "https://example.com/products"
    headers = {"User-Agent": "Mozilla/5.0"}
    response = requests.get(url, headers=headers)
    if response.status_code == 200:
        soup = BeautifulSoup(response.content, "html.parser")
        products = soup.find_all("div", class_="product-item")
        for product in products:
            title = product.find("h2", class_="product-title").text.strip()
            rating = product.find("span", class_="product-rating").text.strip()
            print(f"Product: {title}, Rating: {rating}")
    else:
        print("Failed to fetch the page.")
    

    For JavaScript-rendered ratings, Puppeteer provides better results. It loads the entire page, including dynamic elements, ensuring nothing is missed. Do you use these tools for scraping ratings, or do you prefer APIs when available?

    Senka Leontios replied 5 days, 14 hours ago 1 Member · 0 Replies
  • 0 Replies

Sorry, there were no replies found.

Log in to reply.