Forum Replies Created

  • Using rotating proxies and randomized headers reduces the risk of being blocked by the server, ensuring smoother operation over long scraping sessions.

  • Puppeteer is ideal for scraping Shopee Thailand because it handles JavaScript-rendered pages seamlessly. Scraping reviews involves targeting the right HTML elements that contain user ratings and comments. The challenge is often dealing with multiple review pages or infinite scrolling, so you may need to automate scrolling and capture reviews as they load.

    const puppeteer = require('puppeteer');
    (async () => {
        const browser = await puppeteer.launch({ headless: true });
        const page = await browser.newPage();
        await page.goto('https://shopee.co.th/product-page-url');
        // Scroll to load all reviews
        await page.evaluate(() => {
            window.scrollTo(0, document.body.scrollHeight);
        });
        // Extract reviews
        const reviews = await page.$$eval('.shopee-review-item', reviews => {
            return reviews.map(review => ({
                user: review.querySelector('.review-username').innerText,
                rating: review.querySelector('.shopee-star-rating').innerText,
                text: review.querySelector('.shopee-review-item__content').innerText
            }));
        });
        console.log(reviews);
        await browser.close();
    })();