How to scrape freelancer profiles from Fiverr.com using JavaScript?

Heli Burhan · 2024-12-20T07:05:07+00:00

Scraping freelancer profiles from Fiverr.com using JavaScript can provide valuable insights into freelancer skills, pricing, and reviews. Using Node.js with Puppeteer, you can extract structured data by automating browser interactions to ensure all JavaScript-rendered content is fully loaded before scraping. This process involves navigating to Fiverr’s category pages, identifying elements containing freelancer details, and extracting relevant information such as names, gig descriptions, and pricing. Puppeteer is especially helpful for handling dynamic content and simulating user-like behavior. Below is an example script for scraping Fiverr freelancer profiles.const puppeteer require('puppeteer');(async () > { const browser await puppeteer.launch({ headless: true }); const page await browser.newPage(); const url 'https://www.fiverr.com/categories/graphics-design'; await page.goto(url, { waitUntil: 'networkidle2' }); const freelancers await page.evaluate(() > { const gigs ; const items document.querySelectorAll('.gig-card-layout'); items.forEach(item > { const name item.querySelector('.seller-name')?.textContent.trim() || 'Name not available'; const gigTitle item.querySelector('.gig-title')?.textContent.trim() || 'Gig title not available'; const price item.querySelector('.price')?.textContent.trim() || 'Price not available'; gigs.push({ name, gigTitle, price }); }); return gigs; }); console.log(freelancers); await browser.close();})();This script navigates to Fiverr’s Graphics & Design category, waits for the content to load, and extracts freelancer names, gig titles, and prices. Adding functionality for pagination allows scraping data from all pages within a category. Delays between requests and user-agent rotation help avoid detection by Fiverr’s anti-scraping mechanisms. Storing the extracted data in a structured format such as JSON or a database allows for efficient analysis and long-term use.

General Web Scraping

How to scrape freelancer profiles from Fiverr.com using JavaScript?

Posted by Heli Burhan on 12/20/2024 at 7:05 am
Scraping freelancer profiles from Fiverr.com using JavaScript can provide valuable insights into freelancer skills, pricing, and reviews. Using Node.js with Puppeteer, you can extract structured data by automating browser interactions to ensure all JavaScript-rendered content is fully loaded before scraping. This process involves navigating to Fiverr’s category pages, identifying elements containing freelancer details, and extracting relevant information such as names, gig descriptions, and pricing. Puppeteer is especially helpful for handling dynamic content and simulating user-like behavior. Below is an example script for scraping Fiverr freelancer profiles.
```
const puppeteer = require('puppeteer');
(async () => {
    const browser = await puppeteer.launch({ headless: true });
    const page = await browser.newPage();
    const url = 'https://www.fiverr.com/categories/graphics-design';
    await page.goto(url, { waitUntil: 'networkidle2' });
    const freelancers = await page.evaluate(() => {
        const gigs = [];
        const items = document.querySelectorAll('.gig-card-layout');
        items.forEach(item => {
            const name = item.querySelector('.seller-name')?.textContent.trim() || 'Name not available';
            const gigTitle = item.querySelector('.gig-title')?.textContent.trim() || 'Gig title not available';
            const price = item.querySelector('.price')?.textContent.trim() || 'Price not available';
            gigs.push({ name, gigTitle, price });
        });
        return gigs;
    });
    console.log(freelancers);
    await browser.close();
})();
```
This script navigates to Fiverr’s Graphics & Design category, waits for the content to load, and extracts freelancer names, gig titles, and prices. Adding functionality for pagination allows scraping data from all pages within a category. Delays between requests and user-agent rotation help avoid detection by Fiverr’s anti-scraping mechanisms. Storing the extracted data in a structured format such as JSON or a database allows for efficient analysis and long-term use.
Heli Burhan replied 2 days, 12 hours ago 1 Member · 0 Replies
0 Replies

Sorry, there were no replies found.