-
Prashant Sanjiv became a registered member a year ago
-
Audrey Tareq replied to the discussion How to scrape browser profiles from XBrowser using Python and Selenium? in the forum General Web Scraping a year ago
Reply to How to scrape browser profiles from XBrowser using Python and Selenium?
To bypass detection, I use undetected-chromedriver, which prevents Selenium from being flagged by anti-bot mechanisms. This is crucial for scraping browser profiles.
-
Audrey Tareq replied to the discussion How to convert cURL commands to Python requests? in the forum General Web Scraping a year ago
Reply to How to convert cURL commands to Python requests?
For APIs requiring session management, I use requests.Session() to maintain cookies and headers across multiple requests efficiently.
-
Audrey Tareq started the discussion Scrape product name, price, rating from Mercado Livre Brazil using Puppeteer? in the forum General Web Scraping a year ago
Scrape product name, price, rating from Mercado Livre Brazil using Puppeteer?
To scrape the product name from Mercado Livre Brazil, you need to navigate to the product page and locate the element containing the title, typically found inside an h1 or span tag with a specific class. Puppeteer lets you extract this information after waiting for the page to load. You can use page.$eval() to grab the name once the content…
-
Audrey Tareq changed their photo a year ago
-
Audrey Tareq became a registered member a year ago
-
Uduak Pompeia replied to the discussion How to convert cURL commands to Python requests? in the forum General Web Scraping a year ago
Reply to How to convert cURL commands to Python requests?
When debugging complex requests, I log the entire request and response, including headers and payloads, to identify discrepancies between cURL and Python behavior.
-
Uduak Pompeia replied to the discussion Extracting flight schedules and routes with Python and aiohttp in the forum General Web Scraping a year ago
Reply to Extracting flight schedules and routes with Python and aiohttp
Storing flight data in a database helps track trends, such as price fluctuations or route availability, and makes it easier to query the data later.
-
Uduak Pompeia started the discussion Scrape product name, price, and availability Robinson Thailand using Puppeteer? in the forum General Web Scraping a year ago
Scrape product name, price, and availability Robinson Thailand using Puppeteer?
To scrape the product name from Robinson Thailand, you need to open the product page and wait for the relevant element to load. The product name is usually located in an h1 or div tag with a specific class. Use page.$eval() to extract the name text once the page has finished loading.
const puppeteer = require('puppeteer'); (async () => {
…
-
Uduak Pompeia changed their photo a year ago
- Load More
