-
Hadrianus Kazim replied to the discussion How to scrape job postings using Google Jobs API with Node.js? in the forum General Web Scraping a year ago
Reply to How to scrape job postings using Google Jobs API with Node.js?
For secure authentication, I store API keys and access tokens in environment variables. This prevents hardcoding sensitive credentials in the source code.
-
Hadrianus Kazim started the discussion Use Python to scrape product availability from Ruten Taiwan in the forum General Web Scraping a year ago
Use Python to scrape product availability from Ruten Taiwan
How would you scrape product availability from Ruten, one of Taiwan’s largest online marketplaces? Is the availability clearly displayed on the product page, or is it part of a dynamic element that requires JavaScript to load? Would using Python with BeautifulSoup and requests be enough, or would additional tools like Selenium be necessary if…
-
Hadrianus Kazim changed their photo a year ago
-
Hadrianus Kazim became a registered member a year ago
-
Zaheer Arethusa replied to the discussion How can I scrape product reviews from Shopee Thailand using Node.js n Puppeteer? in the forum General Web Scraping a year ago
Reply to How can I scrape product reviews from Shopee Thailand using Node.js n Puppeteer?
Scraping reviews from Shopee Thailand with Puppeteer involves interacting with the product page, loading additional reviews if necessary, and then parsing the content using custom selectors. It’s important to handle the asynchronous nature of review loading—using Puppeteer’s waiting functions ensures you only scrape reviews after they’ve been…
-
Zaheer Arethusa replied to the discussion How can I scrape product data from Lazada Thailand using Python n BeautifulSoup? in the forum General Web Scraping a year ago
Reply to How can I scrape product data from Lazada Thailand using Python n BeautifulSoup?
When scraping Lazada Thailand, make sure you’re handling the request headers properly. The site may block requests that don’t appear to come from an actual browser, so it’s essential to mimic a real browser using headers. In addition, the structure of the HTML might change across different product categories, so using flexible selectors is a…
-
Zaheer Arethusa started the discussion Use Go to scrape product prices from PChome Taiwan in the forum General Web Scraping a year ago
Use Go to scrape product prices from PChome Taiwan
How would you scrape product prices from PChome, one of Taiwan’s leading e-commerce websites? Are the prices located in a specific HTML element that is consistent across all products? Or does the page structure vary depending on the product category? What about dynamically loaded content—are the prices rendered directly in the HTML or fetched…
-
Zaheer Arethusa changed their photo a year ago
-
Zaheer Arethusa became a registered member a year ago
-
Afnan Ayumi replied to the discussion How to fetch property data using Redfin API with Python? in the forum General Web Scraping a year ago
Reply to How to fetch property data using Redfin API with Python?
To handle rate limits, I use the time.sleep() function to introduce delays between requests. This prevents overwhelming the server and reduces the risk of getting blocked.
- Load More
