{"id":3250,"date":"2025-12-31T12:25:54","date_gmt":"2025-12-31T12:25:54","guid":{"rendered":"https:\/\/rayobyte.com\/community\/?post_type=scraping_project&#038;p=3250"},"modified":"2025-12-31T12:25:54","modified_gmt":"2025-12-31T12:25:54","slug":"build-a-google-shopping-scraper-in-python-for-price-and-product-data","status":"publish","type":"scraping_project","link":"https:\/\/rayobyte.com\/community\/scraping-project\/build-a-google-shopping-scraper-in-python-for-price-and-product-data\/","title":{"rendered":"Build a Google Shopping Scraper in Python for Price and Product Data"},"content":{"rendered":"<h1>Table of contents<\/h1>\n<ul>\n<li><a href=\"#Google Shopping Scraper\">Google Shopping Scraper<\/a><\/li>\n<li><a href=\"#Why Build a Google Shopping Scraper?\">Why Build a Google Shopping Scraper?<\/a><\/li>\n<li><a href=\"#Tools You'll Need\">Tools You&#8217;ll Need<\/a><\/li>\n<li><a href=\"#Prerequisites\">Prerequisites<\/a><\/li>\n<li><a href=\"#Writing the Scraper\">Writing the Scraper<\/a><\/li>\n<li><a href=\"#Code Explanation\"><strong>Code Explanation<\/strong><\/a><\/li>\n<li data-pm-slice=\"1 1 []\"><a href=\"#Important Notes\">Important Notes<\/a><\/li>\n<li><a href=\"#Conclusion\">Conclusion<\/a><\/li>\n<\/ul>\n<h1 id=\"Google Shopping Scraper\">Google Shopping Scraper<\/h1>\n<p>Google Shopping provides a wealth of product and price data, making it a valuable resource for e-commerce analysis. In this tutorial, we&#8217;ll guide you through building a Google Shopping scraper using Python. You&#8217;ll learn how to extract product details, prices, and seller information, enabling you to monitor market trends and competitor pricing.<\/p>\n<h1 id=\"Why Build a Google Shopping Scraper?\">Why Build a Google Shopping Scraper?<\/h1>\n<p>Accessing structured product and price data can be incredibly useful for:<\/p>\n<ul data-spread=\"false\">\n<li><strong>Competitor Analysis<\/strong>: Monitor pricing trends and offers from various sellers.<\/li>\n<li><strong>Market Research<\/strong>: Understand market dynamics by analyzing product availability and seller competition.<\/li>\n<li><strong>Inventory Management<\/strong>: Keep track of products listed by different vendors.<\/li>\n<\/ul>\n<h1 id=\"Tools You'll Need\">Tools You&#8217;ll Need<\/h1>\n<p>We&#8217;ll use the following tools to build our scraper:<\/p>\n<ul data-spread=\"false\">\n<li><strong>Python<\/strong>: The programming language for scripting.<\/li>\n<li><strong>Selenium<\/strong>: A powerful library to interact with web pages.<\/li>\n<li><strong>selenium-stealth<\/strong>: To help evade detection mechanisms by websites.<\/li>\n<li><strong>ChromeDriver<\/strong>: To control the Chrome browser for web scraping.<\/li>\n<li><strong>CSV<\/strong>: To save the scraped data for analysis.<\/li>\n<\/ul>\n<h1 id=\"Prerequisites\">Prerequisites<\/h1>\n<p>Make sure you have Python installed on your system. You can install the required libraries using pip:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\">pip install selenium selenium-stealth<\/pre>\n<p data-pm-slice=\"1 1 []\">Download and install ChromeDriver compatible with your Chrome browser version.<\/p>\n<h1 id=\"'Writing\">Writing the Scraper<\/h1>\n<p>Here is the full Python code for scraping Google Shopping:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\">from selenium import webdriver\nfrom selenium_stealth import stealth\nfrom selenium.webdriver.common.by import By\nfrom selenium.webdriver.chrome.options import Options\nfrom selenium.webdriver.support.wait import WebDriverWait\nfrom selenium.webdriver.support import expected_conditions as EC\nimport time\nimport traceback\nimport csv\n\n# Set up Chrome options\noptions = webdriver.ChromeOptions()\noptions.add_argument(\"start-maximized\")\n# Uncomment the line below to run in headless mode\n# options.add_argument(\"--headless\")\n\noptions.add_experimental_option(\"excludeSwitches\", [\"enable-automation\"])\noptions.add_experimental_option('useAutomationExtension', False)\ndriver = webdriver.Chrome(options=options)\n\n# Enable stealth mode to avoid detection\nstealth(driver,\n        languages=[\"en-US\", \"en\"],\n        vendor=\"Google Inc.\",\n        platform=\"Win32\",\n        webgl_vendor=\"Intel Inc.\",\n        renderer=\"Intel Iris OpenGL Engine\",\n        fix_hairline=True,\n        )\n\n# Set up CSV file\ncsv_file = \"products_data.csv\"\ncsv_columns = [\"Product Title\", \"Num of Vendor\", \"Vendor Name\", \"Price\", \"Image URL\", \"Vendor Link\", \"Product Rating\"]\n\n# Write the header to the CSV file\nwith open(csv_file, mode=\"w\", newline=\"\", encoding=\"utf-8\") as file:\n    writer = csv.DictWriter(file, fieldnames=csv_columns)\n    writer.writeheader()\n\n# Open Google Shopping search page\ndriver.get(\"https:\/\/www.google.com\/search?sca_esv=272b628add5e1644&amp;sxsrf=ADLYWIKOv6zMptOSkrduUIogA-_1QFM4Vw:1736252785259&amp;q=winter+coats&amp;udm=28&amp;fbs=AEQNm0Aa4sjWe7Rqy32pFwRj0UkWxyMMuf0D-HOMEpzq2zertRy7G-dme1ONMLTCBvZzSliUAtuJwXTPBVWQOpeBsM3fUSkUhBhk1d2K-btGU92EzjqhYy18vZp6Kq7rwx5djl86CZ3SmcnTCNSLLpUAYm1ku78BstqWpCyrkw60NVNGecT-nd3TIWJpCihPwU0PqL2HsfQ2bCEzBN5b0AAdvbM5G21Byw&amp;ved=1t:220175&amp;ictx=111&amp;biw=1541&amp;bih=945&amp;dpr=1#ip=1\")\n\n# Scraping data\nall_box = driver.find_elements(By.CSS_SELECTOR, \".SsM98d\")\n\nfor box in all_box:\n    try:\n        box.click()\n        # Wait for the product title to load\n        element = WebDriverWait(driver, 10).until(\n            EC.presence_of_element_located(\n                (By.CSS_SELECTOR, \"div.bi9tFe.PZPZlf[jsname='ZOnBqe'][data-attrid='product_title']\")\n            )\n        )\n\n        time.sleep(3)  # Allow additional time for content to load\n        product_title = driver.find_element(By.CSS_SELECTOR, 'div.bi9tFe').text\n        image = driver.find_element(By.CSS_SELECTOR, '.q5hmpb img.KfAt4d').get_attribute('src')\n        vendors = driver.find_elements(By.CSS_SELECTOR, 'a.P9159d')\n\n        for vendor in vendors:\n            vendor_name = vendor.find_element(By.CSS_SELECTOR, '.uWvFpd').text\n            try:\n                price = vendor.find_element(By.CSS_SELECTOR, '.GBgquf span span').text\n            except:\n                price = vendor.find_element(By.CSS_SELECTOR, 'span.Pgbknd.xUrPFc').text\n\n            try:\n                href = vendor.get_attribute('href')\n            except Exception as e:\n                href = None\n\n            try:\n                rating = vendor.find_element(By.CSS_SELECTOR, 'span.NFq8Ad').text\n            except Exception as e:\n                rating = \"\"\n\n            # Save data to CSV\n            with open(csv_file, mode=\"a\", newline=\"\", encoding=\"utf-8\") as file:\n                writer = csv.DictWriter(file, fieldnames=csv_columns)\n                writer.writerow({\n                    \"Product Title\": product_title,\n                    \"Num of Vendor\": len(vendors),\n                    \"Vendor Name\": vendor_name,\n                    \"Price\": price,\n                    \"Image URL\": image,\n                    \"Vendor Link\": href,\n                    \"Product Rating\": rating,\n                })\n\n            print(f\"nProduct Title: {product_title}nNum of Vendor: {len(vendors)}nVendor Name: {vendor_name}nPrice: {price}nImage URL: {image}nVendor Link: {href}nRating: {rating}n\")\n\n    except Exception as e:\n        print(\"An error occurred:\")\n        traceback.print_exc()\n\n# Close the browser\ndriver.quit()<\/pre>\n<h1 id=\"Code Explanation\"><strong>Code Explanation:<\/strong><\/h1>\n<ul>\n<li><strong>Chrome Options:<\/strong>\n<ul>\n<li>Configures the browser to run maximized and disables automation flags to avoid detection.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Stealth Mode:<\/strong>\n<ul>\n<li>Makes the browser appear human-like to bypass anti-bot measures.<\/li>\n<\/ul>\n<\/li>\n<li><strong>CSV Setup:<\/strong>\n<ul>\n<li>Prepares the file to store scraped data with headers like &#8220;Product Title,&#8221; &#8220;Price,&#8221; etc.<\/li>\n<li><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-3253 size-full\" src=\"https:\/\/rayobyte.com\/community\/wp-content\/uploads\/2025\/01\/Screenshot-2025-01-07-215833.png\" alt=\"\" width=\"1207\" height=\"846\" title=\"\" srcset=\"https:\/\/rayobyte.com\/community\/wp-content\/uploads\/2025\/01\/Screenshot-2025-01-07-215833.png 1207w, https:\/\/rayobyte.com\/community\/wp-content\/uploads\/2025\/01\/Screenshot-2025-01-07-215833-300x210.png 300w, https:\/\/rayobyte.com\/community\/wp-content\/uploads\/2025\/01\/Screenshot-2025-01-07-215833-1024x718.png 1024w, https:\/\/rayobyte.com\/community\/wp-content\/uploads\/2025\/01\/Screenshot-2025-01-07-215833-768x538.png 768w, https:\/\/rayobyte.com\/community\/wp-content\/uploads\/2025\/01\/Screenshot-2025-01-07-215833-624x437.png 624w\" sizes=\"auto, (max-width: 1207px) 100vw, 1207px\" \/><\/li>\n<\/ul>\n<\/li>\n<li><strong>Google Shopping URL:<\/strong>\n<ul>\n<li>Opens Google search with a specific query (<code>winter+coats<\/code>).<\/li>\n<li><strong>Customizable<\/strong>: Replace the query with your desired product search (e.g., <code>summer+shoes<\/code>).<\/li>\n<\/ul>\n<\/li>\n<li><strong>Product Scraping:<\/strong>\n<ul>\n<li>Locates product elements, clicks to expand details, and extracts:\n<ul>\n<li><strong>Product Title, Vendor Name, Price, Image URL, Vendor Link, and Rating.<\/strong><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li><strong>Error Handling:<\/strong>\n<ul>\n<li>Logs errors to avoid crashing when data is missing or inaccessible.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Save to CSV:<\/strong>\n<ul>\n<li>Appends the extracted data into a CSV file for future use.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Close Browser:<\/strong>\n<ul>\n<li>Ensures resources are freed by quitting the browser after scraping.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h1 id=\"Important Notes\" data-pm-slice=\"1 1 []\">Important Notes<\/h1>\n<ul data-spread=\"false\">\n<li><strong>Dynamic Selectors<\/strong>: Google frequently changes its HTML structure. You&#8217;ll need to update the CSS selectors in your scraper when the page structure changes.<\/li>\n<li><strong>Legal Considerations<\/strong>: Scraping data from websites may violate their terms of service. Use this script responsibly.<\/li>\n<\/ul>\n<h1 id=\"Conclusion\">Conclusion<\/h1>\n<p>Building a Google Shopping scraper in Python can provide you with valuable insights into product pricing and market competition. While the process is straightforward, keeping the scraper updated with Google&#8217;s ever-changing HTML is crucial. With this tool, you can extract and analyze data efficiently, giving you an edge in the competitive e-commerce landscape.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Table of contents Google Shopping Scraper Why Build a Google Shopping Scraper? Tools You&#8217;ll Need Prerequisites Writing the Scraper Code Explanation Important Notes Conclusion Google&hellip;<\/p>\n","protected":false},"author":23,"featured_media":3251,"comment_status":"open","ping_status":"closed","template":"","meta":{"rank_math_lock_modified_date":false},"categories":[],"class_list":["post-3250","scraping_project","type-scraping_project","status-publish","has-post-thumbnail","hentry"],"_links":{"self":[{"href":"https:\/\/rayobyte.com\/community\/wp-json\/wp\/v2\/scraping_project\/3250","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rayobyte.com\/community\/wp-json\/wp\/v2\/scraping_project"}],"about":[{"href":"https:\/\/rayobyte.com\/community\/wp-json\/wp\/v2\/types\/scraping_project"}],"author":[{"embeddable":true,"href":"https:\/\/rayobyte.com\/community\/wp-json\/wp\/v2\/users\/23"}],"replies":[{"embeddable":true,"href":"https:\/\/rayobyte.com\/community\/wp-json\/wp\/v2\/comments?post=3250"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rayobyte.com\/community\/wp-json\/wp\/v2\/media\/3251"}],"wp:attachment":[{"href":"https:\/\/rayobyte.com\/community\/wp-json\/wp\/v2\/media?parent=3250"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rayobyte.com\/community\/wp-json\/wp\/v2\/categories?post=3250"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}