• General Web Scraping

  PHP & PostgreSQL Scraping Chotot.com: Extracting Classified Listings, Prices, and Seller Information for Market Research

  Posted by Joel Farzana on 02/12/2025 at 6:33 pm

  The Evolution of PHP: From Personal Home Page to Hypertext Preprocessor

  Introduction to PHP

  PHP, originally created by Rasmus Lerdorf in 1994, started as a simple set of Common Gateway Interface (CGI) binaries written in the C programming language. Initially, it was designed to track visits to Lerdorf’s online resume, but it quickly evolved into a more robust scripting language. The acronym PHP originally stood for “Personal Home Page,” reflecting its initial purpose.

  As the internet grew, so did the capabilities of PHP. By 1997, PHP had transformed into PHP/FI (Form Interpreter), which included basic functionality for web form handling and database interaction. This marked the beginning of PHP’s journey as a server-side scripting language.

  In 1998, PHP 3 was released, marking a significant milestone in its evolution. This version introduced a new parser engine, which allowed for greater extensibility and performance. The acronym PHP was redefined to “PHP: Hypertext Preprocessor,” a recursive acronym that better represented its capabilities.

  PHP 4, released in 2000, brought further improvements with the introduction of the Zend Engine, which enhanced performance and stability. This version solidified PHP’s position as a leading server-side scripting language for web development.

  Today, PHP is a powerful and versatile language used by millions of websites worldwide. Its evolution from a simple tool for personal use to a robust web development language is a testament to its adaptability and widespread adoption.

  Key Features and Advantages of Using PHP in Web Development

  Open Source and Community Support

  One of the most significant advantages of PHP is its open-source nature. Being open source means that PHP is free to use, and its source code is available for anyone to view, modify, and distribute. This has led to a large and active community of developers who contribute to its continuous improvement.

  The PHP community provides extensive documentation, tutorials, and forums where developers can seek help and share knowledge. This collaborative environment fosters innovation and ensures that PHP remains up-to-date with the latest web development trends.

  Moreover, the open-source nature of PHP allows developers to customize and extend its functionality to suit their specific needs. This flexibility is particularly beneficial for businesses looking to create unique and tailored web applications.

  Many popular content management systems (CMS) and frameworks, such as WordPress, Drupal, and Laravel, are built on PHP. This widespread adoption further demonstrates the language’s reliability and effectiveness in web development.

  Overall, the open-source nature and strong community support make PHP an attractive choice for developers seeking a cost-effective and adaptable solution for web development projects.

  Ease of Learning and Use

  PHP is known for its simplicity and ease of learning, making it an ideal choice for beginners in web development. Its syntax is straightforward and similar to other programming languages like C and Java, which makes it accessible to those with prior programming experience.

  The language’s extensive documentation and numerous online resources provide ample support for new developers. Tutorials, guides, and forums are readily available, allowing beginners to quickly grasp the fundamentals of PHP and start building web applications.

  PHP’s ease of use extends to its integration with HTML, which is a core component of web development. Developers can seamlessly embed PHP code within HTML files, enabling dynamic content generation and server-side processing.

  Additionally, PHP’s built-in functions and libraries simplify common web development tasks, such as form handling, database interaction, and file manipulation. This reduces the need for writing complex code from scratch, allowing developers to focus on building functional and efficient applications.

  Overall, PHP’s ease of learning and use makes it an attractive option for both novice and experienced developers looking to create dynamic and interactive web applications.

  Cross-Platform Compatibility

  PHP is a cross-platform language, meaning it can run on various operating systems, including Windows, Linux, macOS, and Unix. This compatibility ensures that PHP applications can be deployed on a wide range of servers and environments.

  The language’s ability to work seamlessly across different platforms is a significant advantage for developers and businesses. It allows for greater flexibility in choosing hosting providers and server configurations, reducing potential compatibility issues.

  PHP’s cross-platform nature also facilitates collaboration among development teams using different operating systems. Developers can work on the same project regardless of their preferred platform, streamlining the development process and enhancing productivity.

  Furthermore, PHP’s compatibility with various web servers, such as Apache, Nginx, and Microsoft IIS, ensures that applications can be hosted on diverse server environments. This versatility is particularly beneficial for businesses with specific hosting requirements or those looking to switch hosting providers.

  In summary, PHP’s cross-platform compatibility provides developers with the flexibility to deploy applications on various operating systems and servers, making it a versatile choice for web development projects.

  Robust Database Support

  PHP offers robust support for a wide range of databases, making it an excellent choice for data-driven web applications. It can interact with popular databases such as MySQL, PostgreSQL, SQLite, Oracle, and Microsoft SQL Server, among others.

  The language’s built-in database functions and extensions simplify the process of connecting to and interacting with databases. For example, PHP’s MySQLi and PDO extensions provide a secure and efficient way to perform database operations, such as querying, inserting, updating, and deleting data.

  PHP’s database support extends to advanced features like prepared statements and transactions, which enhance security and data integrity. Prepared statements help prevent SQL injection attacks by separating SQL code from user input, while transactions ensure that a series of database operations are executed as a single unit.

  Moreover, PHP’s compatibility with various database management systems allows developers to choose the best database solution for their specific needs. This flexibility is particularly beneficial for businesses with existing database infrastructure or those looking to scale their applications.

  Overall, PHP’s robust database support makes it a powerful tool for building data-driven web applications, providing developers with the tools they need to efficiently manage and manipulate data.

  Scalability and Performance

  PHP is known for its scalability and performance, making it suitable for both small-scale projects and large enterprise applications. Its lightweight nature and efficient execution model allow it to handle high traffic loads and deliver fast response times.

  The language’s scalability is further enhanced by its ability to integrate with various caching solutions, such as Memcached and Redis. These caching systems help reduce server load and improve application performance by storing frequently accessed data in memory.

  PHP’s performance is also boosted by its compatibility with content delivery networks (CDNs) and load balancers. CDNs distribute content across multiple servers, reducing latency and improving user experience, while load balancers distribute incoming traffic evenly across servers, ensuring optimal resource utilization.

  Additionally, PHP’s support for asynchronous programming and multithreading allows developers to build applications that can handle multiple tasks simultaneously. This capability is particularly beneficial for applications that require real-time data processing or need to handle numerous concurrent users.

  In conclusion, PHP’s scalability and performance make it a reliable choice for web development projects of all sizes. Its ability to handle high traffic loads and deliver fast response times ensures that applications remain responsive and efficient, even under heavy usage.

  Step 1: PHP Script for Scraping Chotot.com

  This script uses cURL to fetch the HTML content and PHP DOMDocument to parse it.

  <?php

  $targetUrl = “https://www.chotot.com/toan-quoc/mua-ban”; // Example: Chotot’s classified listings page

  // Initialize cURL session
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $targetUrl);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
  curl_setopt($ch, CURLOPT_USERAGENT, “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36”);

  // Execute the request and store the response
  $response = curl_exec($ch);
  curl_close($ch);

  if (!$response) {
  die(“Failed to fetch data from Chotot.com”);
  }

  // Load the response into DOMDocument for parsing
  libxml_use_internal_errors(true); // Suppress HTML parsing warnings
  $dom = new DOMDocument();
  $dom->loadHTML($response);
  libxml_clear_errors();

  // Create XPath object to query the DOM
  $xpath = new DOMXPath($dom);

  // Extract product details (Adjust XPath queries based on actual HTML structure)
  $listings = [];
  $nodes = $xpath->query(“//div[contains(@class, ‘AdItem_adItem__’)]”);

  foreach ($nodes as $node) {
  $title = $xpath->query(“.//h3”, $node)->item(0);
  $price = $xpath->query(“.//span[contains(@class, ‘price’)]”, $node)->item(0);
  $seller = $xpath->query(“.//span[contains(@class, ‘seller’)]”, $node)->item(0);

  if ($title && $price) {
  $listings[] = [
  “title” => trim($title->textContent),
  “price” => trim($price->textContent),
  “seller” => $seller ? trim($seller->textContent) : “Unknown”
  ];
  }
  }

  // Print scraped data
  echo json_encode($listings, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE);

  // Store data in PostgreSQL
  storeDataInPostgreSQL($listings);

  /**
  * Function to insert data into PostgreSQL
  */
  function storeDataInPostgreSQL($data) {
  $host = “localhost”;
  $dbname = “chotot_scraper”;
  $user = “postgres”;
  $password = “yourpassword”;

  try {
  $pdo = new PDO(“pgsql:host=$host;dbname=$dbname”, $user, $password, [PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION]);

  $stmt = $pdo->prepare(“INSERT INTO listings (title, price, seller) VALUES (:title, :price, :seller)”);

  foreach ($data as $item) {
  $stmt->execute([
  ‘:title’ => $item[‘title’],
  ‘:price’ => $item[‘price’],
  ‘:seller’ => $item[‘seller’]
  ]);
  }

  echo “\nData successfully inserted into PostgreSQL.\n”;
  } catch (PDOException $e) {
  die(“Database error: ” . $e->getMessage());
  }
  }

  ?>

  Step 2: PostgreSQL Database Schema

  Before running the script, create a PostgreSQL database and table to store the scraped data.

  CREATE DATABASE chotot_scraper;

  \c chotot_scraper;

  CREATE TABLE listings (
  id SERIAL PRIMARY KEY,
  title TEXT NOT NULL,
  price TEXT NOT NULL,
  seller TEXT
  );

  How It Works

  1. Sends a request to Chotot’s classified listings page.
  2. Uses DOMDocument & XPath to parse HTML and extract:
     • Title
     • Price
     • Seller Name
  3. Prints the extracted data in JSON format.
  4. Stores the data into a PostgreSQL database.

  Joel Farzana replied 1 week, 2 days ago 1 Member · 0 Replies
• 0 Replies

Log In to Reply