News Feed Forums General Web Scraping How can I scrape mobile-only websites from a desktop environment?

  • How can I scrape mobile-only websites from a desktop environment?

    Posted by Bagher Isra on 11/14/2024 at 5:31 am

    You can set your user-agent to a mobile device profile, which tricks the server into thinking you’re on mobile. Just look up user-agent strings for popular mobile devices and apply it in your headers.

    Raul Marduk replied 1 month, 1 week ago 4 Members · 3 Replies
  • 3 Replies
  • Aridai Farzona

    Member
    11/15/2024 at 8:07 am

    Try using Playwright or Puppeteer, both of which allow you to emulate mobile devices completely. They also support different network profiles, which is handy if you need to mimic mobile network speeds.

  • Yannig Avicenna

    Member
    11/15/2024 at 8:28 am

    Sometimes I use proxies with mobile IPs. This, combined with a mobile user-agent, gives the best result because the target server believes the request is truly from a mobile device.

  • Raul Marduk

    Member
    11/15/2024 at 9:48 am

    If the site has a mobile API, consider using it directly. Many mobile sites use lighter APIs that are easier to parse and can be faster than scraping HTML.

Log in to reply.