-
How can I scrape mobile-only websites from a desktop environment?
You can set your user-agent to a mobile device profile, which tricks the server into thinking you’re on mobile. Just look up user-agent strings for popular mobile devices and apply it in your headers.
-
3 Replies
-
Try using Playwright or Puppeteer, both of which allow you to emulate mobile devices completely. They also support different network profiles, which is handy if you need to mimic mobile network speeds.
-
Sometimes I use proxies with mobile IPs. This, combined with a mobile user-agent, gives the best result because the target server believes the request is truly from a mobile device.
-
If the site has a mobile API, consider using it directly. Many mobile sites use lighter APIs that are easier to parse and can be faster than scraping HTML.
Log in to reply.