How to use cURL with PHP Simple HTML DOM for data scraping

By | May 29, 2017

cURL allows transfer of data across a wide variety of protocols, and is a very powerful system. Once you have compiled PHP with cURL support, you can start using the cURL functions. The basic idea behind the cURL functions is that you initialize a cURL session using the curl_init(), then you can set required options for the request via the curl_setopt(), then you can execute the session with the curl_exec() and then you close the session using the curl_close().

curl - proxies - simple html dom

Using proxies allows your requests to go through many different IP addresses and thus appear to be coming from different visitors.

This article addresses following questions:
1) How to use SimpleHTMLDom library with cURL
2) How to use Proxies For Scraping With PHP & cURL
3) How to use Proxies with SimpleHTMLDom library

Here is an example that uses the cURL functions to fetch the webscrapingblog.com homepage using cURL and proxies. After fetching the HTML, we can parse it using Simple HTML DOM library for data scraping.

And there we have it, Script with cURL, proxies and Simple HTML Dom library.

Let me know if you have any questions in comment.

2 thoughts on “How to use cURL with PHP Simple HTML DOM for data scraping

  1. Zubair

    I tried you script by copy/paste.
    It’s not working.
    Not showing anything.
    As soon as I open script, browser page loading is completed immediately and page shows nothing no data not error.

    Reply
    1. Web Scraper Post author

      Just copy/paste don’t work bro.! You need to apply mind. Have you set real proxies in it?

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *