In this post, you will learn how to use the command line tool cURL to transfer data using a proxy, more specifically one of our mobile proxies. If you didn't already know, a proxy is a server that acts as a middleman between you the client, and the destination server itself. To learn more about what proxies and mobile proxies are, check out our post on them here.
Learn how to use cURL with a proxy
Here are quick links to four different methods of using curl with a proxy for your convenience.
What exactly is cURL?
Before we get started, let's dig into what cURL actually is for some context. cURL is a command line tool that can be used for transferring data, simple as that. It is free, open source, and used by virtually every internet-using human on the planet.
The most basic use of curl is to call it against a URL, for example, google.com. This will output the content of the webpage to
STDOUT
curl google.com
This returns to us:
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
Proxy Details
Okay great, now that we know what cURL is, let's get some proxy details set up for the rest of this post.
If you don't already have a proxy, signup for an account and purchase one of our mobile proxy offerings!
For the sake of this post, we will use the following proxy details, based on a test proxy I have running on my local machine.
- Proxy Hostname: 127.0.0.1
- Proxy Port: 10001
- Proxy Type: HTTP
- Username: user
- Password: pass
With command line arguments
The easiest way to use a proxy with cURL is through the use of command-line arguments. This can be done as follows:
curl -U user:pass -x http://127.0.01:10001 whatismyip.akamai.com
This works great for an HTTP proxy as we have here, but in the case that you have a SOCKS5 proxy you can do the following:
curl whatismyip.akamai.com --socks5 user:[email protected]:10001
With environment variables
If you are a fan of using environment variables, it is possible to configure cURL to use them. You can set a variable for each protocol that you need to use, which is super handy. Each variable name follows the
_proxy
export http_proxy="http://user:[email protected]:10001"
export https_proxy="https://user:[email protected]:10001"
Once you have configured these variables for your protocol of choice, you can drop the proxy details from your cURL commands, as it will use these environment variables by default
With an alias
A slightly more permanent option, but a little bit more implicit, is to setup your proxies via an alias. This is great if you need to regularly connect via the same proxy through cURL
This is done by substituting the call to
curl
alias curl="curl -x http://user:[email protected]:10001"
What this means, is that when we call
curl whatismyip.akamai.comcurl -x http://user:[email protected]:10001 whatismyip.akamai.comWith a .curlrc
If you are going with the above approach, a slightly better option is to configure your cURL settings through a configuration file. The file itself can be specified on the command line via the -K argument, and cURL will always search by default for one in your home directory at
~/.curlrcTo configure cURL to use a proxy through the curlrc you can add the following to your configuration file:
proxy = "http://user:[email protected]:10001"
And with that cURL will use your proxy settings from the configuration file!
Conclusion
So there you have it, a quick guide and summary of the easiest ways to use a proxy through cURL!
If you are planning on making a lot of requests through one proxy, I would always recommend the use of a configuration file. This will save you a lot of headaches and time in the long run.
Sign up for an account and join our grid today to keep you ahead of your competitors.
