How to Download Files with Curl Command in Linux

Download Files with Curl Command in Linux

Have you ever needed to download files from the command line but felt overwhelmed by the options? You’re not alone! The curl command is one of Linux’s most powerful tools for transferring data, and mastering it can transform how you handle file downloads. Whether you’re a system administrator managing servers or a developer working with APIs, understanding curl will make your life significantly easier.

What is Curl and Why Should You Use It?

Understanding Curl’s Core Functionality

Curl (short for “Client URL”) is a command-line tool that’s been around since 1998, making it older than Google itself! This versatile utility allows you to transfer data to and from servers using various protocols. Think of curl as your Swiss Army knife for internet communications – it handles everything from simple file downloads to complex API interactions.

The beauty of curl lies in its simplicity and ubiquity. It’s pre-installed on most Linux distributions, macOS, and even available on Windows. With support for over 20 protocols including HTTP, HTTPS, FTP, SFTP, and many more, curl can handle virtually any data transfer scenario you throw at it.

Curl vs Other Download Tools

You might wonder, “Why use curl when I could use wget or a browser?” Great question! While wget is excellent for recursive downloads and mirroring websites, curl excels in flexibility and protocol support. Browsers are great for humans, but curl shines in automation and scripting environments where human interaction isn’t possible.

Curl’s non-interactive nature makes it perfect for scripts, cron jobs, and automated systems. Plus, its extensive option set gives you granular control over every aspect of your downloads – something you simply can’t achieve with a point-and-click interface.

Getting Started with Curl Installation

Checking if Curl is Already Installed

Before diving into downloads, let’s verify curl is available on your system. Open your terminal and type:

curl --version

If curl is installed, you’ll see version information and supported protocols. Most modern Linux distributions include curl by default, so chances are you’re ready to go!

Installing Curl on Different Linux Distributions

If curl isn’t installed, here’s how to get it on popular distributions:

Ubuntu/Debian:

sudo apt update
sudo apt install curl

CentOS/RHEL/Fedora:

sudo yum install curl
# or for newer versions:
sudo dnf install curl

Arch Linux:

sudo pacman -S curl

Basic Curl Syntax and Commands

Understanding the Command Structure

The basic curl syntax follows this pattern:

curl [options] URL

Without any options, curl simply displays the content of the URL in your terminal. This behavior is perfect for quickly viewing text files or API responses, but for file downloads, we need specific options.

Essential Curl Options for File Downloads

Here are the most important options you’ll use for downloading files:

  • -O (capital O): Saves the file with its original name
  • -o (lowercase o): Saves with a custom name you specify
  • -L: Follows redirects automatically
  • -C -: Resumes interrupted downloads
  • -#: Shows a simple progress bar
  • -s: Silent mode (no progress meter)

Simple File Download Methods

Basic File Download with -O Option

The most straightforward way to download a file is using the -O option:

curl -O https://example.com/file.pdf

This command downloads file.pdf and saves it in your current directory with the same name. The -O option is perfect when you want to preserve the original filename and don’t need to rename it.

Here’s a real-world example downloading a Linux kernel archive:

curl -O https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.5.tar.xz

Downloading with Custom File Names using -o

Sometimes you need more control over the filename. The -o option lets you specify exactly what to name your downloaded file:

curl -o my-document.pdf https://example.com/very-long-filename.pdf

You can even specify a different directory:

curl -o /home/user/downloads/report.pdf https://company.com/quarterly-report.pdf

When to Use -O vs -o

Use -O when:

  • You want to keep the original filename
  • Downloading multiple files with their original names
  • The remote filename is already descriptive

Use -o when:

  • You need a specific naming convention
  • Saving to a different directory
  • The original filename is unclear or too long

Advanced Download Techniques

Following Redirects with -L Option

Many websites use redirects to send you from HTTP to HTTPS or to different servers. By default, curl doesn’t follow these redirects, which can cause failed downloads. The -L option solves this:

curl -L -O https://github.com/user/repo/archive/main.zip

Without -L, you might end up with an HTML page instead of your intended file!

Downloading Multiple Files Simultaneously

Curl can download several files in one command:

curl -O https://example.com/file1.zip -O https://example.com/file2.zip

For better organization, you can combine this with custom naming:

curl -o backup1.zip https://server.com/backup1.zip \
     -o backup2.zip https://server.com/backup2.zip

Using Loops for Batch Downloads

When downloading many files, bash loops become invaluable:

for i in {1..10}; do
    curl -O https://example.com/file$i.txt
done

Or from a file containing URLs:

while read url; do
    curl -O "$url"
done < urls.txt

Working with Different Protocols

HTTP and HTTPS Downloads

HTTP and HTTPS are curl’s bread and butter. Most web downloads use these protocols, and curl handles them seamlessly:

# HTTP download
curl -O http://example.com/file.zip

# HTTPS download (most common)
curl -O https://secure.example.com/file.zip

FTP File Downloads

FTP downloads are just as straightforward:

curl -O ftp://ftp.example.com/path/to/file.tar.gz

For password-protected FTP servers:

curl -u username:password -O ftp://ftp.example.com/file.tar.gz

SFTP and Secure File Transfers

SFTP provides encrypted file transfers. While curl supports SFTP, many distributions compile it without SFTP support. When available, use:

curl -u username:password -O sftp://server.com/path/to/file.txt

Authentication and Security

Downloading Password-Protected Files

Web servers often require authentication. Curl supports various authentication methods:

Basic Authentication:

curl -u username:password -O https://secure.example.com/file.zip

From a file (more secure):

curl -u username -O https://secure.example.com/file.zip

(curl will prompt for the password)

Using Credentials with Different Protocols

Each protocol handles authentication differently:

FTP with credentials:

curl -u user:pass ftp://ftp.server.com/file.tar.gz -o file.tar.gz

HTTP with bearer token:

curl -H "Authorization: Bearer YOUR_TOKEN" -O https://api.example.com/file.json

SSL Certificate Handling

Sometimes you’ll encounter SSL certificate issues. While not recommended for production, you can bypass certificate verification:

curl -k -O https://self-signed.example.com/file.zip

The -k flag ignores certificate errors – use this only for testing or known safe sources!

Monitoring and Managing Downloads

Understanding Progress Meters

Curl shows download progress by default. The progress meter displays:

  • Percentage completed
  • Total size
  • Downloaded amount
  • Average speed
  • Time remaining

For a simpler progress bar, use the -# option:

curl -# -O https://example.com/large-file.iso

Setting Download Speed Limits

To avoid overwhelming your connection or the server, limit download speed:

# Limit to 200KB/s
curl --limit-rate 200k -O https://example.com/file.zip

# Limit to 1MB/s
curl --limit-rate 1m -O https://example.com/file.zip

Resume Interrupted Downloads

Large downloads sometimes get interrupted. Curl can resume where it left off:

curl -C - -O https://example.com/large-file.iso

The -C - tells curl to automatically determine where to resume from.

Practical Examples and Use Cases

Downloading Software Packages

System administrators frequently download software packages:

# Download Docker installation script
curl -fsSL https://get.docker.com -o get-docker.sh

# Download and install Node.js
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash

API Data Retrieval

Developers often use curl for API testing and data retrieval:

# Download JSON data
curl -H "Accept: application/json" \
     -o data.json \
     https://api.example.com/users

# Download with authentication
curl -H "Authorization: Bearer TOKEN" \
     -o api-response.json \
     https://api.example.com/protected-data

Automation Scripts

Curl excels in automation scenarios:

#!/bin/bash
# Daily backup download script
DATE=$(date +%Y%m%d)
curl -u backup_user:secure_pass \
     -o "backup_${DATE}.tar.gz" \
     "https://backup.server.com/daily/backup_${DATE}.tar.gz"

Troubleshooting Common Issues

Connection Problems

Common connection issues and solutions:

Timeout errors: Increase timeout with --connect-timeout 30
DNS problems: Use --resolve to specify IP addresses
Proxy issues: Configure proxy with --proxy http://proxy:8080

File Permission Issues

If downloads fail due to permissions:

  • Ensure you have write permissions in the target directory
  • Use sudo only when necessary
  • Consider downloading to /tmp first, then moving files

Best Practices and Tips

  1. Always use HTTPS when possible for security
  2. Include error checking in scripts: curl -f fails silently on HTTP errors
  3. Set reasonable timeouts to avoid hanging processes
  4. Use progress bars (-#) for interactive sessions
  5. Quote URLs containing special characters
  6. Test downloads with small files before automating large transfers
  7. Keep curl updated for the latest security fixes and protocol support

Remember, curl is incredibly powerful, but with great power comes great responsibility. Always verify the source of files you’re downloading and be mindful of server resources when scripting downloads.

Frequently Asked Questions

1. What’s the difference between curl -O and curl -o?

The -O (uppercase) option saves the file with its original name from the server, while -o (lowercase) allows you to specify a custom filename. Use -O when you want to keep the original name and -o when you need control over the filename or save location.

2. How can I resume a partially downloaded file with curl?

Use the -C - option to automatically resume an interrupted download: curl -C - -O https://example.com/large-file.zip. Curl will determine where the download stopped and continue from that point.

3. Why does my download fail even though the URL works in a browser?

This often happens due to redirects. Browsers automatically follow redirects, but curl doesn’t by default. Add the -L option to follow redirects: curl -L -O https://example.com/file.zip.

4. How do I download files that require authentication?

Use the -u option followed by username:password format: curl -u username:password -O https://secure.example.com/file.zip. For better security, omit the password to make curl prompt for it interactively.

5. Can I limit the download speed to avoid overwhelming my connection?

Yes! Use the --limit-rate option to set speed limits: curl --limit-rate 200k -O https://example.com/file.zip limits the download to 200KB/s. You can specify rates in bytes (b), kilobytes (k), or megabytes (m).

Marshall Anthony is a professional Linux DevOps writer with a passion for technology and innovation. With over 8 years of experience in the industry, he has become a go-to expert for anyone looking to learn more about Linux.

Related Posts