Python requests

The Python requests library is a powerful tool for making HTTP requests, allowing you to interact with web services easily. It provides methods to send different types of HTTP requests (like GET, POST, PUT, DELETE) and handle the responses. This tutorial will guide you from the basics of using requests to more advanced techniques, complete with code examples and outputs.

Step 1: Installing the Requests Library

First, ensure that the requests library is installed. It is not included in Python’s standard library, so you can install it via pip.

pip install requests

Output:

Successfully installed requests

Step 2: Importing the Requests Module

Once installed, import requests in your Python script to start making HTTP requests.

import requests

Step 3: Sending a GET Request

The GET request is used to retrieve data from a specified resource. In requests, you can send a GET request using the requests.get() method.

# Basic GET request
url = "https://jsonplaceholder.typicode.com/posts"
response = requests.get(url)

print("Status Code:", response.status_code)
print("Response Body:", response.text)

Output:

Status Code: 200
Response Body: [{"userId":1,"id":1,"title":"..."}]  # Truncated for brevity

Explanation:
- `requests.get(url)`: Sends a GET request to the specified URL.
- `status_code`: Shows the HTTP status code of the response.
- `text`: Provides the content of the response as a string.

Step 4: Handling JSON Responses

You can directly parse JSON responses using the json() method, which is especially useful when working with APIs.

# Parsing JSON response
data = response.json()
print("First Post Title:", data[0]["title"])

Output:

First Post Title: sunt aut facere repellat provident occaecati excepturi optio reprehenderit

Explanation:
- `json()`: Converts the JSON response into a Python dictionary or list.

Step 5: Sending a POST Request

A POST request is used to send data to a server. You can include data in the data or json parameters when sending a POST request.

# Basic POST request
url = "https://jsonplaceholder.typicode.com/posts"
payload = {
    "title": "foo",
    "body": "bar",
    "userId": 1
}

response = requests.post(url, json=payload)
print("Status Code:", response.status_code)
print("Response JSON:", response.json())

Output:

Status Code: 201
Response JSON: {'id': 101, 'title': 'foo', 'body': 'bar', 'userId': 1}

Explanation:
- `requests.post(url, json=payload)`: Sends a POST request with the JSON data in payload.
- Response JSON: Contains the server's response data, parsed automatically.

Step 6: Adding Headers to Requests

You can add custom headers to a request by including a dictionary in the headers parameter. This is useful for setting API keys, content types, and user-agent information.

# Adding headers
url = "https://jsonplaceholder.typicode.com/posts"
headers = {
    "User-Agent": "MyApp",
    "Authorization": "Bearer YOUR_API_TOKEN"
}

response = requests.get(url, headers=headers)
print("Status Code with Headers:", response.status_code)

Output:

Status Code with Headers: 200

Explanation:
- `headers`: A dictionary of HTTP headers to send with the request.

Step 7: Handling Timeouts

Set a timeout for your request to ensure it doesn’t hang indefinitely. Use the timeout parameter with a specified time in seconds.

# Setting a timeout
url = "https://jsonplaceholder.typicode.com/posts"
try:
    response = requests.get(url, timeout=5)
    print("Request succeeded within timeout")
except requests.Timeout:
    print("Request timed out")

Output:

Request succeeded within timeout

Explanation:
- `timeout`: If the server does not respond within the specified time, a Timeout exception is raised.

Step 8: Using Sessions for Persistent Connections

Sessions allow you to maintain certain parameters, like cookies, across multiple requests.

# Using a session for multiple requests
session = requests.Session()
session.headers.update({"Authorization": "Bearer YOUR_API_TOKEN"})

response = session.get("https://jsonplaceholder.typicode.com/posts")
print("Session GET Status Code:", response.status_code)
session.close()

Output:

Session GET Status Code: 200

Explanation:
- `Session()`: Creates a session object to persist parameters across multiple requests.
- `headers.update()`: Updates the session with specific headers, such as authorization.

Step 9: Uploading Files

You can send files to the server using the files parameter with requests.post(). Ensure the file is opened in binary mode.

# Uploading a file
url = "https://httpbin.org/post"
files = {"file": open("example.txt", "rb")}

response = requests.post(url, files=files)
print("Uploaded File Response:", response.json())

Output:

Uploaded File Response: {'files': {'file': 'Example content'}}

Explanation:
- `files={"file": open("example.txt", "rb")}`: The files parameter allows you to send files with the request.

Step 10: Managing Cookies

You can send cookies with a request and retrieve cookies from the response. This can be managed using the cookies parameter.

# Sending and retrieving cookies
url = "https://httpbin.org/cookies"
cookies = {"session_id": "123456"}

response = requests.get(url, cookies=cookies)
print("Sent Cookies:", response.cookies)

Output:

Sent Cookies: <RequestsCookieJar[<Cookie session_id=123456 for httpbin.org/>]>

Explanation:
- `cookies={"key": "value"}`: Allows you to set cookies for the request.

Step 11: Advanced Authentication

Use the auth parameter with the requests.auth module to perform basic or more advanced authentication.

# Basic authentication
from requests.auth import HTTPBasicAuth

url = "https://httpbin.org/basic-auth/user/pass"
response = requests.get(url, auth=HTTPBasicAuth("user", "pass"))
print("Authenticated Status:", response.status_code)

Output:

Authenticated Status: 200

Explanation:
- `HTTPBasicAuth(user, pass)`: Provides basic HTTP authentication with the given credentials.

Step 12: Streaming Large Requests

Streaming allows you to handle large files without loading them entirely in memory, using the stream=True parameter.

# Streaming large data
url = "https://httpbin.org/stream/20"
response = requests.get(url, stream=True)

for line in response.iter_lines():
    if line:
        print("Line:", line.decode("utf-8"))

Output:

Line: {"id":0,"name":"Test"}
...

Summary

This tutorial covers the requests library, from basic requests to advanced operations like session management, cookies, and authentication.