文章目录

1. 1. Python 标准库系列之 requests 模块
   1. 1.1. 安装Requests模块
   2. 1.2. 环境准备
   3. 1.3. 简单的一个 requests 小程序
   4. 1.4. 发送请求
   5. 1.5. 实例

Python 标准库系列之 requests 模块

Requests is the only Non-GMO HTTP library for Python, safe for human consumption.

官方文档：http://docs.python-requests.org/en/master/

安装Requests模块

Requests模块官方提供了两种方式安装：

pip 方式安装

pip install requests

源码方式安装

git clone git://github.com/kennethreitz/requests.git
cd requests
python setup.py install

验证是否安装成功

进入python解释的，导入模块试试，如果导入成功则安装成功，否则就需要检查那里执行错误了呢。

C:\Users\anshengme> python
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2016, 01:54:25) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests

环境准备

安装gunicorn和httpbin

sudo pip3 install gunicorn httpbin

启动一个gunicornServer

⇒  sudo gunicorn httpbin:app
[2016-10-27 11:45:08 +0800] [12175] [INFO] Starting gunicorn 19.6.0
[2016-10-27 11:45:08 +0800] [12175] [INFO] Listening at: https://blog.ansheng.me:8000 (12175)
[2016-10-27 11:45:08 +0800] [12175] [INFO] Using worker: sync
[2016-10-27 11:45:08 +0800] [12178] [INFO] Booting worker with pid: 12178

打开浏览器输入将会得到以下页面，相当于在本地启动一个 http server 便于学习 requests 模块

简单的一个 requests 小程序

下面的一个小程序会通过requests请求'https://blog.ansheng.me:8000/ip'这个 URI 链接，获取到对应的数据。

#!/use/bin/env python3
# _*_ coding:utf-8 _*_
import requests

URL_IP = 'https://blog.ansheng.me:8000/ip'

def use_params_requests():
    # 参数
    params = {'params1': 'Hello', 'params2': 'World'}
    # 发送请求
    response = requests.get(URL_IP, params=params)
    print("响应的状态码：", response.status_code, response.reason)
    print("返回的头部：", response.headers)
    print("把返回的数据转换为json:", response.json())
    print("响应的文本：", response.text)

if __name__ == '__main__':
    use_params_requests()

发送请求

通过 GITHUB 提供的 API 获取用户信息

#!/use/bin/env python3
# _*_ coding:utf-8 _*_
import requests
import json

URL = 'https://api.github.com'


def build_uri(endpoint):
    return '/'.join([URL, endpoint])


def better_print(json_str):
    return json.dumps(json.loads(json_str), indent=4)


def request_method():
    # 获取用户的所有信息
    response = requests.get(build_uri('users/anshengme'))
    print(better_print((response.text)))

    print("\n")

    # 获取用户的邮箱
    response = requests.get(build_uri('user/emails'), auth=("anshengme.com@gmail.com", "xxxxxx"))
    print(better_print((response.text)))


if __name__ == '__main__':
    request_method()

带参数的请求

# 使用params传参
def params_request():
    response = requests.get(build_uri('users'), params={'since': 11})
    print(better_print(response.text))
    print(response.request.headers)
    print(response.url)

# 使用json传参方式
def json_request():
    response = requests.get(build_uri('user'), auth=("username", "email"), json={"name": "asdas"})
    print(better_print(response.text))

异常处理

def timeout_request():
    try:
        response = requests.get(build_uri('user/emails'), timeout=10)
        response.raise_for_status()
    except Exception as e:
        print(e)
    else:
        print(response.text)
        print(response.status_code)

自定义 request

from requests import Request, Session

def hard_request():
    s = Session()
    # 创建请求
    headers = {'User-Agent': 'fake1.3.4'}
    req = Request('GET', build_uri('user/emails'), auth=('anshengme.com@gmail.com', 'xxxxxx'), headers=headers)
    prepped = req.prepare()
    print("请求头》》", prepped.headers)
    # 发送请求
    resp = s.send(prepped)
    print(resp.status_code)
    print(resp.request.headers)
    print(resp.text)

实例

下载图片/文件

#!/use/bin/env python3
# _*_ coding:utf-8 _*_
import requests
from contextlib import closing


# 流传输的模式
def download_img():
    url = "http://www.sinaimg.cn/IT/cr/2016/0331/725124517.jpg"
    # headers = {
    #     'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36'}
    # response = requests.get(url, headers=headers, stream=True)
    response = requests.get(url, stream=True)
    print(response.status_code, response.reason)

    with open('github.jpg', 'wb') as fd:
        for chunk in response.iter_content(128):
            fd.write(chunk)

def download_img_improved():
    url = "http://www.sinaimg.cn/IT/cr/2016/0331/725124517.jpg"
    with closing(requests.get(url, stream=True)) as response:
        # 打开并写入文件
        with open('github1.jpg', 'wb') as fd:
            for chunk in response.iter_content(128):
                fd.write(chunk)

download_img()
download_img_improved()

处理响应的事件钩子

#!/use/bin/env python3
# _*_ coding:utf-8 _*_
import requests

def get_key_info(response, *args, **kwargs):
    print(response.headers["Content-Type"])

def main():
    requests.get("https://www.baidu.com", hooks=dict(response=get_key_info))

if __name__ == "__main__":
    main()