Python爬虫：一个爬取豆瓣电影人像的小案例-Toy模板网

这篇具有很好参考价值的文章主要介绍了Python爬虫：一个爬取豆瓣电影人像的小案例。希望对大家有所帮助。如果存在错误或未考虑完全的地方，请大家不吝赐教，您也可以点击"举报违法"按钮提交疑问。

从谷歌浏览器的开发工具进入
选择图片右键点击检查 Python爬虫：一个爬取豆瓣电影人像的小案例,爬虫,爬虫,python,开发语言

Python爬虫：一个爬取豆瓣电影人像的小案例,爬虫,爬虫,python,开发语言

翻页之后发现网址变化的只有start数值，每次变化值为30
Python爬虫：一个爬取豆瓣电影人像的小案例,爬虫,爬虫,python,开发语言
Python代码

import requests
from bs4 import BeautifulSoup
import time
import os


# 豆瓣影人图片
url = 'https://movie.douban.com/celebrity/1011562/photos/'
res = requests.get(url=url, headers="").text
content = BeautifulSoup(res, "html.parser")
data = content.find_all('div', attrs={'class': 'cover'})
picture_list = []
for d in data:
    plist = d.find('img')['src']
    picture_list.append(plist)
print(picture_list)


# https://movie.douban.com/celebrity/1011562/photos/?type=C&start=30&sortby=like&size=a&subtype=a
def get_poster_url(res):
    content = BeautifulSoup(res, "html.parser")
    data = content.find_all('div', attrs={'class': 'cover'})
    picture_list = []
    for d in data:
        plist = d.find('img')['src']
        picture_list.append(plist)
    return picture_list

# XPath://*[@id="content"]/div/div[1]/ul/li[1]/div[1]/a/img
def download_picture(pic_l):
    if not os.path.exists(r'picture'):
        os.mkdir(r'picture')
    for i in pic_l:
        pic = requests.get(i)
        p_name = i.split('/')[7]
        with open('picture\\' + p_name, 'wb') as f:
            f.write(pic.content)

def fire():
    page = 0
    for i in range(0, 450, 30):
        print("开始爬取第 %s 页" % page)
        url = 'https://movie.douban.com/celebrity/1011562/photos/?type=C&start={}&sortby=like&size=a&subtype=a'.format(i)
        res = requests.get(url=url, headers="").text
        data = get_poster_url(res)
        download_picture(data)
        page += 1
        time.sleep(1)


fire()