溫馨提示×
您好，登錄后才能下訂單哦！
點擊 登錄注冊 即表示同意《億速云用戶服務條款》
您好，登錄后才能下訂單哦！
這篇文章將為大家詳細講解有關Python爬蟲中搜索文檔樹的方法，小編覺得挺實用的，因此分享給大家做個參考，希望大家閱讀完這篇文章后可以有所收獲。
搜索文檔樹
1.find_all(name, attrs, recursive, text, **kwargs)
1)name參數
name參數可以查找所有名字為name的Tag，字符串對象會被自動忽略掉。
a.傳字符串
最簡單的過濾器就是字符串，在搜索方法中傳入一個字符串參數，Beautiful Soup會查找與字符串完整匹配所有的內容，返回一個列表。
#!/usr/bin/python3
# -*- coding:utf-8 -*-
from bs4 import BeautifulSoup
html = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title" name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
# 創建 Beautiful Soup 對象，指定lxml解析器
soup = BeautifulSoup(html, "lxml")
print(soup.find_all("b"))
print(soup.find_all("a"))運行結果
[<b>The Dormouse's story</b>] [<a class="sister" href="http://example.com/elsie" id="link1"><!-- Elsie --></a>, <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>, <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>]
B.傳正則表達式
如果傳入正則表達式作為參數，Beautiful Soup會通過正則表達式match()來匹配內容。
#!/usr/bin/python3
# -*- coding:utf-8 -*-
from bs4 import BeautifulSoup
import re
html = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title" name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
# 創建 Beautiful Soup 對象，指定lxml解析器
soup = BeautifulSoup(html, "lxml")
for tag in soup.find_all(re.compile("^b")):
print(tag.name)運行結果
body b
C.傳列表
如果傳入列表參數，Beautiful Soup會將與列表中任一元素匹配的內容以列表方式返回。
#!/usr/bin/python3 # -*- coding:utf-8 -*- from bs4 import BeautifulSoup html = """ <html><head><title>The Dormouse's story</title></head> <body> <p class="title" name="dromouse"><b>The Dormouse's story</b></p> <p class="story">Once upon a time there were three little sisters; and their names were <a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>, <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>; and they lived at the bottom of a well.</p> <p class="story">...</p> """ # 創建 Beautiful Soup 對象，指定lxml解析器 soup = BeautifulSoup(html, "lxml") print(soup.find_all(['a', 'b']))
2)keyword參數
#!/usr/bin/python3 # -*- coding:utf-8 -*- from bs4 import BeautifulSoup html = """ <html><head><title>The Dormouse's story</title></head> <body> <p class="title" name="dromouse"><b>The Dormouse's story</b></p> <p class="story">Once upon a time there were three little sisters; and their names were <a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>, <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>; and they lived at the bottom of a well.</p> <p class="story">...</p> """ # 創建 Beautiful Soup 對象，指定lxml解析器 soup = BeautifulSoup(html, "lxml") print(soup.find_all(id="link1"))
運行結果
[<a class="sister" href="http://example.com/elsie" id="link1"><!-- Elsie --></a>]
3)text參數
通過text參數可以搜索文檔中的字符串內容，與name參數的可選值一樣，text參數接受字符串，正則表達式，列表。
#!/usr/bin/python3
# -*- coding:utf-8 -*-
from bs4 import BeautifulSoup
import re
html = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title" name="dromouse"><b>The Dormouse's story</b></p>
<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>
<p class="story">...</p>
"""
# 創建 Beautiful Soup 對象，指定lxml解析器
soup = BeautifulSoup(html, "lxml")
# 字符串
print(soup.find_all(text = " Elsie "))
# 列表
print(soup.find_all(text = ["Tillie", " Elsie ", "Lacie"]))
# 正則表達式
print(soup.find_all(text = re.compile("Dormouse")))運行結果
[' Elsie '] [' Elsie ', 'Lacie', 'Tillie'] ["The Dormouse's story", "The Dormouse's story"]
關于Python爬蟲中搜索文檔樹的方法就分享到這里了，希望以上內容可以對大家有一定的幫助，可以學到更多知識。如果覺得文章不錯，可以把它分享出去讓更多的人看到。
免責聲明：本站發布的內容（圖片、視頻和文字）以原創、轉載和分享為主，文章觀點不代表本網站立場，如果涉及侵權請聯系站長郵箱：is@yisu.com進行舉報，并提供相關證據，一經查實，將立刻刪除涉嫌侵權內容。
