在Python3中,要繞過反爬蟲機制,你可以采取以下策略:
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3"
}
url = "https://example.com"
response = requests.get(url, headers=headers)
import requests
proxies = {
"http": "http://proxy.example.com:8080",
"https": "https://proxy.example.com:8080"
}
url = "https://example.com"
response = requests.get(url, proxies=proxies)
import time
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3"
}
url = "https://example.com"
response = requests.get(url, headers=headers)
time.sleep(5) # 等待5秒
import requests
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3",
"Cookies": "your_cookies_here"
}
url = "https://example.com"
response = requests.get(url, headers=headers)
from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://example.com")
請注意,繞過反爬蟲機制可能會違反網站的使用條款。在進行爬蟲開發時,請確保遵守相關法律法規和網站的使用政策。