BeautifulSoup抓取门户网站上的链接

使用BeautifulSoup抓取门户网站上的所有跳转链接

from bs4 import BeautifulSoup
import urllib2

request = urllib2.Request(http://www.163.com)
response = urllib2.urlopen(request)
html_doc = response.read()

soup = BeautifulSoup(html_doc , from_encoding = "gb18030")

for link in soup.find_all(a):
    print(link.get(href))

from bs4 import BeautifulSoup import urllib2 request = urllib2.Request(http://www.163.com) response = urllib2.urlopen(request) html_doc = response.read() soup = BeautifulSoup(html_doc , from_encoding = "gb18030") for link in soup.find_all(a): print(link.get(href))

打印结果

使用BeautifulSoup抓取门户网站上的所有跳转链接 from bs4 import BeautifulSoup import urllib2 request = urllib2.Request(http://www.163.com) response = urllib2.urlopen(request) html_doc = response.read() soup = BeautifulSoup(html_doc , from_encoding = "gb18030") for link in soup.find_all(a): print(link.get(href)) 打印结果