Python网络爬虫urllib库使用指南
前言🍭
前篇简单介绍了什么是网络爬虫及相关概念，这篇开始讲解爬虫中的第一个库——urllib。
urllib🍭
urllib是Python标准库中的一个模块，提供了一些用于处理URL的功能。
使用urllib可以进行URL的解析、发送HTTP请求、文件下载等操作。
1、urllib的基本使用🍉
使用urIlib来获取百度首页的源码🍓
# 使用urI1ib来获取百度首页的源码 import urllib.request # (1)定义一个urI 赢是你要访问的地址 url = 'http://www.baidu.com/' # (2)模拟浏微器向服务器发达求 response阿应 response = urllib.request.urlopen(url) # (3)获取响应中的页面的源码 (content 内容意思) # read方法 返回的是字节形式的二进制数据 content = response.read() # (4)打印数据 print(content)
打印结果：
b'<!DOCTYPE html><!--STATUS OK--><html><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"><meta content="always" name="referrer"><meta name="theme-color" content="#ffffff"><meta name="description" content="\xe5\x85\xa8\xe7\x90\x83\xe9\xa2\x86\xe5\x85\x88\xe7\x9a\x84\xe4\xb8\xad\xe6\x96\x87\xe6\x90\x9c\xe7\xb4\xa2\xe5\xbc\x95\xe6\x93\x8e\xe3\x80\x81\xe8\x87\xb4\xe5\x8a\x9b\xe4\xba\x8e\xe8\xae\xa9\xe7\xbd\x91\xe6\xb0\x91\xe6\x9b\xb4\xe4\xbe\xbf\xe6\x8d\xb7\xe5\x9c\xb0\xe8\x8e\xb7\xe5\x8f\x96\xe4\xbf\xa1\xe6\x81\xaf\xef\xbc\x8c\xe6\x89\xbe\xe5\x88\xb0\xe6\x89\x80\xe6\xb1\x82\xe3\x80\x82\xe7\x99\xbe\xe5\xba\xa6\xe8\xb6\x85\xe8\xbf\x87\xe5\x8d\x83\xe4\xba\xbf\xe7\x9a\x84\xe4\xb8\xad\xe6\x96\x87\xe7\xbd\x91\xe9\xa1\xb5\xe6\x95\xb0\xe6\x8d\xae\xe5\xba\x93\xef\xbc\x8c\xe5\x8f\xaf\xe4\xbb\xa5\xe7\x9e\xac\xe9\x97\xb4\xe6\x89\xbe\xe5\x88\xb0\xe7\x9b\xb8\xe5\x85\xb3\xe7\x9a\x84\xe6\x90\x9c\xe7\xb4\xa2\xe7\xbb\x93\xe6\x9e\x9c\xe3\x80\x82"><link rel="shortcut icon" href="https://www.baidu.com/favicon.ico" type="image/x-icon" /><link rel="search" type="application/opensearchdescription+xml" href="/content-search.xml" title="\xe7\x99\xbe\xe5\xba\xa6\xe6\x90\x9c\xe7\xb4\xa2" /><link rel="icon" sizes="any" mask href="https://www.baidu.com/favicon.ico"><link rel="dns-prefetch" href="//dss0.bdstatic.com"/><link rel="dns-prefetch" href="//dss1.bdstatic.com"/><link rel="dns-prefetch" href="//ss1.bdstatic.com"/><link rel="dns-prefetch" href="//sp0.baidu.com"/><link rel="dns-prefetch" href="//sp1.baidu.com"/><link rel="dns-prefetch" href="//sp2.baidu.com"/><link rel="dns-prefetch" href="//pss.bdstatic.com"/><link rel="apple-touch-icon-precomposed" href="https://psstatic.cdn.bcebos.com/video/wiseindex/aa6eef91f8b5b1a33b454c401_1660835115000.png"><title>\xe7\x99\xbe\xe5\xba\xa6\xe4\xb8\x80\xe4\xb8\x8b\xef\xbc\x8c\xe4\xbd\xa0\xe5\xb0\xb1\xe7\x9f\xa5\xe9 ......
可以看到开头字母为b，这代表这返回的是字节形式的二进制数据。我们想要看懂它需要将它转换成字符串，也就是解码（二进制->字符串）。
源码：源码
# 使用urI1ib来获取百度首页的源码
import urllib.request
# (1)定义一个urI 赢是你要访问的地址
url = 'http://www.baidu.com'
# (2)模拟浏微器向服务器发达求 response阿应
response = urllib.request.urlopen(url)
# (3)获取响应中的页面的源码 (content 内容意思)
# read方法 返回的是字节形式的二进制数据 我们使用decode进行主动解码，将其转换成字符串
content = response.read().decode("utf-8")
# (4)打印数据
print(content)打印结果：
2、一个类型六个方法🍉
一个类型🍓
为什么要讲一个类型呢？因为后面要讲的一个库叫request库，request库也有相对应的响应，但是它的类型不一样，为了进行一个对比，才进行这样的一个讲解。
import urllib.request url = "http://www.baidu.com" # 模拟浏览器向服务器发送请求 response = urllib.request.urlopen(url) # 一个类型六个方法 print(type(response))
类型打印：
<class 'http.client.HTTPResponse'>
从上面我们可以知道response的数据类型是HttpResponse，我们需要去记住。
六个方法🍓
Ⅰ、read方法
read方法是按照一个字节一个字节的去读
import urllib.request url = "http://www.baidu.com" # 模拟浏览器向服务器发送请求 response = urllib.request.urlopen(url) # 按照一个字节一个字节的去读 content = response.read() print(content)
read方法按字节读
import urllib.request url = "http://www.baidu.com" # 模拟浏览器向服务器发送请求 response = urllib.request.urlopen(url) # read方法按字节读 content = response.read(5) print(content)
打印结果：
b'<!DOC'
Ⅱ、readline方法按行读
import urllib.request url = "http://www.baidu.com" # 模拟浏览器向服务器发送请求 response = urllib.request.urlopen(url) # readline方法按行读 content = response.readline() print(content)
打印结果：
b'<!DOCTYPE html><!--STATUS OK--><html><head><meta http-equiv="Content-Type" content="text/html;charset=utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"><meta content="always" name="referrer"><meta name="theme-color" content="#ffffff"><meta name="description" content="\xe5\x85\xa8\xe7\x90\x83\xe9\xa2\x86\xe5\x85\x88\xe7\x9a\x84\xe4\xb8\xad\xe6\x96\x87\xe6\x90\x9c\xe7\xb4\xa2\xe5\xbc\x95\xe6\x93\x8e\xe3\x80\x81\xe8\x87\xb4\xe5\x8a\x9b\xe4\xba\x8e\xe8\xae\xa9\xe7\xbd\x91\xe6\xb0\x91\xe6\x9b\xb4\xe4\xbe\xbf\xe6\x8d\xb7\xe5\x9c\xb0\xe8\x8e\xb7\xe5\x8f\x96\xe4\xbf\xa1\xe6\x81\xaf\xef\xbc\x8c\xe6\x89\xbe\xe5\x88\xb0\xe6\x89\x80\xe6\xb1\x82\xe3\x80\x82\xe7\x99\xbe\xe5\xba\xa6\xe8\xb6\x85\xe8\xbf\x87\xe5\x8d\x83\xe4\xba\xbf\xe7\x9a\x84\xe4\xb8\xad\xe6\x96\x87\xe7\xbd\x91\xe9\xa1\xb5\xe6\x95\xb0\xe6\x8d\xae\xe5\xba\x93\xef\xbc\x8c\xe5\x8f\xaf\xe4\xbb\xa5\xe7\x9e\xac\xe9\x97\xb4\xe6\x89\xbe\xe5\x88\xb0\xe7\x9b\xb8\xe5\x85\xb3\xe7\x9a\x84\xe6\x90\x9c\xe7\xb4\xa2\xe7\xbb\x93\xe6\x9e\x9c\xe3\x80\x82"><link rel="shortcut icon" href="https://www.baidu.com/favicon.ico" type="image/x-icon" /><link rel="search" type="application/opensearchdescription+xml" href="/content-search.xml" title="\xe7\x99\xbe\xe5\xba\xa6\xe6\x90\x9c\xe7\xb4\xa2" /><link rel="icon" sizes="any" mask href="https://www.baidu.com/favicon.ico"><link rel="dns-prefetch" href="//dss0.bdstatic.com"/><link rel="dns-prefetch" href="//dss1.bdstatic.com"/><link rel="dns-prefetch" href="//ss1.bdstatic.com"/><link rel="dns-prefetch" href="//sp0.baidu.com"/><link rel="dns-prefetch" href="//sp1.baidu.com"/><link rel="dns-prefetch" href="//sp2.baidu.com"/><link rel="dns-prefetch" href="//pss.bdstatic.com"/><link rel="apple-touch-icon-precomposed" href="https://psstatic.cdn.bcebos.com/video/wiseindex/aa6eef91f8b5b1a33b454c401_1660835115000.png"><title>\xe7\x99\xbe\xe5\xba\xa6\xe4\xb8\x80\xe4\xb8\x8b\xef\xbc\x8c\xe4\xbd\xa0\xe5\xb0\xb1\xe7\x9f\xa5\xe9\x81\x93</title><style index="newi" type="text/css">#form .bdsug{top:39px}.bdsug{display:none;position:absolute;width:535px;background:#fff;border:1px solid #ccc!important;_overflow:hidden;box-shadow:1px 1px 3px #ededed;-webkit-box-shadow:1px 1px 3px #ededed;-moz-box-shadow:1px 1px 3px #ededed;-o-box-shadow:1px 1px 3px #ededed}.bdsug li{width:519px;color:#000;font:14px arial;line-height:25px;padding:0 8px;position:relative;cursor:default}.bdsug li.bdsug-s{background:#f0f0f0}.bdsug-store span,.bdsug-store b{color:#7A77C8}.bdsug-store-del{font-size:12px;color:#666;text-decoration:underline;position:absolute;right:8px;top:0;cursor:pointer;display:none}.bdsug-s .bdsug-store-del{display:inline-block}.bdsug-ala{display:inline-block;border-bottom:1px solid #e6e6e6}.bdsug-ala h3{line-height:14px;background:url(//www.baidu.com/img/sug_bd.png?v=09816787.png) no-repeat left center;margin:6px 0 4px;font-size:12px;font-weight:400;color:#7B7B7B;padding-left:20px}.bdsug-ala p{font-size:14px;font-weight:700;padding-left:20px}#m .bdsug .bdsug-direct p{color:#00c;font-weight:700;line-height:34px;padding:0 8px;margin-top:0;cursor:pointer;white-space:nowrap;overflow:hidden}#m .bdsug .bdsug-direct p img{width:16px;height:16px;margin:7px 6px 9px 0;vertical-align:middle}#m .bdsug .bdsug-direct p span{margin-left:8px}#form .bdsug .bdsug-direct{width:auto;padding:0;border-bottom:1px solid #f1f1f1}#form .bdsug .bdsug-direct p i{font-size:12px;line-height:100%;font-style:normal;font-weight:400;color:#fff;background-color:#2b99ff;display:inline;text-align:center;padding:1px 5px;*padding:2px 5px 0;margin-left:8px;overflow:hidden}.bdsug .bdsug-pcDirect{color:#000;font-size:14px;line-height:30px;height:30px;background-color:#f8f8f8}.bdsug .bdsug-pc-direct-tip{position:absolute;right:15px;top:8px;width:55px;height:15px;display:block;background:url(http://pss.bdstatic.com/r/www/cache/static/global/img/pc_direct_42d6311.png) no-repeat 0 0}.bdsug li.bdsug-pcDirect-s{background-color:#f0f0f0}.bdsug .bdsug-pcDirect-is{color:#000;font-size:14px;line-height:22px;background-color:#f5f5f5}.bdsug .bdsug-pc-direct-tip-is{position:absolute;right:15px;top:3px;width:55px;height:15px;display:block;background:url(http://pss.bdstatic.com/r/www/cache/static/global/img/pc_direct_42d6311.png) no-repeat 0 0}.bdsug li.bdsug-pcDirect-is-s{background-color:#f0f0f0}.bdsug .bdsug-pcDirect-s .bdsug-pc-direct-tip,.bdsug .bdsug-pcDirect-is-s .bdsug-pc-direct-tip-is{background-position:0 -15px}.bdsug .bdsug-newicon{color:#929292;opacity:.7;font-size:12px;display:inline-block;line-height:22px;letter-spacing:2px}.bdsug .bdsug-s .bdsug-newicon{opacity:1}.bdsug .bdsug-newicon i{letter-spacing:0;font-style:normal}.bdsug .bdsug-feedback-wrap{display:none}.toggle-underline{text-decoration:none}.toggle-underline:hover{text-decoration:underline}.bdpfmenu,.usermenu{border:1px solid #d1d1d1;position:absolute;width:105px;top:36px;z-index:302;box-shadow:1px 1px 5px #d1d1d1;-webkit-box-shadow:1px 1px 5px #d1d1d1;-moz-box-shadow:1px 1px 5px #d1d1d1;-o-box-shadow:1px 1px 5px #d1d1d1}.bdpfmenu{font-size:12px;background-color:#fff}.bdpfmenu a,.usermenu a{display:block;text-align:left;margin:0!important;padding:0 9px;line-height:26px;text-decoration:none}.briiconsbg{background-repeat:no-repeat;background-size:300px 18px;background-image:url(http://pss.bdstatic.com/r/www/cache/static/home/img/icons_0c37e9b.png);background-image:url(http://pss.bdstatic.com/r/www/cache/static/home/img/icons_809ae65.gif)\\9}.bdpfmenu a:link,.bdpfmenu a:visited,#u .usermenu a:link,#u .usermenu a:visited{background:#fff;color:#333}.bdpfmenu a:hover,.bdpfmenu a:active,#u .usermenu a:hover,#u .usermenu a:active{background:#38f;text-decoration:none;color:#fff}.bdpfmenu{width:70px}#wrapper .bdnuarrow{width:0;height:0;font-size:0;line-height:0;display:block;position:absolute;top:-10px;left:50%;margin-left:-5px}#wrapper .bdnuarrow em,#wrapper .bdnuarrow i{width:0;height:0;font-size:0;line-height:0;display:block;position:absolute;border:5px solid transparent;border-style:dashed dashed solid}#wrapper .bdnuarrow em{border-bottom-color:#d8d8d8;top:-1px}#wrapper .bdnuarrow i{border-bottom-color:#fff;top:0}#gxszHead .prefpanelclose{cursor:pointer;width:16px;height:16px;float:right;margin-top:7px;background-position:-248px 0}#gxszHead .prefpanelclose:hover{background-position:-264px 0}.s_ipt::-webkit-input-placeholder{padding-left:3px;color:#aaa;font-size:13px}.s_ipt::-moz-placeholder{padding-left:3px;color:#aaa;font-size:13px}.s_ipt:-ms-input-placeholder{padding-left:3px;color:#aaa;font-size:13px}.s_ipt::placeholder{padding-left:3px;color:#aaa;font-size:13px}.kw-placeholder{position:absolute;top:0;left:0;color:#aaa;font-size:13px;height:40px;line-height:40px;padding-left:10px;max-width:360px;z-index:99;pointer-events:none}.kw-placeholder.kw-placehlder-high{height:40px;line-height:40px}.kw-placeholder.placeholders-hidden{visibility:hidden}#head_wrapper #form .bdsug-new{width:544px;top:35px;border-radius:0 0 10px 10px;border:2px solid #4E6EF2!important;border-top:0!important;box-shadow:none;font-family:Arial,sans-serif;z-index:1}#head_wrapper.sam_head_wrapper2 #form .bdsug-new{width:545px;z-index:1;border:1px solid #4E6EF2!important;border-top:0!important}#head_wrapper #form .bdsug-new ul{margin:7px 14px 0;padding:8px 0 7px;background:0 0;border-top:2px solid #f5f5f6}#head_wrapper #form .bdsug-new ul li{width:auto;padding-left:14px;margin-left:-14px;margin-right:-14px;color:#626675;line-height:28px;background:0 0;font-family:Arial,sans-serif}#head_wrapper #form .bdsug-new ul li .sug-search-icon,#head_wrapper #form .bdsug-new ul li .sug-history-icon{margin-right:4px;color:#222}#head_wrapper #form .bdsug-new ul li span{color:#626675}#head_wrapper #form .bdsug-new ul li b{font-weight:400;color:#222}#head_wrapper #form .bdsug-new .bdsug-store-del{font-size:13px;text-decoration:none;color:#9195A3;right:16px}#head_wrapper #form .bdsug-new .bdsug-store-del:hover{color:#315EFB;cursor:pointer}#head_wrapper #form .bdsug-new ul li:hover,#head_wrapper #form .bdsug-new ul li:hover span,#head_wrapper #form .bdsug-new ul li:hover b{cursor:pointer}.wrapper_new #form .bdsug-new .bdsug-s{background-color:#F5F5F6!important}.wrapper_new #form .sam_search .bdsug-new .bdsug-s{background-color:#F1F3FD!important}#head_wrapper #form .sam_search .bdsug-new .bdsug-s{background-color:#F1F3FD!important}#head .s-down #form .bdsug-new{top:32px}.s-skin-hasbg #head_wrapper #form .bdsug-new{border-color:#4569ff!important;border-top:0!important}.s-skin-hasbg #head_wrapper.s-down #form .bdsug-new{border-color:#4e6ef2!important;border-top:0!important}.s-skin-hasbg #head_wrapper.s-down #form.sam_search .bdsug-new{border-color:rgba(0,0,0,.05)!important;border-top:1px solid rgba(0,0,0,.05)!important;top:54px!important}#head_wrapper #form .bdsug-new .bdsug-s,#head_wrapper #form .bdsug-new .bdsug-s span,#head_wrapper #form .bdsug-new .bdsug-s b{color:#315EFB}#head_wrapper #form .bdsug-new>div span:hover,#head_wrapper #form .bdsug-new>div a:hover{color:#315EFB!important}#head_wrapper #form #kw.new-ipt-focus{border-color:#4e6ef2}#head_wrapper #form .bdsug-new ul li{}#head_wrapper #form .bdsug-new ul li .sug-hot-orange,#head_wrapper #form .bdsug-new ul li .sug-hot-grey,#head_wrapper #form .bdsug-new ul li .sug-hot-blue{display:inline-block;width:12px;height:12px;font-size:12px;line-height:12px;padding:2px;text-align:center;font-weight:500;margin-left:6px;vertical-align:text-bottom}#head_wrapper #form .bdsug-new ul li .sug-hot-orange{display:inline-block;color:#fff;background:#F60;border-radius:4px}#head_wrapper #form .bdsug-new ul li .sug-new-tag{text-align:center;margin-left:6px;box-sizing:border-box;font-size:12px;line-height:14px;padding:1px 4px;font-weight:500}#head_wrapper #form .bdsug-new ul li .sug-new-tag-grey{color:#858585;border:1px solid rgba(133,133,133,.5);border-radius:4px}#head_wrapper #form .bdsug-new ul li .sug-new-tag-blue{color:#36F;border:1px solid rgba(51,102,255,.4);border-radius:4px}#head_wrapper #form .bdsug-new ul li .sug-new-tag-orange{color:#F33;border:1px solid rgba(255,51,51,.4);border-radius:4px}#head_wrapper #form .bdsug-new ul li .sug-tag-text{display:inline-block;box-sizing:border-box;margin-left:6px;height:18px;padding:2px 4px;font-size:12px;line-height:12px;font-weight:500;text-align:center;border-radius:4px;vertical-align:middle;border-width:1px;border-style:solid}#head_wrapper #form .bdsug-new ul li .sug-tag-img{height:18px;max-width:80px;vertical-align:middle;margin-left:6px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap{display:block;height:40px;padding:7px 0;color:#222}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap:visited{color:#222}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .left-img-wrap{position:relative}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .left-img-wrap::before{content:"";width:38px;height:38px;position:absolute;top:0;right:0;left:0;bottom:0;border:1px solid rgba(0,0,0,.06);border-radius:9px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .left-img{box-sizing:border-box;float:left;width:40px;height:40px;border:1px solid rgba(0,0,0,.06);border-radius:9px;margin-right:6px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .content{float:left}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .header{margin-bottom:6px;line-height:18px;height:18px;vertical-align:middle}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .name{float:left;font-size:18px;color:#222;font-weight:500}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .tag-img{width:18px;margin-left:4px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .more{line-height:14px;width:240px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .brief,#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .info{font-size:14px;color:#222;font-weight:400}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .brief{margin-right:6px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .right-btn{float:right;margin-top:5px;margin-right:6px;width:60px;height:24px;text-align:center;font-size:14px;color:#36F;line-height:24px;background-image:linear-gradient(111deg,#e8f7ff 0,#edf0ff 100%);border-radius:12px}#head_wrapper #form .bdsug-new ul li .direct-sug-wrap .right-btn:hover{background:#315efb;color:#fff}#head_wrapper #form .bdsug-new ul li:hover .direct-sug-wrap .name{color:#315EFB}#head_wrapper #form .bdsug-new ul li:hover .direct-sug-wrap .brief{color:#222}#head_wrapper.s-down #form .sam-bdsug.bdsug-new{top:52px}#head_wrapper #form .sam-bdsug.bdsug-new{width:100%;box-shadow:0 4px 4px 0 rgba(0,0,0,.1);border:1px solid rgba(0,0,0,.05)!important;border-radius:12px;top:56px}#head_wrapper #form .sam-bdsug.bdsug-new ul{border:0;padding:0 0 7px}#head_wrapper #form .sam-bdsug.bdsug-new ul li{line-height:32px}#head_wrapper #form .sam-bdsug.bdsug-new ul .bdsug-s{background-color:#F1F3FD!important}#head_wrapper #form .sam-bdsug.bdsug-new .bdsug-store-del{right:15px}.sam_search .sam_search_rec,.sam_search .sam_search_soutu{z-index:1;display:none;position:absolute;top:50%;margin-top:-12px;font-size:24px;color:#4E6EF2;height:24px;line-height:24px;width:24px;cursor:pointer;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0);transition:transform .3s ease}.sam_search .sam_search_rec{right:54px}.sam_search .sam_search_soutu{right:14px}.sam_search .sam_search_rec:hover,.sam_search .sam_search_soutu:hover{color:#1D4FFF!important;transform:scale(1.08,1.08)}.sam_search .sam_search_rec_hover,.sam_search .sam_search_soutu_hover{background:#626675;border-radius:8px;height:32px;width:76px;text-align:center;line-height:32px;font-size:13px;color:#FFF;position:absolute;z-index:2;top:50px}.sam_search .sam_search_rec_hover:before,.sam_search .sam_search_soutu_hover:before{content:\'\';border:4px solid transparent;border-bottom:4px solid #626675;position:absolute;left:50%;top:-8px;margin-left:-4px}.sam_search .sam_search_rec_hover{right:29px}.sam_search .sam_search_soutu_hover{display:none;right:-12px}</style><style type="text/css" index="superbase">blockquote,body,button,dd,dl,dt,fieldset,form,h1,h2,h3,h4,h5,h6,hr,input,legend,li,ol,p,pre,td,textarea,th,ul{margin:0;padding:0}\n'Ⅲ、readlines按行读直到读完
import urllib.request url = "http://www.baidu.com" # 模拟浏览器向服务器发送请求 response = urllib.request.urlopen(url) # readlines按行读直到读完 content = response.readlines() print(content)
Ⅳ、getcode获取状态码
import urllib.request url = "http://www.baidu.com" # 模拟浏览器向服务器发送请求 response = urllib.request.urlopen(url) # getcode获取状态码 print(response.getcode())
打印结果:
200
常见状态码
在HTTP协议中，常见的状态码用于表示服务器对请求的处理结果。以下是一些常见的状态码及其含义：
200 OK：请求成功，服务器成功处理了请求并返回了所需的数据。
301 Moved Permanently：永久重定向，请求的资源已永久移动到新的URL。
302 Found：临时重定向，请求的资源暂时移动到了新的URL。
400 Bad Request：请求无效，服务器无法理解或处理请求。
401 Unauthorized：未授权，请求需要身份验证。
403 Forbidden：禁止访问，服务器拒绝请求访问资源。
404 Not Found：未找到，服务器无法找到请求的资源。
500 Internal Server Error：服务器内部错误，服务器在处理请求过程中发生了错误。
502 Bad Gateway：错误的网关，作为代理或网关的服务器从上游服务器接收到无效的响应。
503 Service Unavailable：服务不可用，服务器暂时无法处理请求，通常是由于过载或维护。
这只是一些常见的状态码，HTTP协议中还有更多的状态码，每个状态码都有特定的含义和用途。在开发或使用HTTP请求时，了解常见的状态码可以帮助你更好地理解服务器对请求的响应。
Ⅴ、geturl获取url地址
import urllib.request url = "http://www.baidu.com" # 模拟浏览器向服务器发送请求 response = urllib.request.urlopen(url) # getcode获取状态码 print(response.geturl())
打印结果：
http://www.baidu.com
Ⅵ、getheaders方法获取headers（一个状态信息）
import urllib.request url = "http://www.baidu.com" # 模拟浏览器向服务器发送请求 response = urllib.request.urlopen(url) # getcode获取状态码 print(response.getheaders())
打印信息：
[('Content-Length', '410774'), ('Content-Security-Policy', "frame-ancestors 'self' https://chat.baidu.com http://mirror-chat.baidu.com https://fj-chat.baidu.com https://hba-chat.baidu.com https://hbe-chat.baidu.com https://njjs-chat.baidu.com https://nj-chat.baidu.com https://hna-chat.baidu.com https://hnb-chat.baidu.com http://debug.baidu-int.com;"), ('Content-Type', 'text/html; charset=utf-8'), ('Date', 'Sat, 23 Sep 2023 10:36:09 GMT'), ('P3p', 'CP=" OTI DSP COR IVA OUR IND COM "'), ('P3p', 'CP=" OTI DSP COR IVA OUR IND COM "'), ('Server', 'BWS/1.1'), ('Set-Cookie', 'BAIDUID=50E2111E01C6C8C836180B5DFDFF669E:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com'), ('Set-Cookie', 'BIDUPSID=50E2111E01C6C8C836180B5DFDFF669E; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com'), ('Set-Cookie', 'PSTM=1695465369; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com'), ('Set-Cookie', 'BAIDUID=50E2111E01C6C8C85CBF086F53BBCFB9:FG=1; max-age=31536000; expires=Sun, 22-Sep-24 10:36:09 GMT; domain=.baidu.com; path=/; version=1; comment=bd'), ('Traceid', '1695465369023779380213415997353182430819'), ('Vary', 'Accept-Encoding'), ('X-Ua-Compatible', 'IE=Edge,chrome=1'), ('Connection', 'close')]3、下载🍉
我们可以通过爬虫（urllib.request.urlretrieve()）将网页、图片、视频下载下来。
下载网页🍓
下载百度的网页
import urllib.request # 下载网页 url_page = "http://www,.baidu, com" # urI代表的是下载的路径 filename文件的名字# 在python中 可以变量的名字 也可以直接写值 urllib.request.urlretrieve(url_page, 'baidu.html')
下载好了之后浏览器打开：
下载图片🍓
import urllib.request # 下载图片 url_img="https://profile-avatar.yssmx.cn/82351d3cb9754fda97cb85258d7e74a0_m0_63951142.jpg!1" urllib.request.urlretrieve(url_img, filename="tx.jpg")
把我的头像链接输入，下载下来：
下载视频🍓
再找到里面的src
这就是我们的视频地址
import urllib.request # 下载网页 # url_page = "http://www.baidu.com" # # urI代表的是下载的路径 filename文件的名字# 在python中 可以变量的名字 也可以直接写值 # urllib.request.urlretrieve(url_page, 'baidu.html') url_video="https://v16m-default.akamaized.net/5ec5b30ab203de382ab98284de179428/650f2c82/video/tos/alisg/tos-alisg-v-0051c001-sg/oAoAVUpAqIizX8BESwEmjdNsBWAfpFhywEKEVq/?a=0&ch=0&cr=0&dr=0&er=0&lr=default&cd=0%7C0%7C0%7C0&br=2016&bt=1008&bti=cmd5ZGgxZGZiZmIrY2E6&cs=0&ds=3&ft=dl9~j-Inz7TlNPJZiyq8Z&mime_type=video_mp4&qs=13&rc=M3A8ZDQ6ZmhrbDMzODYzNEBpM3A8ZDQ6ZmhrbDMzODYzNEAyXmBycjRnYGhgLS1kMC1zYSMyXmBycjRnYGhgLS1kMC1zcw%3D%3D&l=202309231156594B50CCEF2A58CC50DD84&btag=e00068000https://v16m-default.akamaized.net/5ec5b30ab203de382ab98284de179428/650f2c82/video/tos/alisg/tos-alisg-v-0051c001-sg/oAoAVUpAqIizX8BESwEmjdNsBWAfpFhywEKEVq/?a=0&ch=0&cr=0&dr=0&er=0&lr=default&cd=0%7C0%7C0%7C0&br=2016&bt=1008&bti=cmd5ZGgxZGZiZmIrY2E6&cs=0&ds=3&ft=dl9~j-Inz7TlNPJZiyq8Z&mime_type=video_mp4&qs=13&rc=M3A8ZDQ6ZmhrbDMzODYzNEBpM3A8ZDQ6ZmhrbDMzODYzNEAyXmBycjRnYGhgLS1kMC1zYSMyXmBycjRnYGhgLS1kMC1zcw%3D%3D&l=202309231156594B50CCEF2A58CC50DD84&btag=e00068000" urllib.request.urlretrieve(url_video, "dm.mp4")
运行代码
下载好了之后我们在pycharm是打不开的，得去本地文件中找：
点击就可以看了： 文章来源：https://www.toymoban.com/news/detail-715606.html
文章来源地址https://www.toymoban.com/news/detail-715606.html
到了这里，关于Python网络爬虫库urllib详解的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！
![[Python]爬虫基础——urllib库](https://imgs.yssmx.com/Uploads/2024/02/435181-1.png)
