使用scrapy 抓包遇到的问题
1.2.pip更新到指定版本:
python3 -m pip install --user --upgrade pip==9.0.3
2.获取所有自己安装的python版本
$: ls /Library/Frameworks/Python.framework/Versions
$: which python3
$: which python2
/usr/local/bin/python -V
/usr/local/bin/pythonw -V
$: ls /usr/bin/python*
$: ln -s /usr/bin/python3.8 /usr/bin/python
$: whereis python3.8
杂七杂八???
sudo -H pip install scrapy
alias python='/usr/bin/python3'
scrapy startproject ailiaotest
scrapy genspider itcast "itcast.cn"
部分功能 Python 2 不在支持,需升级为python3
Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release
mac下多个版本的python如何删除?
1、删掉第三方安装的2.7版本的·
$: sudo rm -rf /Library/Frameworks/Python.framework/Versions/2.7
2、再删掉directory
$: sudo rm -rf "/Applications/Python 2.7"
3、再删掉链接
$: cd /usr/local/bin/
$: ls -l /usr/local/bin | grep '../Library/Frameworks/Python.framework/Versions/2.7' | awk '{print $9}' | tr -d @ | xargs rm
mac下安装pip
python2.7 安装pip
$: sudo easy_install pip
python3 安装pip
$: curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
升级python3后遇到的问题
一、python2升级 python3 后 scrapy所有命令前加 python3 -m
二、pip命令 都改为pip3
pip的使用
1.安装package
$ pip3 install scrapy
2.列出安装的packages
$: pip3 freeze
3.安装特定版本的package 通过使用==, >=, <=, >, <来指定一个版本号。
$ pip3 install 'scrapy <2.0'
$ pip3 install 'scrapy>2.0,<2.0.3'
4.升级包 升级包到当前最新的版本,可以使用-U 或者 --upgrade
$ pip3 install -U scrapy
5.卸载包
$ pip3 uninstall scrapy
6.查询包
pip3 search "scrapy"
使用命令创建scrapy项目
1.第一步创建工程
python3 -m scrapy startproject example[项目名]
2.创建爬虫
$: cd ailiaouser
spider模版:
$: python3 -m scrapy genspider example[爬虫名] "example.com"[域名]
crawlspider模版:
$: python3 -m scrapy genspider -t crawl spridername "example.com"[域名]
3.编写items、pipelines 、settings 和 爬虫文件
4.执行程序
python3 -m scrapy crawl spidername
Python如何获取当前时间
import time
datatime = time.time()
print("当前时间戳为:" + datatime)
Python 中如何http%3A%2F%2F处理
from urllib.parse import unquote
url = "http%3A%2F%2Fimg3.weilingapp.com%2Fblog%2F202205%2F29%2F20%2F5c05b7fa898e15be5fe5628b4fcf0168.jpg"
str = unquote(url)
print(str)
{{o.name}}
{{m.name}}