1、使用requests库访问网址 2、使用xpath技术提取网页目标图片网址 3、通过os库保存图片到本地 #coding = utf-8 import requests import os from lxml import etree from urllib.parse import quote,unquote # import urllib.request class DmdSpider(object): name = "dmd" save_path = 'E:/python/manhuadao/pics/' __picNo = 0 # 访问漫画岛网页 def start_request(self, url): header = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36'} r = requests.get(url, params=header) # print(r.text) self.__get_imgs(r) # 自动跳转下一页 html = etree.HTML(r.text) next_urls = html.xpath('//div[@class="read-bottom"]//a[@class="next"]/@href') # print(next) for href in next_urls: nexturl = href....
原文网址
在python中使用virtualenv创建虚拟环境 注意：要使用cmd命令行，不能使用Power Shell1、使用pip安装 pip install virtualenv
2、创建项目目录并用cd进入 mkdir E:\python\Virtualenv cd E:\python\Virtualenv 3、创建虚拟环境 virtualenv -p c:\Python36\python.exe env
或者
virtualenv env
此命令会在当前目录下创建一个env目录
4、启动虚拟环境 env\Scripts\activate
5、离开虚拟环境 deactivate
6、删除env文件夹即可
原文网址
1. Using a virtual environment (recommended) pip install virtualenv
详细操作
2. Creating a new Scrapy project scrapy startproject tutorial
3. Writing a spider to crawl a site and extract data This is the code for our first Spider. Save it in a file named quotes_spider.py under the tutorial/spiders directory in your project:
import scrapy class QuotesSpider(scrapy.Spider): name = "quotes" def start_requests(self): urls = [ 'http://quotes.toscrape.com/page/1/', 'http://quotes.toscrape.com/page/2/', ] for url in urls: yield scrapy....
1. 配置npm镜像源 1.1 临时使用 npm --registry https://registry.npm.taobao.org install --save-dev electron
1.2 持久使用 npm config set registry https://registry.npm.taobao.org
1.3 配置后可通过下面方式来验证是否成功 npm config get registry 或 npm info electron 1.4 通过cnpm使用 npm install -g cnpm --registry=https://registry.npm.taobao.org
1.5 使用 cnpm install express
1. Use npm run example 1.1 Clone repository git clone https://github.com/electron/electron-quick-start
1.2 Go into the repository cd electron-quick-start
1.3 Install dependencies PS:为了加快下载依赖模块的速度，通过修改镜像源ELECTRON_MIRROR=https://npm.taobao.org/mirrors/electron/ npm install
或者是
npm install --registry=https://registry.npm.taobao.org
1.4 Run the app npm start
2. Use yarn create a electron program 2.1 Create a program folder md electron-yarn-cz
进入项目文件夹
cd electron-yarn-cz
2.2 Create a program config file yarn init
2.3 Instal electron environment yarn add electron --dev --registry=https://registry.npm.taobao.org
2.4 Add main....