前言

一直以来使用Selenium绕过Cloudflare请求Coinbase做上币监控,但是Selenium非常吃内存,2核4G的机子非常吃力,又在网上找其他的替代方式,发现还可以使用"FlareSolverr"绕过CloudFlare进行网页抓取。

搭建FlareSolverr

FlareSolverr需要单独搭建,使用docker容器可以一键部署 docker run -d \ --name=flaresolverr \ -p 8191:8191 \ -e LOG_LEVEL=info \ --restart unless-stopped \ ghcr.io/flaresolverr/flaresolverr:latest 搭建完成后,在浏览器输入http://ip:8191 (或设置反向代理后使用https://反代域名:8191)

浏览器显示:

{"msg":"FlareSolverr is ready!","version":"v2.2.4","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0"}

就表示FlareSolverr已经搭建成功了。

##使用方式

  1. Example Bash request:
curl -L -X POST 'http://localhost:8191/v1' \
-H 'Content-Type: application/json' \
--data-raw '{
  "cmd": "request.get",
  "url":"http://www.google.com/",
  "maxTimeout": 60000
}'
  1. Example Python request:
import requests
    
    url = "http://localhost:8191/v1"
    headers = {"Content-Type": "application/json"}
    data = {
        "cmd": "request.get",
        "url": "http://www.google.com/",
        "maxTimeout": 60000
    }
    response = requests.post(url, headers=headers, json=data)
    print(response.text)