Compare commits

...

52 Commits

Author SHA1 Message Date
ilike2burnthing
63b6fc53e3 Bump version 3.4.4 (#1609) 2025-11-04 23:04:55 +00:00
flower
8d72617219 Bump dependencies, Chrome, and some other general fixes (#1607)
Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>
2025-11-04 22:49:32 +00:00
ilike2burnthing
8a8b9415c3 Bump version 3.4.3 (#1601) 2025-10-28 10:21:38 +00:00
ilike2burnthing
16722ef963 Update proxy extension. Fixes #1534 2025-10-28 00:01:30 +00:00
ilike2burnthing
bbc24e9d86 Bump version 3.4.2 (#1590) 2025-10-09 20:05:32 +01:00
acg5159
7dfdfc5e33 Add log file support (#1480)
Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>
2025-10-09 19:55:32 +01:00
Esteban Thilliez
136422c85c Add returnScreenshot parameter to screenshot the final web page (#1439) 2025-10-08 10:59:39 +01:00
flower
05a72f2709 bump: dependencies (#1585) 2025-10-05 16:25:13 +01:00
flower
da810830da Bump prometheus-client to 0.23.1 (#1583) 2025-10-02 12:32:01 +01:00
Warrenberberd
d27f57c27c Add quote protection for password containing it (#858)
Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>
2025-10-01 06:46:08 +01:00
eZ4RK0
a916d93779 Handle empty string and keys without value in postData. resolves #1548 (#1550) 2025-10-01 04:56:57 +01:00
Kishan Joshi
0d889cb0b2 Add proxy envs (#1499)
Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>
2025-09-20 13:43:58 +01:00
Kennedy Oliveira
d430404de8 Add optional wait time after resolving the challenge before returning (#1046)
Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>
2025-09-20 04:59:16 +01:00
flower
d3b1ba6e88 Bump dependencies & CI actions (#1578) 2025-09-18 21:20:15 +01:00
ilike2burnthing
75e5b190d6 Bump version 3.4.1 (#1576) 2025-09-15 19:01:04 +01:00
ilike2burnthing
cdc3db3c21 Change access denied title check to use startswith. resolves #1574 2025-09-15 18:55:40 +01:00
ilike2burnthing
2dbb0442e0 Fix regex pattern syntax in utils.py 2025-08-25 04:50:05 +01:00
ilike2burnthing
6faab19533 Bump version 3.4.0 (#1564) 2025-08-25 04:21:44 +01:00
ilike2burnthing
af0a7af757 Remove disable software rasterizer option for ARM builds 2025-08-25 04:17:22 +01:00
Alex Naidis
ff74b50b60 Modernize and upgrade application (#1540)
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
2025-08-25 03:55:06 +01:00
ilike2burnthing
3e51ac1188 Update README.md 2025-06-26 05:30:44 +01:00
ilike2burnthing
6627de4fa6 Bump version 3.3.25 (#1523) 2025-06-14 03:52:28 +01:00
ilike2burnthing
fe649255f2 Revert "Fix Chrome GL erros in ASUSTOR NAS"
This reverts commit 8316350b98.
2025-06-14 03:42:08 +01:00
dependabot[bot]
3e338fce2e Bump requests from 2.32.3 to 2.32.4 (#1516)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-11 02:48:00 +01:00
ilike2burnthing
3dd3e7559d u_c: remove apparent c&p typo
https://github.com/ultrafunkamsterdam/undetected-chromedriver/pull/1933
2025-06-06 08:04:00 +01:00
ilike2burnthing
f21c1d51bc Restore example service file. #1204 2025-06-04 23:13:00 +01:00
ilike2burnthing
957347f73a Bump version 3.3.24 (#1505) 2025-06-04 19:02:06 +01:00
ilike2burnthing
c55080b0ec Remove hidden character 2025-06-04 18:54:48 +01:00
ilike2burnthing
639bfca020 Bump version 3.3.23 (#1504) 2025-06-04 18:51:31 +01:00
ilike2burnthing
237694df76 Update base image to bookworm. resolves #1503 2025-06-04 18:44:17 +01:00
ilike2burnthing
6e5d6f1795 Bump version 3.3.22 (#1500) 2025-06-03 05:54:25 +01:00
ilike2burnthing
30804a86e5 Bump Chromium to v137 for build 2025-06-03 05:39:46 +01:00
ilike2burnthing
e0bdaf7745 Don't open devtools 2025-06-03 05:38:55 +01:00
ilike2burnthing
795365dbe4 Change from click to keys
credit to @sh4dowb - #1497
2025-06-03 05:38:06 +01:00
ilike2burnthing
ce5369dd41 Update bug_report.yml
fix accidental delete
2025-03-04 02:05:23 +00:00
ilike2burnthing
600b09d498 Update bug_report.yml 2025-03-04 02:04:29 +00:00
ilike2burnthing
d1f19405a1 Update README.md. closes #1267 2025-01-21 20:27:36 +00:00
ilike2burnthing
82a1366d34 Remove dead directory link from readme. resolve #1436 2025-01-20 17:37:14 +00:00
dependabot[bot]
a2fe9e7776 Bump waitress from 2.1.2 to 3.0.1 (#1418)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-29 01:53:34 +00:00
ilike2burnthing
6cc628df9e Disable search engine choice screen 2024-11-24 18:31:50 +00:00
Eduard Tykhoniuk
8b1851eeb1 Fix headless=false stalling 2024-11-24 18:30:35 +00:00
ilike2burnthing
54668a11e7 bug_report: lint fix again 2024-09-28 08:08:49 +01:00
ilike2burnthing
701d8fb4ff bug_report: change to input 2024-09-28 08:07:56 +01:00
ilike2burnthing
39a265ccb8 bug_report: lint fix 2024-09-28 07:58:14 +01:00
ilike2burnthing
e32b247014 bug_report: default=0 2024-09-28 07:57:11 +01:00
ilike2burnthing
0d8fe8fe50 bug_report: no booleans allowed 2024-09-28 07:51:57 +01:00
ilike2burnthing
718da3a36f bug_report: add 'no really' drop down
maybe this will help... 🙄
2024-09-28 07:50:11 +01:00
ilike2burnthing
a798561338 Bump requests version
*.0 was yanked
2024-07-30 02:38:13 +01:00
Bogdan
eb680efc90 Don't build docker images for PRs from forks (#1281) 2024-07-20 22:08:40 +03:00
ilike2burnthing
0f8f0bec25 revert and bump action version 2024-07-20 19:41:49 +01:00
ilike2burnthing
3d9bc5627b Change to GITHUB_TOKEN for GHRC login 2024-07-20 14:21:34 +01:00
ilike2burnthing
dd7eaee2e3 Bump requirements
resolves Dependabot alerts
2024-07-12 17:11:40 +01:00
23 changed files with 256 additions and 179 deletions

View File

@@ -29,6 +29,13 @@ body:
options:
- label: I have read the Discussions
required: true
- type: input
attributes:
label: Have you ACTUALLY checked all these?
description: Please do not waste our time and yours; these checks are there for a reason, it is not just so you can tick boxes for fun. If you type <b>YES</b> and it is clear you did not or have put in no effort, your issue will be closed and locked without comment. If you type <b>NO</b> but still open this issue, you will be permanently blocked for timewasting.
placeholder: YES or NO
validations:
required: true
- type: textarea
attributes:
label: Environment

View File

@@ -1,4 +1,4 @@
name: autotag
name: Autotag
on:
push:
@@ -9,11 +9,10 @@ jobs:
tag-release:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Auto Tag
- name: Checkout repository
uses: actions/checkout@v5
- name: Auto Tag
uses: Klemensas/action-autotag@stable
with:
GITHUB_TOKEN: "${{ secrets.GH_PAT }}"

View File

@@ -1,9 +1,9 @@
name: release-docker
name: Docker release
on:
push:
tags:
- 'v*.*.*'
- "v*.*.*"
pull_request:
branches:
- master
@@ -14,10 +14,11 @@ concurrency:
jobs:
build-docker-images:
runs-on: ubuntu-22.04
if: ${{ !github.event.pull_request.head.repo.fork }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v5
- name: Downcase repo
run: echo REPOSITORY=$(echo ${{ github.repository }} | tr '[:upper:]' '[:lower:]') >> $GITHUB_ENV
@@ -42,8 +43,8 @@ jobs:
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
@@ -56,7 +57,7 @@ jobs:
password: ${{ secrets.GH_PAT }}
- name: Build and push
uses: docker/build-push-action@v5
uses: docker/build-push-action@v6
with:
context: .
file: ./Dockerfile

View File

@@ -1,19 +1,19 @@
name: release
name: Release
on:
push:
tags:
- 'v*.*.*'
- "v*.*.*"
jobs:
create-release:
name: Create release
runs-on: ubuntu-22.04
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Checkout repository
uses: actions/checkout@v5
with:
fetch-depth: 0 # get all commits, branches and tags (required for the changelog)
fetch-depth: 0
- name: Build changelog
id: github_changelog
@@ -22,74 +22,43 @@ jobs:
changelog="${changelog//'%'/'%25'}"
changelog="${changelog//$'\n'/'%0A'}"
changelog="${changelog//$'\r'/'%0D'}"
echo "##[set-output name=changelog;]${changelog}"
echo "changelog=${changelog}" >> $GITHUB_ENV
- name: Create release
id: create_release
uses: actions/create-release@v1
uses: softprops/action-gh-release@v2
with:
body: ${{ env.changelog }}
env:
GITHUB_TOKEN: ${{ secrets.GH_PAT }}
with:
tag_name: ${{ github.ref }}
release_name: ${{ github.ref }}
body: ${{ steps.github_changelog.outputs.changelog }}
draft: false
prerelease: false
build-linux-package:
name: Build Linux binary
build-package:
name: Build binaries
needs: create-release
runs-on: ubuntu-22.04
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Checkout repository
uses: actions/checkout@v5
with:
fetch-depth: 0 # get all commits, branches and tags (required for the changelog)
fetch-depth: 0
- name: Setup Python
uses: actions/setup-python@v4
uses: actions/setup-python@v6
with:
python-version: '3.11'
python-version: "3.14"
- name: Build artifacts
run: |
python -m pip install -r requirements.txt
python -m pip install pyinstaller==5.13.0
python -m pip install pyinstaller==6.16.0
cd src
python build_package.py
- name: Upload release artifacts
uses: alexellis/upload-assets@0.4.0
uses: softprops/action-gh-release@v2
with:
files: ./dist/flaresolverr_*
env:
GITHUB_TOKEN: ${{ secrets.GH_PAT }}
with:
asset_paths: '["./dist/flaresolverr_*"]'
build-windows-package:
name: Build Windows binary
needs: create-release
runs-on: windows-2022
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0 # get all commits, branches and tags (required for the changelog)
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Build artifacts
run: |
python -m pip install -r requirements.txt
python -m pip install pyinstaller==5.13.0
cd src
python build_package.py
- name: Upload release artifacts
uses: alexellis/upload-assets@0.4.0
env:
GITHUB_TOKEN: ${{ secrets.GH_PAT }}
with:
asset_paths: '["./dist/flaresolverr_*"]'

View File

@@ -1,5 +1,47 @@
# Changelog
## v3.4.4 (2025/11/04)
* Bump dependencies, Chromium, and some other general fixes. Thanks @flowerey
## v3.4.3 (2025/10/28)
* Update proxy extension
## v3.4.2 (2025/10/09)
* Bump dependencies & CI actions. Thanks @flowerey
* Add optional wait time after resolving the challenge before returning. Thanks @kennedyoliveira
* Add proxy ENVs. Thanks @Robokishan
* Handle empty string and keys without value in postData. Thanks @eZ4RK0
* Add quote protection for password containing it. Thanks @warrenberberd
* Add returnScreenshot parameter to screenshot the final web page. Thanks @estebanthi
* Add log file support. Thanks @acg5159
## v3.4.1 (2025/09/15)
* Fix regex pattern syntax in utils.py
* Change access denied title check to use startswith
## v3.4.0 (2025/08/25)
* Modernize and upgrade application. Thanks @TheCrazyLex
* Remove disable software rasterizer option for ARM builds. Thanks @smrodman83
## v3.3.25 (2025/06/14)
* Remove `use-gl` argument. Thanks @qwerty12
* u_c: remove apparent c&p typo. Thanks @ok3721
* Bump requirements
## v3.3.24 (2025/06/04)
* Remove hidden character
## v3.3.23 (2025/06/04)
* Update base image to bookworm. Thanks @rwjack
## v3.3.22 (2025/06/03)
* Disable search engine choice screen
* Fix headless=false stalling. Thanks @MAKMED1337
* Change from click to keys. Thanks @sh4dowb
* Don't open devtools
* Bump Chromium to v137 for build
* Bump requirements
## v3.3.21 (2024/06/26)
* Add challenge selector to catch reloading page on non-English systems
* Escape values for generated form used in request.post. Thanks @mynameisbogdan

View File

@@ -1,4 +1,4 @@
FROM python:3.11-slim-bullseye as builder
FROM python:3.14-slim-bookworm as builder
# Build dummy packages to skip installing them and their dependencies
RUN apt-get update \
@@ -12,7 +12,7 @@ RUN apt-get update \
&& equivs-build adwaita-icon-theme \
&& mv adwaita-icon-theme_*.deb /adwaita-icon-theme.deb
FROM python:3.11-slim-bullseye
FROM python:3.14-slim-bookworm
# Copy dummy packages
COPY --from=builder /*.deb /
@@ -38,7 +38,12 @@ RUN dpkg -i /libgl1-mesa-dri.deb \
# Create flaresolverr user
&& useradd --home-dir /app --shell /bin/sh flaresolverr \
&& mv /usr/bin/chromedriver chromedriver \
&& chown -R flaresolverr:flaresolverr .
&& chown -R flaresolverr:flaresolverr . \
# Create config dir
&& mkdir /config \
&& chown flaresolverr:flaresolverr /config
VOLUME /config
# Install Python dependencies
COPY requirements.txt .
@@ -62,17 +67,17 @@ ENTRYPOINT ["/usr/bin/dumb-init", "--"]
CMD ["/usr/local/bin/python", "-u", "/app/flaresolverr.py"]
# Local build
# docker build -t ngosang/flaresolverr:3.3.21 .
# docker run -p 8191:8191 ngosang/flaresolverr:3.3.21
# docker build -t ngosang/flaresolverr:3.4.4 .
# docker run -p 8191:8191 ngosang/flaresolverr:3.4.4
# Multi-arch build
# docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
# docker buildx create --use
# docker buildx build -t ngosang/flaresolverr:3.3.21 --platform linux/386,linux/amd64,linux/arm/v7,linux/arm64/v8 .
# docker buildx build -t ngosang/flaresolverr:3.4.4 --platform linux/386,linux/amd64,linux/arm/v7,linux/arm64/v8 .
# add --push to publish in DockerHub
# Test multi-arch build
# docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
# docker buildx create --use
# docker buildx build -t ngosang/flaresolverr:3.3.21 --platform linux/arm/v7 --load .
# docker run -p 8191:8191 --platform linux/arm/v7 ngosang/flaresolverr:3.3.21
# docker buildx build -t ngosang/flaresolverr:3.4.4 --platform linux/arm/v7 --load .
# docker run -p 8191:8191 --platform linux/arm/v7 ngosang/flaresolverr:3.4.4

View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2023 Diego Heras (ngosang / ngosang@hotmail.es)
Copyright (c) 2025 Diego Heras (ngosang / ngosang@hotmail.es)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@@ -77,7 +77,7 @@ This is the recommended way for Windows users.
> **Warning**
> Installing from source code only works for x64 architecture. For other architectures see Docker images.
* Install [Python 3.11](https://www.python.org/downloads/).
* Install [Python 3.14](https://www.python.org/downloads/).
* Install [Chrome](https://www.google.com/intl/en_us/chrome/) (all OS) or [Chromium](https://www.chromium.org/getting-involved/download-chromium/) (just Linux, it doesn't work in Windows) web browser.
* (Only in Linux) Install [Xvfb](https://en.wikipedia.org/wiki/Xvfb) package.
* (Only in macOS) Install [XQuartz](https://www.xquartz.org/) package.
@@ -87,10 +87,10 @@ This is the recommended way for Windows users.
### From source code (FreeBSD/TrueNAS CORE)
* Run `pkg install chromium python39 py39-pip xorg-vfbserver` command to install the required dependencies.
* Run `pkg install chromium python313 py313-pip xorg-vfbserver` command to install the required dependencies.
* Clone this repository and open a shell in that path.
* Run `python3.9 -m pip install -r requirements.txt` command to install FlareSolverr dependencies.
* Run `python3.9 src/flaresolverr.py` command to start FlareSolverr.
* Run `python3.14 -m pip install -r requirements.txt` command to install FlareSolverr dependencies.
* Run `python3.14 src/flaresolverr.py` command to start FlareSolverr.
### Systemd service
@@ -187,7 +187,9 @@ session. When you no longer need to use a session you should make sure to close
| maxTimeout | Optional, default value 60000. Max timeout to solve the challenge in milliseconds. |
| cookies | Optional. Will be used by the headless browser. Eg: `"cookies": [{"name": "cookie1", "value": "value1"}, {"name": "cookie2", "value": "value2"}]`. |
| returnOnlyCookies | Optional, default false. Only returns the cookies. Response data, headers and other parts of the response are removed. |
| returnScreenshot | Optional, default false. Captures a screenshot of the final rendered page after all challenges and waits are completed. The screenshot is returned as a Base64-encoded PNG string in the `screenshot` field of the response. |
| proxy | Optional, default disabled. Eg: `"proxy": {"url": "http://127.0.0.1:8888"}`. You must include the proxy schema in the URL: `http://`, `socks4://` or `socks5://`. Authorization (username/password) is not supported. (When the `session` parameter is set, the proxy is ignored; a session specific proxy can be set in `sessions.create`.) |
| waitInSeconds | Optional, default none. Length to wait in seconds after solving the challenge, and before returning the results. Useful to allow it to load dynamic content. |
> **Warning**
> If you want to use Cloudflare clearance cookie in your scripts, make sure you use the FlareSolverr User-Agent too. If they don't match you will see the challenge.
@@ -264,12 +266,15 @@ This is the same as `request.get` but it takes one more param:
| Name | Default | Notes |
|--------------------|------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|
| LOG_LEVEL | info | Verbosity of the logging. Use `LOG_LEVEL=debug` for more information. |
| LOG_FILE | none | Path to capture log to file. Example: `/config/flaresolver.log`. |
| LOG_HTML | false | Only for debugging. If `true` all HTML that passes through the proxy will be logged to the console in `debug` level. |
| PROXY_URL | none | URL for proxy. Will be overwritten by `request` or `sessions` proxy, if used. Example: `http://127.0.0.1:8080`. |
| PROXY_USERNAME | none | Username for proxy. Will be overwritten by `request` or `sessions` proxy, if used. Example: `testuser`. |
| PROXY_PASSWORD | none | Password for proxy. Will be overwritten by `request` or `sessions` proxy, if used. Example: `testpass`. |
| CAPTCHA_SOLVER | none | Captcha solving method. It is used when a captcha is encountered. See the Captcha Solvers section. |
| TZ | UTC | Timezone used in the logs and the web browser. Example: `TZ=Europe/London`. |
| LANG | none | Language used in the web browser. Example: `LANG=en_GB`. |
| HEADLESS | true | Only for debugging. To run the web browser in headless mode or visible. |
| BROWSER_TIMEOUT | 40000 | If you are experiencing errors/timeouts because your system is slow, you can try to increase this value. Remember to increase the `maxTimeout` parameter too. |
| TEST_URL | https://www.google.com | FlareSolverr makes a request on start to make sure the web browser is working. You can change that URL if it is blocked in your country. |
| PORT | 8191 | Listening port. You don't need to change this if you are running on Docker. |
| HOST | 0.0.0.0 | Listening interface. You don't need to change this if you are running on Docker. |
@@ -317,8 +322,10 @@ solve a captcha.
If this is the case, FlareSolverr will return the error `Captcha detected but no automatic solver is configured.`
FlareSolverr can be customized to solve the CAPTCHA automatically by setting the environment variable `CAPTCHA_SOLVER`
to the file name of one of the adapters inside the [/captcha](src/captcha) directory.
to the file name of one of the adapters inside the `/captcha` directory.
## Related projects
* C# implementation => https://github.com/FlareSolverr/FlareSolverrSharp

View File

@@ -7,9 +7,12 @@ services:
container_name: flaresolverr
environment:
- LOG_LEVEL=${LOG_LEVEL:-info}
- LOG_FILE=${LOG_FILE:-none}
- LOG_HTML=${LOG_HTML:-false}
- CAPTCHA_SOLVER=${CAPTCHA_SOLVER:-none}
- TZ=Europe/London
ports:
- "${PORT:-8191}:8191"
volumes:
- /var/lib/flaresolver:/config
restart: unless-stopped

19
flaresolverr.service Normal file
View File

@@ -0,0 +1,19 @@
[Unit]
Description=FlareSolverr
After=network.target
[Service]
SyslogIdentifier=flaresolverr
Restart=always
RestartSec=5
Type=simple
User=flaresolverr
Group=flaresolverr
Environment="LOG_LEVEL=info"
Environment="CAPTCHA_SOLVER=none"
WorkingDirectory=/opt/flaresolverr
ExecStart=/opt/flaresolverr/flaresolverr
TimeoutStopSec=30
[Install]
WantedBy=multi-user.target

View File

@@ -1,6 +1,6 @@
{
"name": "flaresolverr",
"version": "3.3.21",
"version": "3.4.4",
"description": "Proxy server to bypass Cloudflare protection",
"author": "Diego Heras (ngosang / ngosang@hotmail.es)",
"license": "MIT"

View File

@@ -1,13 +1,14 @@
bottle==0.12.25
waitress==2.1.2
selenium==4.15.2
bottle==0.13.4
waitress==3.0.2
selenium==4.38.0
func-timeout==4.3.5
prometheus-client==0.17.1
# required by undetected_chromedriver
requests==2.31.0
certifi==2023.7.22
websockets==11.0.3
# only required for linux and macos
xvfbwrapper==0.2.9; platform_system != "Windows"
# only required for windows
pefile==2023.2.7; platform_system == "Windows"
prometheus-client==0.23.1
# Required by undetected_chromedriver
requests==2.32.5
certifi==2025.10.5
websockets==15.0.1
packaging==25.0
# Only required for Linux and macOS
xvfbwrapper==0.2.15; platform_system != "Windows"
# Only required for Windows
pefile==2024.8.26; platform_system == "Windows"

View File

@@ -5,7 +5,7 @@ import logging
def logger_plugin(callback):
"""
Bottle plugin to use logging module
http://bottlepy.org/docs/dev/plugindev.html
https://bottlepy.org/docs/dev/plugindev.html
Wrap a Bottle request so that a log line is emitted after it's handled.
(This decorator can be extended to take the desired logger as a param.)

View File

@@ -18,7 +18,7 @@ def setup():
def prometheus_plugin(callback):
"""
Bottle plugin to expose Prometheus metrics
http://bottlepy.org/docs/dev/plugindev.html
https://bottlepy.org/docs/dev/plugindev.html
"""
def wrapper(*args, **kwargs):
actual_response = callback(*args, **kwargs)

View File

@@ -25,7 +25,7 @@ def clean_files():
def download_chromium():
# https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Linux_x64/
revision = "1260008" if os.name == 'nt' else '1260015'
revision = "1522586" if os.name == 'nt' else '1522586'
arch = 'Win_x64' if os.name == 'nt' else 'Linux_x64'
dl_file = 'chrome-win' if os.name == 'nt' else 'chrome-linux'
dl_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir, 'dist_chrome')

View File

@@ -10,6 +10,7 @@ class ChallengeResolutionResultT:
response: str = None
cookies: list = None
userAgent: str = None
screenshot: str | None = None
def __init__(self, _dict):
self.__dict__.update(_dict)
@@ -41,8 +42,10 @@ class V1RequestBase(object):
url: str = None
postData: str = None
returnOnlyCookies: bool = None
returnScreenshot: bool = None
download: bool = None # deprecated v2.0.0, not used
returnRawHtml: bool = None # deprecated v2.0.0, not used
waitInSeconds: int = None
def __init__(self, _dict):
self.__dict__.update(_dict)

View File

@@ -13,6 +13,10 @@ from dtos import V1RequestBase
import flaresolverr_service
import utils
env_proxy_url = os.environ.get('PROXY_URL', None)
env_proxy_username = os.environ.get('PROXY_USERNAME', None)
env_proxy_password = os.environ.get('PROXY_PASSWORD', None)
class JSONErrorBottle(Bottle):
"""
@@ -50,7 +54,14 @@ def controller_v1():
"""
Controller v1
"""
req = V1RequestBase(request.json)
data = request.json or {}
if (('proxy' not in data or not data.get('proxy')) and env_proxy_url is not None and (env_proxy_username is None and env_proxy_password is None)):
logging.info('Using proxy URL ENV')
data['proxy'] = {"url": env_proxy_url}
if (('proxy' not in data or not data.get('proxy')) and env_proxy_url is not None and (env_proxy_username is not None or env_proxy_password is not None)):
logging.info('Using proxy URL, username & password ENVs')
data['proxy'] = {"url": env_proxy_url, "username": env_proxy_username, "password": env_proxy_password}
req = V1RequestBase(data)
res = flaresolverr_service.controller_v1_endpoint(req)
if res.__error_500__:
response.status = 500
@@ -70,12 +81,13 @@ if __name__ == "__main__":
# fix ssl certificates for compiled binaries
# https://github.com/pyinstaller/pyinstaller/issues/7229
# https://stackoverflow.com/questions/55736855/how-to-change-the-cafile-argument-in-the-ssl-module-in-python3
# https://stackoverflow.com/q/55736855
os.environ["REQUESTS_CA_BUNDLE"] = certifi.where()
os.environ["SSL_CERT_FILE"] = certifi.where()
# validate configuration
log_level = os.environ.get('LOG_LEVEL', 'info').upper()
log_file = os.environ.get('LOG_FILE', None)
log_html = utils.get_config_log_html()
headless = utils.get_config_headless()
server_host = os.environ.get('HOST', '0.0.0.0')
@@ -93,6 +105,13 @@ if __name__ == "__main__":
logging.StreamHandler(sys.stdout)
]
)
if log_file:
log_file = os.path.realpath(log_file)
log_path = os.path.dirname(log_file)
os.makedirs(log_path, exist_ok=True)
logging.getLogger().addHandler(logging.FileHandler(log_file))
# disable warning traces from urllib3
logging.getLogger('urllib3').setLevel(logging.ERROR)
logging.getLogger('selenium.webdriver.remote.remote_connection').setLevel(logging.WARNING)

View File

@@ -10,6 +10,7 @@ from func_timeout import FunctionTimedOut, func_timeout
from selenium.common import TimeoutException
from selenium.webdriver.chrome.webdriver import WebDriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.expected_conditions import (
presence_of_element_located, staleness_of, title_is)
from selenium.webdriver.common.action_chains import ActionChains
@@ -255,17 +256,8 @@ def _resolve_challenge(req: V1RequestBase, method: str) -> ChallengeResolutionT:
def click_verify(driver: WebDriver):
try:
logging.debug("Try to find the Cloudflare verify checkbox...")
iframe = driver.find_element(By.XPATH, "//iframe[starts-with(@id, 'cf-chl-widget-')]")
driver.switch_to.frame(iframe)
checkbox = driver.find_element(
by=By.XPATH,
value='//*[@id="content"]/div/div/label/input',
)
if checkbox:
actions = ActionChains(driver)
actions.move_to_element_with_offset(checkbox, 5, 7)
actions.click(checkbox)
actions.perform()
actions.pause(5).send_keys(Keys.TAB).pause(1).send_keys(Keys.SPACE).perform()
logging.debug("Cloudflare verify checkbox found and clicked!")
except Exception:
logging.debug("Cloudflare verify checkbox not found on the page.")
@@ -290,22 +282,6 @@ def click_verify(driver: WebDriver):
time.sleep(2)
def get_correct_window(driver: WebDriver) -> WebDriver:
if len(driver.window_handles) > 1:
for window_handle in driver.window_handles:
driver.switch_to.window(window_handle)
current_url = driver.current_url
if not current_url.startswith("devtools://devtools"):
return driver
return driver
def access_page(driver: WebDriver, url: str) -> None:
driver.get(url)
driver.start_session()
driver.start_session() # required to bypass Cloudflare
def _evil_logic(req: V1RequestBase, driver: WebDriver, method: str) -> ChallengeResolutionT:
res = ChallengeResolutionT({})
res.status = STATUS_OK
@@ -317,8 +293,7 @@ def _evil_logic(req: V1RequestBase, driver: WebDriver, method: str) -> Challenge
if method == 'POST':
_post_request(req, driver)
else:
access_page(driver, req.url)
driver = get_correct_window(driver)
driver.get(req.url)
# set cookies if required
if req.cookies is not None and len(req.cookies) > 0:
@@ -330,8 +305,7 @@ def _evil_logic(req: V1RequestBase, driver: WebDriver, method: str) -> Challenge
if method == 'POST':
_post_request(req, driver)
else:
access_page(driver, req.url)
driver = get_correct_window(driver)
driver.get(req.url)
# wait for the page
if utils.get_config_log_html():
@@ -341,7 +315,7 @@ def _evil_logic(req: V1RequestBase, driver: WebDriver, method: str) -> Challenge
# find access denied titles
for title in ACCESS_DENIED_TITLES:
if title == page_title:
if page_title.startswith(title):
raise Exception('Cloudflare has blocked this request. '
'Probably your IP is banned for this site, check in your web browser.')
# find access denied selectors
@@ -416,18 +390,26 @@ def _evil_logic(req: V1RequestBase, driver: WebDriver, method: str) -> Challenge
if not req.returnOnlyCookies:
challenge_res.headers = {} # todo: fix, selenium not provides this info
if req.waitInSeconds and req.waitInSeconds > 0:
logging.info("Waiting " + str(req.waitInSeconds) + " seconds before returning the response...")
time.sleep(req.waitInSeconds)
challenge_res.response = driver.page_source
if req.returnScreenshot:
challenge_res.screenshot = driver.get_screenshot_as_base64()
res.result = challenge_res
return res
def _post_request(req: V1RequestBase, driver: WebDriver):
post_form = f'<form id="hackForm" action="{req.url}" method="POST">'
query_string = req.postData if req.postData[0] != '?' else req.postData[1:]
query_string = req.postData if req.postData and req.postData[0] != '?' else req.postData[1:] if req.postData else ''
pairs = query_string.split('&')
for pair in pairs:
parts = pair.split('=')
parts = pair.split('=', 1)
# noinspection PyBroadException
try:
name = unquote(parts[0])
@@ -437,9 +419,11 @@ def _post_request(req: V1RequestBase, driver: WebDriver):
continue
# noinspection PyBroadException
try:
value = unquote(parts[1])
value = unquote(parts[1]) if len(parts) > 1 else ''
except Exception:
value = parts[1]
value = parts[1] if len(parts) > 1 else ''
# Protection of " character, for syntax
value=value.replace('"','&quot;')
post_form += f'<input type="text" name="{escape(quote(name))}" value="{escape(quote(value))}"><br>'
post_form += '</form>'
html_content = f"""
@@ -451,5 +435,3 @@ def _post_request(req: V1RequestBase, driver: WebDriver):
</body>
</html>"""
driver.get("data:text/html;charset=utf-8,{html_content}".format(html_content=html_content))
driver.start_session()
driver.start_session() # required to bypass Cloudflare

View File

@@ -21,11 +21,11 @@ class TestFlareSolverr(unittest.TestCase):
proxy_socks_url = "socks5://127.0.0.1:1080"
google_url = "https://www.google.com"
post_url = "https://httpbin.org/post"
cloudflare_url = "https://nowsecure.nl"
cloudflare_url = "https://nowsecure.nl/"
cloudflare_url_2 = "https://idope.se/torrent-list/harry/"
ddos_guard_url = "https://anidex.info/"
ddos_guard_url = "https://www.litres.ru/"
fairlane_url = "https://www.pararius.com/apartments/amsterdam"
custom_cloudflare_url = "https://www.muziekfabriek.org"
custom_cloudflare_url = "https://www.muziekfabriek.org/"
cloudflare_blocked_url = "https://cpasbiens3.fr/index.php?do=search&subaction=search"
app = TestApp(flaresolverr.app)
@@ -162,7 +162,7 @@ class TestFlareSolverr(unittest.TestCase):
self.assertIn(self.ddos_guard_url, solution.url)
self.assertEqual(solution.status, 200)
self.assertIs(len(solution.headers), 0)
self.assertIn("<title>AniDex</title>", solution.response)
self.assertIn("<title>Литрес", solution.response)
self.assertGreater(len(solution.cookies), 0)
self.assertIn("Chrome/", solution.userAgent)

View File

@@ -471,7 +471,7 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
self.patcher.executable_path
)
super(Chrome, self).__init__(
super().__init__(
service=service,
options=options,
keep_alive=keep_alive,
@@ -507,8 +507,6 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
"Page.addScriptToEvaluateOnNewDocument",
{
"source": """
Object.defineProperty(window, "navigator", {
Object.defineProperty(window, "navigator", {
value: new Proxy(navigator, {
has: (target, key) => (key === "webdriver" ? false : key in target),
@@ -729,10 +727,8 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
def start_session(self, capabilities=None, browser_profile=None):
if not capabilities:
capabilities = self.options.to_capabilities()
super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
capabilities
)
# super(Chrome, self).start_session(capabilities, browser_profile)
super().start_session(capabilities)
# super(Chrome, self).start_session(capabilities, browser_profile) # Original explicit call commented out
def find_elements_recursive(self, by, value):
"""

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env python3
# this module is part of undetected_chromedriver
from distutils.version import LooseVersion
from packaging.version import Version as LooseVersion
import io
import json
import logging
@@ -12,6 +12,7 @@ import random
import re
import shutil
import string
import subprocess
import sys
import time
from urllib.request import urlopen
@@ -222,7 +223,7 @@ class Patcher(object):
pass
release = self.fetch_release_number()
self.version_main = release.version[0]
self.version_main = release.major
self.version_full = release
self.unzip_package(self.fetch_package())
@@ -327,11 +328,11 @@ class Patcher(object):
"""
zip_name = f"chromedriver_{self.platform_name}.zip"
if self.is_old_chromedriver:
download_url = "%s/%s/%s" % (self.url_repo, self.version_full.vstring, zip_name)
download_url = "%s/%s/%s" % (self.url_repo, str(self.version_full), zip_name)
else:
zip_name = zip_name.replace("_", "-", 1)
download_url = "https://storage.googleapis.com/chrome-for-testing-public/%s/%s/%s"
download_url %= (self.version_full.vstring, self.platform_name, zip_name)
download_url %= (str(self.version_full), self.platform_name, zip_name)
logger.debug("downloading from %s" % download_url)
return urlretrieve(download_url)[0]
@@ -373,10 +374,31 @@ class Patcher(object):
"""
exe_name = os.path.basename(exe_name)
if IS_POSIX:
r = os.system("kill -f -9 $(pidof %s)" % exe_name)
# Using shell=True for pidof, consider a more robust pid finding method if issues arise.
# pgrep can be an alternative: ["pgrep", "-f", exe_name]
# Or psutil if adding a dependency is acceptable.
command = f"pidof {exe_name}"
try:
result = subprocess.run(command, shell=True, capture_output=True, text=True, check=True)
pids = result.stdout.strip().split()
if pids:
subprocess.run(["kill", "-9"] + pids, check=False) # Changed from -f -9 to -9 as -f is not standard for kill
return True
return False # No PIDs found
except subprocess.CalledProcessError: # pidof returns 1 if no process found
return False # No process found
except Exception as e:
logger.debug(f"Error killing process on POSIX: {e}")
return False
else:
r = os.system("taskkill /f /im %s" % exe_name)
return not r
try:
# TASKKILL /F /IM chromedriver.exe
result = subprocess.run(["taskkill", "/f", "/im", exe_name], check=False, capture_output=True)
# taskkill returns 0 if process was killed, 128 if not found.
return result.returncode == 0
except Exception as e:
logger.debug(f"Error killing process on Windows: {e}")
return False
@staticmethod
def gen_random_cdc():

View File

@@ -1,11 +1,12 @@
import json
import logging
import os
import platform
import re
import shutil
import urllib.parse
import tempfile
import sys
import tempfile
import urllib.parse
from selenium.webdriver.chrome.webdriver import WebDriver
import undetected_chromedriver as uc
@@ -57,18 +58,21 @@ def create_proxy_extension(proxy: dict) -> str:
manifest_json = """
{
"version": "1.0.0",
"manifest_version": 2,
"manifest_version": 3,
"name": "Chrome Proxy",
"permissions": [
"proxy",
"tabs",
"unlimitedStorage",
"storage",
"<all_urls>",
"webRequest",
"webRequestBlocking"
"webRequestAuthProvider"
],
"background": {"scripts": ["background.js"]},
"host_permissions": [
"<all_urls>"
],
"background": {
"service_worker": "background.js"
},
"minimum_chrome_version": "76.0.0"
}
"""
@@ -129,21 +133,18 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
options = uc.ChromeOptions()
options.add_argument('--no-sandbox')
options.add_argument('--window-size=1920,1080')
options.add_argument('--disable-search-engine-choice-screen')
# todo: this param shows a warning in chrome head-full
options.add_argument('--disable-setuid-sandbox')
options.add_argument('--disable-dev-shm-usage')
# this option removes the zygote sandbox (it seems that the resolution is a bit faster)
options.add_argument('--no-zygote')
# attempt to fix Docker ARM32 build
IS_ARMARCH = platform.machine().startswith(('arm', 'aarch'))
if IS_ARMARCH:
options.add_argument('--disable-gpu-sandbox')
options.add_argument('--disable-software-rasterizer')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
# fix GL errors in ASUSTOR NAS
# https://github.com/FlareSolverr/FlareSolverr/issues/782
# https://github.com/microsoft/vscode/issues/127800#issuecomment-873342069
# https://peter.sh/experiments/chromium-command-line-switches/#use-gl
options.add_argument('--use-gl=swiftshader')
language = os.environ.get('LANG', None)
if language is not None:
@@ -156,6 +157,7 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
proxy_extension_dir = None
if proxy and all(key in proxy for key in ['url', 'username', 'password']):
proxy_extension_dir = create_proxy_extension(proxy)
options.add_argument("--disable-features=DisableLoadExtensionCommandLineSwitch")
options.add_argument("--load-extension=%s" % os.path.abspath(proxy_extension_dir))
elif proxy and 'url' in proxy:
proxy_url = proxy['url']
@@ -173,8 +175,6 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
# For normal headless mode:
# options.add_argument('--headless')
options.add_argument("--auto-open-devtools-for-tabs")
# if we are inside the Docker container, we avoid downloading the driver
driver_exe_path = None
version_main = None
@@ -197,6 +197,8 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
windows_headless=windows_headless, headless=get_config_headless())
except Exception as e:
logging.error("Error starting Chrome: %s" % e)
# No point in continuing if we cannot retrieve the driver
raise e
# save the patched driver to avoid re-downloads
if driver_exe_path is None:
@@ -298,7 +300,7 @@ def extract_version_nt_folder() -> str:
paths = [f.path for f in os.scandir(path) if f.is_dir()]
for path in paths:
filename = os.path.basename(path)
pattern = '\d+\.\d+\.\d+\.\d+'
pattern = r'\d+\.\d+\.\d+\.\d+'
match = re.search(pattern, filename)
if match and match.group():
# Found a Chrome version.

View File

@@ -1 +1 @@
WebTest==3.0.0
WebTest==3.0.7