Fix Chrome v124+ not closing on Windows. resolves #1161 (#1193)

Co-authored-by: root@library.moodysalon.net <root@library.moodysalon.net>
Co-authored-by: ilike2burnthing <59480337+ilike2burnthing@users.noreply.github.com>

CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## v3.3.18 (2024/05/20)
+
+* Fix LANG ENV for Linux
+* Fix Chrome v124+ not closing on Windows. Thanks @RileyXX
+
+## v3.3.17 (2024/04/09)
+
+* Fix file descriptor leak in service on quit(). Thanks @zkulis
+
+## v3.3.16 (2024/02/28)
+
+* Fix of the subprocess.STARTUPINFO() call. Thanks @ceconelo
+* Add FreeBSD support. Thanks @Asthowen
+* Use headless configuration properly. Thanks @hashworks
+
 ## v3.3.15 (2024/02/20)
 
 * Fix looping challenges

Dockerfile

@@ -62,17 +62,17 @@ ENTRYPOINT ["/usr/bin/dumb-init", "--"]
 CMD ["/usr/local/bin/python", "-u", "/app/flaresolverr.py"]
 
 # Local build
-# docker build -t ngosang/flaresolverr:3.3.15 .
-# docker run -p 8191:8191 ngosang/flaresolverr:3.3.15
+# docker build -t ngosang/flaresolverr:3.3.18 .
+# docker run -p 8191:8191 ngosang/flaresolverr:3.3.18
 
 # Multi-arch build
 # docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
 # docker buildx create --use
-# docker buildx build -t ngosang/flaresolverr:3.3.15 --platform linux/386,linux/amd64,linux/arm/v7,linux/arm64/v8 .
+# docker buildx build -t ngosang/flaresolverr:3.3.18 --platform linux/386,linux/amd64,linux/arm/v7,linux/arm64/v8 .
 #   add --push to publish in DockerHub
 
 # Test multi-arch build
 # docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
 # docker buildx create --use
-# docker buildx build -t ngosang/flaresolverr:3.3.15 --platform linux/arm/v7 --load .
-# docker run -p 8191:8191 --platform linux/arm/v7 ngosang/flaresolverr:3.3.15
+# docker buildx build -t ngosang/flaresolverr:3.3.18 --platform linux/arm/v7 --load .
+# docker run -p 8191:8191 --platform linux/arm/v7 ngosang/flaresolverr:3.3.18

README.md

@@ -45,7 +45,8 @@ Supported architectures are:
 | ARM32        | linux/arm/v7 |
 | ARM64        | linux/arm64  |
 
-We provide a `docker-compose.yml` configuration file. Clone this repository and execute `docker-compose up -d` to start
+We provide a `docker-compose.yml` configuration file. Clone this repository and execute
+`docker-compose up -d` _(Compose V1)_ or `docker compose up -d` _(Compose V2)_ to start
 the container.
 
 If you prefer the `docker cli` execute the following command.
@@ -78,11 +79,19 @@ This is the recommended way for Windows users.
 
 * Install [Python 3.11](https://www.python.org/downloads/).
 * Install [Chrome](https://www.google.com/intl/en_us/chrome/) (all OS) or [Chromium](https://www.chromium.org/getting-involved/download-chromium/) (just Linux, it doesn't work in Windows) web browser.
-* (Only in Linux / macOS) Install [Xvfb](https://en.wikipedia.org/wiki/Xvfb) package.
+* (Only in Linux) Install [Xvfb](https://en.wikipedia.org/wiki/Xvfb) package.
+* (Only in macOS) Install [XQuartz](https://www.xquartz.org/) package.
 * Clone this repository and open a shell in that path.
 * Run `pip install -r requirements.txt` command to install FlareSolverr dependencies.
 * Run `python src/flaresolverr.py` command to start FlareSolverr.
 
+### From source code (FreeBSD/TrueNAS CORE)
+
+* Run `pkg install chromium python39 py39-pip xorg-vfbserver` command to install the required dependencies.
+* Clone this repository and open a shell in that path.
+* Run `python3.9 -m pip install -r requirements.txt` command to install FlareSolverr dependencies.
+* Run `python3.9 src/flaresolverr.py` command to start FlareSolverr.
+
 ### Systemd service
 
 We provide an example Systemd unit file `flaresolverr.service` as reference. You have to modify the file to suit your needs: paths, user and environment variables.
@@ -95,7 +104,7 @@ curl -L -X POST 'http://localhost:8191/v1' \
 -H 'Content-Type: application/json' \
 --data-raw '{
   "cmd": "request.get",
-  "url":"http://www.google.com/",
+  "url": "http://www.google.com/",
   "maxTimeout": 60000
 }'
 ```
@@ -269,8 +278,8 @@ This is the same as `request.get` but it takes one more param:
 
 Environment variables are set differently depending on the operating system. Some examples:
 * Docker: Take a look at the Docker section in this document. Environment variables can be set in the `docker-compose.yml` file or in the Docker CLI command.
-* Linux: Run `export LOG_LEVEL=debug` and then start FlareSolverr in the same shell.
-* Windows: Open `cmd.exe`, run `set LOG_LEVEL=debug` and then start FlareSolverr in the same shell.
+* Linux: Run `export LOG_LEVEL=debug` and then run `flaresolverr` in the same shell.
+* Windows: Open `cmd.exe`, run `set LOG_LEVEL=debug` and then run `flaresolverr.exe` in the same shell.
 
 ## Prometheus exporter
 

package.json

@@ -1,6 +1,6 @@
 {
   "name": "flaresolverr",
-  "version": "3.3.15",
+  "version": "3.3.18",
   "description": "Proxy server to bypass Cloudflare protection",
   "author": "Diego Heras (ngosang / ngosang@hotmail.es)",
   "license": "MIT"

requirements.txt

@@ -7,7 +7,7 @@ prometheus-client==0.17.1
 requests==2.31.0
 certifi==2023.7.22
 websockets==11.0.3
-# only required for linux
-xvfbwrapper==0.2.9
+# only required for linux and macos
+xvfbwrapper==0.2.9; platform_system != "Windows"
 # only required for windows
-pefile==2023.2.7
+pefile==2023.2.7; platform_system == "Windows"

src/flaresolverr_service.py

@@ -245,6 +245,7 @@ def _resolve_challenge(req: V1RequestBase, method: str) -> ChallengeResolutionT:
         raise Exception('Error solving the challenge. ' + str(e).replace('\n', '\\n'))
     finally:
         if not req.session and driver is not None:
+            driver.close()
             driver.quit()
             logging.debug('A used instance of webdriver has been destroyed')
 

src/sessions.py

@@ -66,6 +66,7 @@ class SessionsStorage:
             return False
 
         session = self.sessions.pop(session_id)
+        session.driver.close()
         session.driver.quit()
         return True
 

src/undetected_chromedriver/__init__.py

@@ -451,8 +451,10 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
                 options.binary_location, *options.arguments
             )
         else:
-            startupinfo = subprocess.STARTUPINFO()
+            startupinfo = None
             if os.name == 'nt' and windows_headless:
+                # STARTUPINFO() is Windows only
+                startupinfo = subprocess.STARTUPINFO()
                 startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
             browser = subprocess.Popen(
                 [options.binary_location, *options.arguments],
@@ -769,7 +771,9 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
 
     def quit(self):
         try:
+            self.service.stop()
             self.service.process.kill()
+            self.command_executor.close()
             self.service.process.wait(5)
             logger.debug("webdriver process ended")
         except (AttributeError, RuntimeError, OSError):
@@ -784,15 +788,6 @@ class Chrome(selenium.webdriver.chrome.webdriver.WebDriver):
             logger.debug("gracefully closed browser")
         except Exception as e:  # noqa
             pass
-        # Force kill Chrome process in Windows
-        # https://github.com/FlareSolverr/FlareSolverr/issues/772
-        if os.name == 'nt':
-            try:
-                subprocess.call(['taskkill', '/f', '/pid', str(self.browser_pid)],
-                                stdout=subprocess.DEVNULL,
-                                stderr=subprocess.DEVNULL)
-            except Exception:
-                pass
         if (
             hasattr(self, "keep_user_data_dir")
             and hasattr(self, "user_data_dir")

src/undetected_chromedriver/devtool.py

@@ -187,4 +187,5 @@ def test():
 
     time.sleep(10)
 
+    driver.close()
     driver.quit()

src/undetected_chromedriver/patcher.py

@@ -21,7 +21,7 @@ from multiprocessing import Lock
 
 logger = logging.getLogger(__name__)
 
-IS_POSIX = sys.platform.startswith(("darwin", "cygwin", "linux", "linux2"))
+IS_POSIX = sys.platform.startswith(("darwin", "cygwin", "linux", "linux2", "freebsd"))
 
 
 class Patcher(object):
@@ -80,9 +80,14 @@ class Patcher(object):
             os.makedirs(self.data_path, exist_ok=True)
 
         if not executable_path:
-            self.executable_path = os.path.join(
-                self.data_path, "_".join([prefix, self.exe_name])
-            )
+            if sys.platform.startswith("freebsd"):
+                self.executable_path = os.path.join(
+                    self.data_path, self.exe_name
+                )
+            else:
+                self.executable_path = os.path.join(
+                    self.data_path, "_".join([prefix, self.exe_name])
+                )
 
         if not IS_POSIX:
             if executable_path:
@@ -127,6 +132,9 @@ class Patcher(object):
             else:
                 self.platform_name = "mac-x64"
             self.exe_name %= ""
+        if self.platform.startswith("freebsd"):
+            self.platform_name = "freebsd"
+            self.exe_name %= ""
 
     def auto(self, executable_path=None, force=False, version_main=None, _=None):
         """
@@ -166,26 +174,56 @@ class Patcher(object):
         if force is True:
             self.force = force
 
-        try:
-            os.unlink(self.executable_path)
-        except PermissionError:
-            if self.force:
-                self.force_kill_instances(self.executable_path)
-                return self.auto(force=not self.force)
-            try:
-                if self.is_binary_patched():
-                    # assumes already running AND patched
-                    return True
-            except PermissionError:
-                pass
-            # return False
-        except FileNotFoundError:
-            pass
 
-        release = self.fetch_release_number()
-        self.version_main = release.version[0]
-        self.version_full = release
-        self.unzip_package(self.fetch_package())
+        if self.platform_name == "freebsd":
+            chromedriver_path = shutil.which("chromedriver")
+
+            if not os.path.isfile(chromedriver_path) or not os.access(chromedriver_path, os.X_OK):
+                logging.error("Chromedriver not installed!")
+                return
+
+            version_path = os.path.join(os.path.dirname(self.executable_path), "version.txt")
+
+            process = os.popen(f'"{chromedriver_path}" --version')
+            chromedriver_version = process.read().split(' ')[1].split(' ')[0]
+            process.close()
+
+            current_version = None
+            if os.path.isfile(version_path) or os.access(version_path, os.X_OK):
+                with open(version_path, 'r') as f:
+                    current_version = f.read()
+
+            if current_version != chromedriver_version:
+                logging.info("Copying chromedriver executable...")
+                shutil.copy(chromedriver_path, self.executable_path)
+                os.chmod(self.executable_path, 0o755)
+
+                with open(version_path, 'w') as f:
+                    f.write(chromedriver_version)
+
+                logging.info("Chromedriver executable copied!")
+        else:
+            try:
+                os.unlink(self.executable_path)
+            except PermissionError:
+                if self.force:
+                    self.force_kill_instances(self.executable_path)
+                    return self.auto(force=not self.force)
+                try:
+                    if self.is_binary_patched():
+                        # assumes already running AND patched
+                        return True
+                except PermissionError:
+                    pass
+                # return False
+            except FileNotFoundError:
+                pass
+
+            release = self.fetch_release_number()
+            self.version_main = release.version[0]
+            self.version_full = release
+            self.unzip_package(self.fetch_package())
+
         return self.patch()
 
     def driver_binary_in_use(self, path: str = None) -> bool:

src/utils.py

@@ -5,6 +5,7 @@ import re
 import shutil
 import urllib.parse
 import tempfile
+import sys
 
 from selenium.webdriver.chrome.webdriver import WebDriver
 import undetected_chromedriver as uc
@@ -138,7 +139,7 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
 
     language = os.environ.get('LANG', None)
     if language is not None:
-        options.add_argument('--lang=%s' % language)
+        options.add_argument('--accept-lang=%s' % language)
 
     # Fix for Chrome 117 | https://github.com/FlareSolverr/FlareSolverr/issues/910
     if USER_AGENT is not None:
@@ -182,9 +183,12 @@ def get_webdriver(proxy: dict = None) -> WebDriver:
 
     # downloads and patches the chromedriver
     # if we don't set driver_executable_path it downloads, patches, and deletes the driver each time
-    driver = uc.Chrome(options=options, browser_executable_path=browser_executable_path,
-                       driver_executable_path=driver_exe_path, version_main=version_main,
-                       windows_headless=windows_headless, headless=windows_headless)
+    try:
+        driver = uc.Chrome(options=options, browser_executable_path=browser_executable_path,
+                           driver_executable_path=driver_exe_path, version_main=version_main,
+                           windows_headless=windows_headless, headless=get_config_headless())
+    except Exception as e:
+        logging.error("Error starting Chrome: %s" % e)
 
     # save the patched driver to avoid re-downloads
     if driver_exe_path is None:
@@ -310,6 +314,7 @@ def get_user_agent(driver=None) -> str:
         raise Exception("Error getting browser User-Agent. " + str(e))
     finally:
         if driver is not None:
+            driver.close()
             driver.quit()