Adding hurl tests to Dolibarr (#35643)

* Adding hurl tests to Dolibarr

* modifying run.sh to get it past commit checks

* change the APIHEADER to be read from the environment

* spell fix

* test to see if build system will accept this file

* deleting script method that doesn't work

* beginning to make hurl tests accept environment variables for the location

---------

Co-authored-by: Jon Bendtsen <xcodeauthor@jonb.dk>
Co-authored-by: Laurent Destailleur <eldy@destailleur.fr>

test/hurl/README.md

@@ -0,0 +1,152 @@
+# hurling Dolibarr
+
+## What’s Hurl?
+Hurl is an Open Source command line tool that makes http requests. https://hurl.dev/
+
+### Why
+
+We can use hurl to make (thorough and comprehensive) **automatic tests** for Dolibarr to ensure that our changes do not unintended break stuff. We can and should test both the API, as well as the GUI part of Dolibarr. Doing this will allow us to release faster and better quality software. It will also allow us to do **test driven development** where we write the (hurl) tests before we write our code to satisfy the (hurl) tests, which again will allow us to release faster and better quality software.
+
+### Getting hurl
+
+We defer you to https://hurl.dev/docs/installation.html
+
+### Why hurl?
+
+This author wanted a http request test tool which was Open Source and CLI based for possible CI/CD integration. Hurl stood out as easy to get started, yet powerful and capable. There may very well be other tools, but it is more important to get started using A tool, because the biggest work lies in writing all the tests, and that work is almost identical regardless of the tool used to do the tests.
+
+## When and where to test
+
+Hurl tests should be run **early and often** to give the maximum value - and all should pass.
+
+### Locally on your _development_ computer
+
+**!! DO NOT RUN AGAINST YOUR PRODUCTION DOLIBARR !!**
+
+At some time in the future you should preferably as a Dolibarr developer, tester, engineer, QA, ... have run these hurl tests before you make a commit, but we are not there yet!
+
+The hurl tests may very well **be destructive**, in fact some **are destructive** because we do need to test all the functionality of Dolibarr, including **breaking and deleting** stuff, so please be careful about the target Dolibarr you run it against.
+
+#### Using hurl for health/configuration testing
+
+You may want to look into using hurl to regularly test the integrity, health and configuration of your production Dolibarr, but that is something you would have to write yourself, and perhaps avoid destructive tests? In the future we may provide a hurl configuration to get started.
+
+### During CI/CD
+
+At some time in the future we should run these hurl tests as part of the CI/CD chain during commit checks, but we are not there yet!
+
+## Timeline
+
+This is a prelininary timeline for hurling Dolibarr
+
+### Pre 2026
+
+We discuss and try out hurl for manual testing pre commit.
+
+### 2026
+
+During the various developer camps we introduce hurl to devcamp participants and they help expand the tests so we collectively get a thorough and comprehensive automatic test suite for Dolibarr.
+
+Perhaps we starting using it as part of the CI/CD chain, though probably only as a warning. Maybe it is run against releases only?
+
+We figure out how to only do those hurl tests which a relevant for the commit.
+
+### 2027
+
+We enable strict automatic testing during the CI/CD chain such that the checks will fail if hurl tests fail.
+
+All changes and new development must come with relevant hurl tests.
+
+DoliStore modules should come with hurl tests.
+
+We start exploring the duration part of hurl to do performance improvements.
+
+### 2028
+
+We have a very thorough and comprehensive automatic QA test for each and every part of Dolibarr.
+
+All changes and new development must come with thorough and comprehensive hurl tests.
+
+DoliStore modules must come with hurl tests.
+
+## Howto Usage
+
+1. Create the .settings/ directory
+2. Set these environment variables to know how to authenticate and where to reach your Dolibarr installation
+
+   DOLAPIKEY="DOLAPIKEY: _replace_with_your_Dolibarr_Token_for_API_"
+
+   DOLIHOST="http://example.net/"
+
+   DOLIPORT="8080"
+
+   DOLISUBURL="/dolibarr" # if you're not just using / and no need to set /
+
+3. On Linux and mac execute ./run.sh
+
+   On Windows? Please test and submit a PR on this file documenting how to run it on Windows incl. a script like run.sh - but just for Windows.
+
+4. Write new hurl tests during your development and submit using a PR - preferably using the very same PR that submits your changes, improvements and refinements of Dolibarr.
+
+## Directory and file structure for hurl tests
+
+To avoid ending up with thousands of files in the same directory, we should use directories and possible subdirectories where relevant.
+
+Broadly the hurl tests are split in api/ tests, gui/ tests and public/ tests respectively.
+
+Below that the directory name should be the first part of the endpoint, this could be the module name, but the orders/ API endpoint does not match the module name 'commande', and public/onlinesign/ might be used by multiple modules.
+
+### Undecided discrepancy
+
+It is currently undecided how we have to handle the discrepancy between the http endpoint and the module name! Let's together try, learn, discuss and finally agree on a decision!
+
+
+## Self contained .hurl files
+
+Each individual .hurl file should be entirely self contained, and where that is impractical (or impossible), this is where you use numbers bigger than 00 (no authentication needed) and 10+ (authentication needed) for the start of the filename. Any files with a bigger number than 10 are assumed to require authentication headers.
+
+- 00 in the beginning of the filename means that no authentication is required to run this test
+- 10 in the beginning of the filename means that authentication is required to run the test
+- 20 is reserved for future usage
+- 30 in the beginning of the filename is where you would POST data for later tests
+- 40 in the beginning of the filename is where you would GET data that was POST'ed in 30_'s files
+- 50 in the beginning of the filename is where you would PUT data that was POST'ed in 30_'s files
+- 60 in the beginning of the filename is where you would GET data that was PUT'ed in 50_'s files
+- 70 in the beginning of the filename is where you would DELETE data
+- 80 in the beginning of the filename is where you would GET data that was just DELETE'd to make sure it is DELETE'd
+- 90 is reserved for future usage
+
+You may use x[0-9] file names if absolutely needed.
+
+### Endpoint should be part of filename
+
+To make it quicker to see what the file does, the endpoint should be part of the filename, but make it after the number.
+
+### HTTP method may be part of the filename
+
+You may add the HTTP method as part of the filename, but make it after the number, and just before .hurl
+
+### Separator _ is more readable
+
+_ was chosen as separator, because it is more readable than .
+
+### Examples:
+
+- api/00_explorer.hurl
+- api/setup/10_setup_modules.hurl
+- api/setup/10_setup_conf.hurl
+- api/setup/10_setup_company.hurl
+- api/status/00_status.hurl
+- api/status/10_status.hurl
+- api/00_foobar.hurl
+- api/login/00_login_POST.hurl
+- api/login/00_login_GET.hurl
+- gui/00_HOME.hurl
+- public/payment/00_payment_newpayment.hurl
+- public/onlinesign/00_onlinesign_newonlinesign.hurl
+
+## Translations and Number separator
+
+Given that Dolibarr may respond differently depending on the language configuration of Dolibarr and/or the language configuration of the webbrowser, we should have that in mind when writing out hurl test files, such that they **must** be able to be used on any Dolibarr language configuration.
+
+Thus the check in `public/payment/00_payment_newpayment.hurl` might fail on your Dolibarr installation, if so, please submit a PR with a fix.

test/hurl/api/00_explorer.hurl

@@ -0,0 +1,3 @@
+# Check API Explorer
+GET http://{{hostnport}}/api/index.php/explorer/#/
+HTTP 200

test/hurl/api/00_foobar.hurl

@@ -0,0 +1,3 @@
+# Check endpoint which does not exist
+GET http://{{hostnport}}/api/index.php/foobar
+HTTP 501

test/hurl/api/login/00_login_GET.hurl

@@ -0,0 +1,6 @@
+# GET Login should fail
+GET http://{{hostnport}}/api/index.php/login?login=foo&password=bar
+HTTP 403
+[Asserts]
+jsonpath "$.error.code" == 403
+jsonpath "$.error.message" matches "Forbidden: Access denied"

test/hurl/api/login/00_login_POST.hurl

@@ -0,0 +1,7 @@
+# POST Login should fail
+POST http://{{hostnport}}/api/index.php/login
+{
+    "login" : "foo",
+    "password" : "bar"
+}
+HTTP 403

test/hurl/api/setup/10_setup_company.hurl

@@ -0,0 +1,7 @@
+# Check status
+GET http://localhost:8080/api/index.php/setup/company
+HTTP 200
+[Asserts]
+jsonpath "$.id" == 0     # Check the value of the id
+jsonpath "$.status" == 1
+jsonpath "$.module" == "societe"

test/hurl/api/setup/10_setup_conf.hurl

@@ -0,0 +1,3 @@
+# Check status
+GET http://localhost:8080/api/index.php/setup/conf
+HTTP 200

test/hurl/api/setup/10_setup_modules.hurl

@@ -0,0 +1,5 @@
+# Check status
+GET http://localhost:8080/api/index.php/setup/modules
+HTTP 200
+[Asserts]
+jsonpath "$" count > 2

test/hurl/api/status/00_status.hurl

@@ -0,0 +1,3 @@
+# Check status
+GET http://localhost:8080/api/index.php/status
+HTTP 401

test/hurl/api/status/10_status.hurl

@@ -0,0 +1,3 @@
+# Check status
+GET http://localhost:8080/api/index.php/status
+HTTP 200

test/hurl/gui/00_HOME.hurl

@@ -0,0 +1,3 @@
+# Check status
+GET http://localhost:8080/
+HTTP 200

test/hurl/public/onlinesign/00_onlinesign_newonlinesign.hurl

@@ -0,0 +1,2 @@
+GET http://localhost:8080/public/onlinesign/newonlinesign.php
+HTTP 400

test/hurl/public/payment/00_payment_newpayment.hurl

@@ -0,0 +1,3 @@
+GET http://localhost:8080/public/payment/newpayment.php
+HTTP 200
+`Bad parameters - amount or source`

test/hurl/run.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+
+if [[ -z ${DOLIHOST+x} ]]; then
+	DOLIHOST="localhost"
+fi
+hostnport="${DOLIHOST}"
+if [[ -z ${DOLIPORT+x} ]]; then
+	hostnport="${hostnport}:8080"
+else
+	hostnport="${hostnport}:${DOLIPORT}"
+fi
+if [[ -z ${DOLISUBURL+x} ]]; then
+	DOLISUBURL=""
+fi
+if [[ "" != "${DOLISUBURL}" ]]; then
+	hostnport="${hostnport}/${DOLISUBURL}"
+fi
+
+echo "First we run tests that do not require authentication"
+find api/ gui/ public/ -type f -iname '00*.hurl' -exec hurl --variable "hostnport=${hostnport}" --test "{}" \;
+
+# Now we get ready to run tests that do require authentication
+if [[ -z ${DOLAPIKEY+x} ]]; then
+	echo "DOLAPIKEY bash variable is unset, no API tests that require authentication"
+else
+	echo "Now we are ready to run API tests that do require authentication"
+	find api/ -type f -iname '10*.hurl' -not -iname '00*.hurl' -exec hurl --header "${DOLAPIKEY}" --test "{}" \;
+fi
+
+if [[ -z ${GUIHEADER+x} ]]; then
+	echo "GUIHEADER bash variable is unset, no GUI tests that require authentication"
+else
+	echo "Now we are ready to run GUI tests that do require authentication"
+	find gui/ -type f -iname '10*.hurl' -not -iname '00*.hurl' -exec hurl --header "${GUIHEADER}" --test "{}" \;
+fi