Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -165,3 +165,4 @@ cli-export-config.json
cli-import-config.json
robot_tests/
reports/
openapi.yml
7 changes: 0 additions & 7 deletions CreatePiPWheel.bat

This file was deleted.

5 changes: 0 additions & 5 deletions CreatePiPWheel.sh

This file was deleted.

48 changes: 29 additions & 19 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,35 @@
# Contributing
## Setting up project for the first time
1. Write all dependencies into pyproject.toml
2. Create venv (python -m venv .venv) and activate it (".venv\scripts\activate" or "source .venv/bin/activate")
3. Update pip (python -m pip install -U pip)
3. Install pip-tools (pip install pip-tools)
4. Update/Create `requirements.txt` (when dependencies have been updated in pyproject.toml)
- with Development dependencies
> pip-compile --extra dev
- or only with Runtime dependencies
> pip-compile
5. Install dependencies (pip install -U -r requirements.txt)
6. Install project into local venv (pip install -e .[dev])

## Creating whl and publish
1. Install 'setuptools' (pip install setuptools)
2. Run CreatePipWheel Script
## Setting up project for the first time

1. Create venv and activate it:
```bash
python -m venv .venv
source .venv/bin/activate # Linux/macOS
.venv\scripts\activate # Windows
```
2. Install project with dev dependencies:
```bash
pip install -e .[dev]
```

## Generate documentation in Word format
[Install Pandoc](https://pandoc.org/installing.html)
## Building and publishing

```shell
pandoc -s README.md -M title="imbus TestBench - Robot Code Generator" -M subtitle=Benutzerhandbuch -M toc-title=Inhaltsverzeichnis --toc -o Benutzerhandbuch.docx
```bash
check-manifest --update
python -m build
twine check dist/*
twine upload dist/*
```

## Updating the data model

`testbench2robotframework/model.py` is generated from the TestBench OpenAPI spec using [datamodel-code-generator](https://github.com/koxudaxi/datamodel-code-generator).

1. Download the OpenAPI YAML from the TestBench Swagger documentation (e.g. `openapi.yml`).
2. Generate the model (settings are in `pyproject.toml` under `[tool.datamodel-codegen]`):
```bash
invoke generate-model --input openapi.yml
```
This runs `datamodel-codegen` and injects `__VERSION__` from the spec's `info.version` field into `model.py`.
3. Review the generated file
21 changes: 11 additions & 10 deletions ExampleConfiguration/json_config.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,14 @@
"output-directory": "{root}/Generated",
"log-suite-numbering": true,
"resource-directory": "{root}/Resources",
"resource-directory-regex": ".*\\[Robot-Resources\\].*",
"clean": true,
"compound-keyword-logging": "GROUP",
"reference-behaviour": "ATTACHMENT",
"attachment-conflict-behaviour": "USE_EXISTING",
"testcase-splitting-regex": "^StopWithRestart\\..*",
"phase-pattern": "{testcase} : Phase {index}/{length}",
"metadata": {},
"library-mapping": {
"SeleniumLibrary": "SeleniumLibrary timeout=10 implicit_wait=1 run_on_failure=Capture Page Screenshot",
"SuperRemoteLibrary": "Remote http://127.0.0.1:8270 WITH NAME SuperRemoteLibrary"
Expand All @@ -20,20 +26,15 @@
"MyKeywords": "{root}/../MyKeywords.resource",
"MyOtherKeywords": "{resourceDirectory}/subdir/MyOtherKeywords.resource"
},
"forcedImport": {
"libraries": [
],
"resources":
[
],
"variables": [
]
"forced-import": {
"libraries": [],
"resources": [],
"variables": []
},
"testCaseSplitPathRegEx": "^StopWithRestart\\..*",
"console-logging": {
"logLevel": "debug"
},
"file-logging": {
"logLevel": "info"
}
}
}
5 changes: 3 additions & 2 deletions ExampleConfiguration/pyproject_example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ fully-qualified = false
output-directory = "{root}/Generated"
log-suite-numbering = false
clean = true
compound-keyword-logging = true
compound-keyword-logging = "GROUP"
resource-directory = "{root}/Resources"
testcase-split-regex = ".*StopWithRestart.*"
resource-directory-regex = ".*\\[Robot-Resources\\].*"
testcase-splitting-regex = ".*StopWithRestart.*"
phase-pattern = "{testcase} : Phase {index}/{length}"
reference-behaviour = "ATTACHMENT"
attachment-conflict-behaviour = "USE_EXISTING"
Expand Down
5 changes: 3 additions & 2 deletions ExampleConfiguration/toml_config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ fully-qualified = false
output-directory = "{root}/Generated"
log-suite-numbering = false
clean = true
compound-keyword-logging = true
compound-keyword-logging = "GROUP"
resource-directory = "{root}/Resources"
testcase-split-regex = ".*StopWithRestart.*"
resource-directory-regex = ".*\\[Robot-Resources\\].*"
testcase-splitting-regex = ".*StopWithRestart.*"
phase-pattern = "{testcase} : Phase {index}/{length}"
reference-behaviour = "ATTACHMENT"
attachment-conflict-behaviour = "USE_EXISTING"
Expand Down
131 changes: 9 additions & 122 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,131 +1,18 @@
# TestBench2RobotFramework

**TestBench2RobotFramework** is a CLI tool used to convert a TestBench JSON report into Robot Framework test suites and to write the execution results provided by Robot Framework to the TestBench report.
testbench2robotframework is a CLI tool to convert a TestBench JSON report into Robot Framework test suites and to write the execution results provided by Robot Framework back to the TestBench report.

## Installation
This can be used for automated test execution with Robot Framework, triggered by a CI/CD pipeline.

You can install TestBench2RobotFramework via pip using the following command:
## Documentation

```powershell
pip install testbench2robotframework
```
The documentation is available in the [docs](./docs) folder of this repository.

Python 3.10 or higher is required to run this tool.
## Releases

## Remark
TestBench2RobotFramework requires TestBench version >= 4. If you're running an older version please contact the TestBench support for information on how to connect your Version of TestBench to Robot Framework. The TestBench Report can be either be exported via the TestBench Rest API with tools like the testbench-cli-reporter or directly from the client.
Releases are published on GitHub and available via [PyPI](https://pypi.org/project/testbench2robotframework/).

## Usage
## Requirements

TestBench2RobotFramework supports two main use cases, which are described in more detail in the following sections:

1. Generating Robot Framework test suites from a TestBench report.
2. Fetching results from a Robot Framework output XML file and saving them back to a TestBench report.

### Generating Robot Framework Test Suites
To generate Robot Framework test suites, use the `generate-tests` subcommand:

```powershell
testbench2robotframework generate-tests TESTBENCH_REPORT
```

This command generates a Robot Framework test suite for each test case set specified in the `TESTBENCH_REPORT`.

![](./images/testthemen.PNG)
![](./images/generated.PNG)

The example above demonstrates how Robot Framework test suites are generated based on the *Test Theme Tree* defined in TestBench.



#### Configuration

There are multiple configuration options available for **TestBench2RobotFramework** that can be used to customize the generated test suites. Options can be specified either via the command line, in a `pyproject.toml` file or in a `robot.toml` file.

To use options via the command line, the following syntax is used:

```powershell
testbench2robotframework generate-tests [OPTIONS] TESTBENCH_REPORT
```

| Option | Description |
|--------|-------------|
| `-c`, `--config PATH` | Path to a configuration file for TestBench2RobotFramework. |
| `--clean` | Deletes all files present in the output-directory before new test suites are created. |
| `-d`, `--output-directory PATH` | Directory or ZIP archive containing the generated test suites. |
| `--compound-keyword-logging` | Mode for logging compound keywords. Options: `GROUP`, `COMMENT`, or `NONE`. |
| `--fully-qualified` | Calls Robot Framework keywords by their fully qualified names in the generated test suites. |
| `--log-suite-numbering` | Enables logging of the test suite numbering. |
| `--metadata` | Add extra metadata to the settings of the generated Robot Framework test suite. Provide entries as key:value pairs, where *key* is the metadata name and *value* is the corresponding value. Values may also be Python expressions. The special variable '$tcs' gives access to the TestBench Python model of the test case set. |
| `--resource-directory PATH` | Directory containing the Robot Framework resource files. |
| `--resource-directory-regex TEXT` | Regex that can be used to identify the TestBench Subdivision that corresponds to the <resource-directory>. Resources will be imported relative to this subdivision based on the test elements structure in TestBench. |
| `--library-regex TEXT` | Regular expression used to identify TestBench subdivisions corresponding to Robot Framework libraries. |
| `--library-root TEXT` | TestBench root subdivision whose direct children correspond to Robot Framework libraries. |
| `--resource-regex TEXT` | Regular expression used to identify TestBench subdivisions corresponding to Robot Framework resources. |
| `--resource-root TEXT` | TestBench root subdivision whose direct children correspond to Robot Framework resources. |
| `--library-mapping TEXT` | Library import statement to use when a keyword from the specified TestBench subdivision is encountered. |
| `--resource-mapping TEXT` | Resource import statement to use when a keyword from the specified TestBench subdivision is encountered. |
| `--help` | Displays the help message and exits. |
| `--version` | Writes the TestBench2RobotFramework, Robot Framework and Python version to console. |


### Saving Robot Framework Results

Saving the results requires a Robot Framework output XML file, along with the original TestBench report from which the test suites were generated.

Use the following command:

```powershell
testbench2robotframework fetch-results [OPTIONS] ROBOT_RESULT TESTBENCH_REPORT
```

| Option | Description |
|--------|-------------|
| `-c`, `--config PATH` | Path to a configuration file for TestBench2RobotFramework. |
| `-d`, `--output-directory PATH` | Path to the directory or ZIP file where the updated TestBench JSON report (with results) should be saved. |
| `--help` | Displays the help message and exits. |



### Using pyproject.toml
All CLI options available for ``testbench2robotframework`` can also be defined in your ``pyproject.toml`` file, ``robot.toml``, or a workspace-local ``.robot.toml``. This offers a convenient way to store and reuse configuration settings, particularly in larger projects or automated environments.

#### Example
```toml
[tool.testbench2robotframework]
library-regex = ['(?:.*\.)?(?P<resourceName>[^.]+?)\s*\[Robot-Library\].*']
resource-regex = ['(?:.*\.)?(?P<resourceName>[^.]+?)\s*\[Robot-Resource\].*']
library-root = ["RF", "RF-Library"]
resource-root = ["RF-Resource"]
fully-qualified = false
output-directory = "{root}/Generated"
log-suite-numbering = false
clean = true
compound-keyword-logging = GROUP
resource-directory = "{root}/Resources"
resource-directory-regex = ".*\\[Robot-Resources\\].*"
reference-behaviour = "ATTACHMENT"
attachment-conflict-behaviour = "USE_EXISTING"

[tool.testbench2robotframework.library-mapping]
SeleniumLibrary = "SeleniumLibrary timeout=10 implicit_wait=1 run_on_failure=Capture Page Screenshot"
SuperRemoteLibrary = "Remote http://127.0.0.1:8270 WITH NAME SuperRemoteLibrary"

[tool.testbench2robotframework.resource-mapping]
MyKeywords = "{root}/../MyKeywords.resource"
MyOtherKeywords = "{resourceDirectory}/subdir/MyOtherKeywords.resource"

[tool.testbench2robotframework.forced-import]
libraries = ["test.py"]
resources = []
variables = []

[tool.testbench2robotframework.console-logging]
logLevel = "INFO"
logFormat = "%(levelname)s: %(message)s"

[tool.testbench2robotframework.file-logging]
logLevel = "DEBUG"
logFormat = "%(asctime)s - %(filename)s:%(lineno)d - %(levelname)8s - %(message)s"
fileName = "testbench2robotframework.log"
```
- Python 3.10 or higher
- TestBench version >= 4
1 change: 0 additions & 1 deletion atest/robot/libs/json_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,4 +43,3 @@
def create_json_configuration_file(path: Path):
with path.open("w") as json_file:
json.dump(data, json_file, indent=2)

11 changes: 3 additions & 8 deletions atest/robot/libs/pyproject_config.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,8 @@
import tomli_w

data = {
"tool": {
"testbench2robotframework": {
"generationDirectory": "{root}/toml_config_tests"
}
}
}
data = {"tool": {"testbench2robotframework": {"generationDirectory": "{root}/toml_config_tests"}}}

Comment on lines 1 to +4

def create_toml_configuration_file():
with open("pyproject.toml", "w") as toml_file:
toml_file.write(tomli_w.dumps(data))
toml_file.write(tomli_w.dumps(data))
12 changes: 8 additions & 4 deletions create_json_schema.py
Original file line number Diff line number Diff line change
@@ -1,20 +1,24 @@
import json
import re


with open("testbench2robotframework/model.py", "r", encoding="utf8") as model_py:
with open("testbench2robotframework/model.py", encoding="utf8") as model_py:
model_str = model_py.read()

pydantic_model = re.sub(r"(@dataclass\n)(class .*?)(:)", r"\2(BaseModel)\3", model_str)
pydantic_model = re.sub(r"( {4}@classmethod.*?)(\nclass|$)", r"\2", pydantic_model, flags=re.DOTALL)
pydantic_model = re.sub(r"from dataclasses import dataclass", r"from pydantic import BaseModel", pydantic_model, flags=re.DOTALL)
pydantic_model = re.sub(
r"from dataclasses import dataclass",
r"from pydantic import BaseModel",
pydantic_model,
flags=re.DOTALL,
)


with open("pydantic_model.py", "w", encoding="utf8") as pydantic_model_py:
pydantic_model_py.write(pydantic_model)

from pydantic_model import *
from yaml import dump, Dumper
from yaml import Dumper, dump

with open("model.json", "w") as schema:
schema.write(json.dumps(AllModels.model_json_schema(), indent=2))
Expand Down
4 changes: 4 additions & 0 deletions docs/configuration/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"label": "Configuration",
"position": 4
}
Loading