Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Model Context Protocol (MCP) server that enables AI assistants (Claude, Cursor,
- **Automatic API Sessions** - groups MCP changes in Testomat.io history using API sessions
- **Run Management** - status transitions via `status_event` parameter
- **TQL-Only Search** - `tests_list/tests_search` and `runs_list/runs_search` use `tql` as the single search/filter input
- **Built-In TQL Reference** - MCP tool descriptions include exact TQL fields, syntax, and examples for agents
- **Built-In TQL Reference** - TQL parameters include the exact field whitelist and examples; `tql_help` provides syntax details on demand

## Quick Start

Expand Down Expand Up @@ -257,7 +257,7 @@ NODE_EXTRA_CA_CERTS=/path/to/company-root-ca.pem testomatio-mcp --token <TOKEN>
- **Search** - No dedicated `/search` endpoints. MCP search tools delegate to list tools; for `tests` and `runs` the MCP interface is intentionally simplified to `tql`, while other entities stay closer to Public API v2 filters
- **TQL** - Use `tql` as the single search/filter input for `tests_list/tests_search` and `runs_list/runs_search`
- **TQL Syntax** - For user-facing syntax details and more examples, see the official TQL docs: https://docs.testomat.io/advanced/tql/
- **TQL Scope** - The full agent-oriented whitelist of documented fields lives inside MCP tool descriptions for `tests` and `runs`
- **TQL Scope** - TQL parameter descriptions keep the documented field whitelist in-band; call `tql_help` for syntax details and additional examples
- **Issue Linking** - Scoped helpers available: `{entity}_issues_link/unlink`
- **Attachments** - Scoped helpers available for tests, suites, and testruns: `{entity}_attachments_list/upload/delete`. Upload sends one local file path as multipart field `file`.
- **Enterprise Package** - Analytics tools are intentionally exposed only by `@testomatio/mcp-enterprise`, not by the standard `@testomatio/mcp` package
Expand Down
14 changes: 13 additions & 1 deletion docs/tools.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Tools Reference

Complete reference for all 90 MCP tools available in the Testomat.io MCP Server.
Complete reference for the MCP tools available in the Testomat.io MCP Server.

## Table of Contents

Expand Down Expand Up @@ -45,6 +45,18 @@ Check server status and active configuration.

---

### tql_help

Full TQL (Testomat.io Query Language) reference — syntax, filter variables (tests + runs), and examples.

**Usage:** Look up TQL syntax and the available filter fields before composing a `tql`/`q` filter for tests, runs, plans, or analytics.

**Parameters:** None

**Returns:** the complete TQL reference (syntax, tests variables, runs variables, examples).

---

## Test Management

### tests_list
Expand Down
8 changes: 8 additions & 0 deletions src/mcp/definitions/system.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,13 @@ export const SYSTEM_TOOLS = [
"type": "object",
"properties": {}
}
},
{
"name": "tql_help",
"description": "Full TQL (Testomat.io Query Language) reference — syntax, filter variables (tests + runs), and examples. Call this before writing a `tql` or `q` filter.",
"inputSchema": {
"type": "object",
"properties": {}
}
}
];
82 changes: 31 additions & 51 deletions src/mcp/definitions/tql-reference.js
Original file line number Diff line number Diff line change
Expand Up @@ -89,62 +89,42 @@ const RUNS_TQL_EXAMPLES = [
const COMMON_TQL_SYNTAX =
"Supported syntax includes logical operators `and`, `or`, `not`, equality operators `==` and `!=`, list membership `in [...]`, `%` for partial text match on supported text fields, and parentheses for grouping. Ordered comparisons `>`, `<`, `>=`, `<=` are for ordered fields such as `priority`, dates, and numeric counters/durations. Use quotes for string values, for example `state == 'automated'`.";

export const TESTS_TQL_REFERENCE =
`TQL (Testomat.io Query Language) is a string expression passed in \`tql\` to filter tests. ${COMMON_TQL_SYNTAX} ` +
`Documented test variables: ${TESTS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Documented examples: ${TESTS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}. ` +
'Do not invent undocumented fields or syntax. If a query fails, simplify it to one documented predicate.';
const TESTS_TQL_FIELDS = TESTS_TQL_VARIABLES.join(', ');
const RUNS_TQL_FIELDS = RUNS_TQL_VARIABLES.join(', ');

export const TESTS_TQL_INPUT_DESCRIPTION =
`TQL filter for tests. Documented variables: ${TESTS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Examples: ${TESTS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}.`;
export const TQL_FULL_REFERENCE = [
'TQL (Testomat.io Query Language) is a string expression used to filter tests and runs.',
COMMON_TQL_SYNTAX,
'',
`Tests filter variables: ${TESTS_TQL_VARIABLES.map((v) => `\`${v}\``).join(', ')}.`,
`Tests examples: ${TESTS_TQL_EXAMPLES.map((e) => `\`${e}\``).join(', ')}.`,
'',
`Runs filter variables: ${RUNS_TQL_VARIABLES.map((v) => `\`${v}\``).join(', ')}.`,
'Runs also support boolean flags used without comparison, e.g. `failed`, `finished`, `automated`, `with_defect`.',
`Runs examples: ${RUNS_TQL_EXAMPLES.map((e) => `\`${e}\``).join(', ')}.`,
'',
'Parameter name: tests/runs/plans use `tql`; analytics tools use `q`.',
'Do not invent undocumented fields or syntax. If a query fails, simplify it to one documented predicate.',
].join('\n');

export const TESTS_TQL_REFERENCE =
'Filter tests with `tql` (TQL); call `tql_help` for the syntax and full field list.';
export const RUNS_TQL_REFERENCE =
`TQL (Testomat.io Query Language) is a string expression passed in \`tql\` to filter runs. ${COMMON_TQL_SYNTAX} ` +
'Runs also support boolean flags without comparison such as `failed`, `finished`, `automated`, or `with_defect`. ' +
`Documented run variables: ${RUNS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Documented examples: ${RUNS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}. ` +
'Do not invent undocumented fields or syntax. If a query fails, simplify it to one documented predicate.';

export const RUNS_TQL_INPUT_DESCRIPTION =
`TQL filter for runs. Documented variables: ${RUNS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Examples: ${RUNS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}.`;

'Filter runs with `tql` (TQL); runs also accept boolean flags. Call `tql_help` for the syntax and full field list.';
export const PLANS_TQL_REFERENCE =
`TQL (Testomat.io Query Language) is a string expression passed in \`tql\` to select tests included in a plan. ${COMMON_TQL_SYNTAX} ` +
`Plan TQL uses documented test variables: ${TESTS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Documented examples: ${TESTS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}. ` +
'Use `tql` when you want the API to resolve matching tests automatically instead of sending explicit `test_ids`. Do not invent undocumented fields or syntax. If a query fails, simplify it to one documented predicate.';

export const PLANS_TQL_INPUT_DESCRIPTION =
'TQL filter for selecting tests included in the plan. ' +
`Documented test variables: ${TESTS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Examples: ${TESTS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}.`;

'Select tests for the plan with `tql` (TQL); the API resolves matching tests. Call `tql_help` for the syntax and full field list.';
export const ANALYTICS_TESTS_TQL_REFERENCE =
`TQL (Testomat.io Query Language) is a string expression passed in \`q\` to filter enterprise analytics test reports. ${COMMON_TQL_SYNTAX} ` +
'For analytics tools, the API parameter name is `q`, not `tql`. ' +
`Documented analytics test variables: ${TESTS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Documented examples: ${TESTS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}. ` +
'Do not invent undocumented fields or syntax. If a query fails, simplify it to one documented predicate.';

export const ANALYTICS_TESTS_TQL_INPUT_DESCRIPTION =
'TQL filter for analytics test reports. The API parameter name is `q`, not `tql`. ' +
`Documented variables: ${TESTS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Examples: ${TESTS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}.`;

'Filter analytics test reports with `q` (TQL). Call `tql_help` for the syntax and full field list.';
export const ANALYTICS_STATS_TQL_REFERENCE =
`TQL (Testomat.io Query Language) is a string expression passed in \`q\` to filter enterprise analytics aggregated reports. ${COMMON_TQL_SYNTAX} ` +
'For analytics tools, the API parameter name is `q`, not `tql`. ' +
'According to the official Analytics docs, analytics queries are configured using supported query variables for two data sources: `Tests Variables` and `Runs Variables`. ' +
`Documented Tests Variables: ${TESTS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Documented Runs Variables: ${RUNS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Documented Tests examples: ${TESTS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Documented Runs examples: ${RUNS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}. ` +
'Use Tests Variables for test-centric filters and Runs Variables for run-centric filters such as `plan`, `rungroup`, `env`, `finished_at`, or `has_test_tag`. Do not invent undocumented fields or syntax. If a query fails, simplify it to one documented predicate.';
'Filter analytics aggregated reports with `q` (TQL; tests or runs variables). Call `tql_help` for the syntax and full field list.';

export const TESTS_TQL_INPUT_DESCRIPTION =
`TQL filter for tests. Fields: ${TESTS_TQL_FIELDS}. Call \`tql_help\` for syntax. Examples: \`priority == 'high'\`, \`state == 'automated'\`, \`suite % 'Checkout'\`.`;
export const RUNS_TQL_INPUT_DESCRIPTION =
`TQL filter for runs. Fields: ${RUNS_TQL_FIELDS}. Call \`tql_help\` for syntax. Examples: \`finished and with_defect\`, \`env in ['Windows', 'Linux']\`, \`has_retries > 2\`.`;
export const PLANS_TQL_INPUT_DESCRIPTION =
`TQL to select tests for the plan. Fields: ${TESTS_TQL_FIELDS}. Call \`tql_help\` for syntax. Examples: \`priority == 'high'\`, \`tag in ['smoke', 'stage1']\`.`;
export const ANALYTICS_TESTS_TQL_INPUT_DESCRIPTION =
`TQL filter (param \`q\`) for analytics test reports. Fields: ${TESTS_TQL_FIELDS}. Call \`tql_help\` for syntax. Examples: \`priority == 'high'\`, \`state == 'automated'\`.`;
export const ANALYTICS_STATS_TQL_INPUT_DESCRIPTION =
'TQL filter for analytics aggregated reports. The API parameter name is `q`, not `tql`. ' +
`Documented Tests Variables: ${TESTS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Documented Runs Variables: ${RUNS_TQL_VARIABLES.map((item) => `\`${item}\``).join(', ')}. ` +
`Examples: ${TESTS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}, ${RUNS_TQL_EXAMPLES.map((item) => `\`${item}\``).join(', ')}.`;
`TQL filter (param \`q\`) for analytics reports. Test fields: ${TESTS_TQL_FIELDS}. Run fields: ${RUNS_TQL_FIELDS}. Call \`tql_help\` for syntax. Examples: \`priority == 'high'\`, \`finished_at >= '2025-07-01' and failed\`.`;
2 changes: 2 additions & 0 deletions src/mcp/tool-registry.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { DEFAULT_TOOL_RESPONSE } from '../config/constants.js';
import { ApiError, NotImplementedToolError } from '../core/errors.js';
import { textResponse } from '../helpers/mcp-response.js';
import { TOOL_DEFINITIONS } from './tool-definitions.js';
import { TQL_FULL_REFERENCE } from './definitions/tql-reference.js';
import { handlerMethods } from './registry/handlers.js';
import { attachmentMethods } from './registry/attachments.js';
import { issueMethods } from './registry/issues.js';
Expand Down Expand Up @@ -35,6 +36,7 @@ export class ToolRegistry {
baseUrl: this.config.baseUrl,
apiVersion: 'v2',
}),
tql_help: async () => textResponse(TQL_FULL_REFERENCE),
};

this.registerEntityCrudHandlers(handlers);
Expand Down