Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,15 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.

## Unreleased

### A message no longer routes to a specialist because a longer word contained a connector's name

When the intent router falls back — it is unreachable, or it declines — and exactly one coworker can
reach a system the message names, the message goes to that coworker. The name was matched as a bare
substring, so "how do I deal with a slacker" matched the **slack** connector and "una jirafa" (a
giraffe) matched **jira**: a message naming neither system was pinned, for the life of the thread,
to a specialist that could not answer it. A connector's name now has to appear on a word boundary,
so a system named on its own still routes and one buried inside another word does not.

### A Bot's shell can no longer reach the embedded database without a password

In the all-in-one image the cluster was `trust`-auth on loopback, and the Bot's shell runs in the
Expand Down
36 changes: 28 additions & 8 deletions server/src/routing/classify.ts
Original file line number Diff line number Diff line change
Expand Up @@ -123,16 +123,40 @@ export function routingPrompt(
].join("\n");
}

/**
* Whether a message names a system, by its id or by that id with separators loosened.
*
* Matched at word boundaries rather than as a bare substring. `haystack.includes("slack")` is true
* of "how do I handle a slacker", and `includes("jira")` is true of the Spanish for giraffe,
* "jirafa" — so a message that names neither system was read as naming one, and in a fallback the
* whole conversation was pinned to that specialist. A system id has to sit on its own here: bounded
* by a non-alphanumeric character or an edge of the message, not buried inside a longer word.
*
* The id is still matched with separators loosened, so `google-drive` answers to "google drive" as
* somebody would type it, and its raw form is matched too. Deliberately not fuzzy beyond that: a
* router that guesses at near-misses is a router nobody can predict.
*/
function messageNames(haystack: string, system: string): boolean {
const spelled = system.toLowerCase().replace(/[-_]+/g, " ");
return bounded(haystack, spelled) || bounded(haystack, system.toLowerCase());
}

/** `needle` present in `haystack`, on word boundaries, with `needle`'s own characters taken literally. */
function bounded(haystack: string, needle: string): boolean {
if (!needle) return false;
const escaped = needle.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
return new RegExp(`(?:^|[^a-z0-9])${escaped}(?:[^a-z0-9]|$)`).test(haystack);
}

/**
* The one coworker that can reach a system this message names, when there is exactly one.
*
* A hint for the router became a decision for the fallback, and only there. A confident match on
* purpose still wins: a specialist with no connectors is the right answer to a question about its
* specialism, and this must not turn reach into a filter that overrides that.
*
* Matched on the system's own id with separators loosened, so `google-drive` answers to "Google
* Drive" as somebody would type it. Deliberately not fuzzy beyond that: a router that guesses at
* near-misses is a router nobody can predict.
* A message names a system by {@link messageNames}: its id, or that id with separators loosened, at
* a word boundary.
*/
function onlyCoworkerReaching(
text: string,
Expand All @@ -142,11 +166,7 @@ function onlyCoworkerReaching(
const named = new Set<string>();
for (const candidate of candidates) {
for (const system of candidate.reaches ?? []) {
const spelled = system.toLowerCase().replace(/[-_]+/g, " ");
if (
haystack.includes(spelled) ||
haystack.includes(system.toLowerCase())
) {
if (messageNames(haystack, system)) {
named.add(system);
}
}
Expand Down
48 changes: 48 additions & 0 deletions server/tests/routing-classify.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,54 @@ describe("falling back to somebody who can actually answer", () => {
expect(decision.agentId).toBe("general-assistant");
});

/*
* A system id buried inside a longer word does not name the system.
*
* `includes("slack")` is true of "slacker" and `includes("jira")` is true of "jirafa" — the
* Spanish for giraffe — so a message that names neither system was read as naming one and, in a
* fallback, pinned the whole conversation to that specialist. The id has to sit on its own.
*/
const NAMED: RoutingCandidate[] = [
{ id: "general", name: "General", roleDescription: "everyday work" },
{
id: "slackbot",
name: "Slack",
roleDescription: "chat",
reaches: ["slack"],
},
{
id: "jirabot",
name: "Jira",
roleDescription: "tickets",
reaches: ["jira"],
},
];

test("a system id inside an unrelated word is not a match", async () => {
const slacker = await BROKEN.route(
"how do I deal with a slacker on my team",
NAMED,
"general",
);
expect(slacker.agentId).toBe("general");

const giraffe = await BROKEN.route(
"escribe un cuento sobre una jirafa",
NAMED,
"general",
);
expect(giraffe.agentId).toBe("general");
});

test("the same system named on its own still routes to its holder", async () => {
const decision = await BROKEN.route(
"post this update to slack for me",
NAMED,
"general",
);
expect(decision.agentId).toBe("slackbot");
});

test("uses the default when two coworkers reach the same system", async () => {
// Not a decision this can make. Two holders is exactly the case the router is for.
const shared: RoutingCandidate[] = [
Expand Down