Local RAG implementation for skills discovery when they grow past a certain limit

[kyteinsky]

How to use GitHub

• Please use the 👍 reaction to show that you are interested into the same feature.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

---

Feature request

Which Nextcloud Version are you currently using: 34.0.0

Is your feature request related to a problem? Please describe.
The skills' names and descriptions are extracted and put in the context of the model for the skills discovery by the model. For 20 skills, this could roughy take 5500 tokens, or 2750 tokens on average (see #170 (comment)).
But skills have many uses from documentation for process automation to document writing adapted from a style, not to mention they are combined with the admin provided skills too. See #170 (comment)

Describe the solution you'd like
One of the ideas could be to use local RAG to not inject everything but provide skill results to the model based on need.
The other would be to use subagents that will see all the skills (maybe could also be implemented for tools) and respond with the appropriate ones, which the main model can then call directly.

Describe alternatives you've considered

1. Limiting the no. of skills injected
2. Only injecting the skills names instead of names + short descriptions

Additional context
Skills PR: #206
Discussion: #170

[TheBest-Name]

Okay I have a few more ideas one being when the docker container is first put up, have a config option to either have a local version or an external version and possibly a third version as well local will just download it separately so that way if you choose an external backend it doesn't bloat the docker container.
Okay the second option for an external would be to borrow the implementation for the context agent backend if you could coordinate with them and just kind of borrow their functionality then that would be one less thing you would have to run however it would be completely optional if you select that option for that back end

That would make it a lot more accessible and then you can choose what kind of model and everything else you want to run, I would assume.

Of course it would be the same thing for the small LLM idea you could just do a llm from the llm container or else you could choose a external LLM provider or use a local small model.

And this could be me being stupid but what if you could also think of using both again this may be me going a little crazy here but it might be something to assume using both that way the first model even if it doesn't explain it well the second model can then kind of clean up the explanation of what the tool's supposed to do before going into the vector search, I'm not sure how much that would help but it may be something.

Either way it would still be nice to have it be external because then you can control how many tokens get used and you can throttle or whatever else you need to do to control your CPU and GPU and credits if you use credits.

Thank you for your consideration!

[kyteinsky]

yeah having an external endpoint/service sounds like a good idea too.

[TheBest-Name]

I'm just curious because this thought just came to me, if you were to put it to an external backend would you also be able to use an external backend for the skills themselves to or to be able to hook external skills also up to that same backend?

I was just thinking because then you could have the next cloud skills but then you could also have skills that are from different other accounts kind of almost like an MCP server for skills.
You could link up and reuse your skills for multiple different AIs using the same backend.

Also for an external API, what API would you want to choose? Like an openAI? As you would put more support to it also would you be adding other APIs for different backends or is that usually something that they all share nowadays?