LettuceAI 2.2.0 and 2.2.1 both crash when opening Settings → Models

[schlegan]

Describe the bug
Hello, I never used Lettuce AI before. Today I did a fresh install of the 2.2.1 windows, cpu version on windows 10 and tried to follow the setup process. When I opened the setting page for choosing a local LLM model as part of the setup process, the whole app closed after a few seconds, regardless of what I do. There is no error message and this problem persists after restarts, fresh installs and repeated attempts.

I've tried to debug via Chat-GPT and it describes the problem as follows:

Windows 10/11, CPU Intel Xeon E5-2670 v2 (Ivy Bridge, AVX but no BMI2). LettuceAI 2.2.0 and 2.2.1 both crash when opening Settings → Models, before selecting a model. Event Viewer reports exception 0xc000001d. WinDbg identifies the faulting instruction as shlx edx,r8d,edx at lettuceai!onig_builtin_fail+0x1157bb. SHLX is a BMI2 instruction unsupported by Ivy Bridge. The Windows CPU build appears to have been compiled with BMI2 instructions enabled.

That onig_... naming is from Oniguruma, a regular-expression library. onig_builtin_fail is indeed an Oniguruma function.

To Reproduce
Steps to reproduce the behavior, as they have happend on my machine:

1. Have windows 10 with an Intel Xeon E5-2670 v2 CPU or any other cpu that doesn't support BMI2 instructions.
2. Install LettuceAI 2.2.1 fresh.
3. Open LettuceAI.
4. Click 'Get Started'.
5. Click 'Yes, I know how this works'.
6. Click 'I want to use local LLMs'.
7. Click 'use my own gguf files'.
8. Wait a few seconds and LettuceAI should close on its own.

Expected behavior
For everything to keep working or at least an error message.

Additional context

• The CUDA version has the same problem.

[MegalithOfficial]

Im not sure if llama.cpp itself supports that CPU.

[schlegan]

Im not sure if llama.cpp itself supports that CPU.

I've tried to look that up. The cpu itself support the AVX instruction set, which is a supported option in the llama.cpp makefile as seen under: https://github.com/ggml-org/llama.cpp/blob/master/ggml/CMakeLists.txt

Therefore it should be supported. Chat-GPT also says it's supported.

[MegalithOfficial]

Im not sure if llama.cpp itself supports that CPU.

I've tried to look that up. The cpu itself support the AVX instruction set, which is a supported option in the llama.cpp makefile as seen under: https://github.com/ggml-org/llama.cpp/blob/master/ggml/CMakeLists.txt

Therefore it should be supported. Chat-GPT also says it's supported.

Then you need compile the app locally using the right compile-time flags.

Since its a niche CPU, I wont be adding it into our compilers. It will increase the space the binaries use. No offense.

[schlegan]

Im not sure if llama.cpp itself supports that CPU.

I've tried to look that up. The cpu itself support the AVX instruction set, which is a supported option in the llama.cpp makefile as seen under: https://github.com/ggml-org/llama.cpp/blob/master/ggml/CMakeLists.txt
Therefore it should be supported. Chat-GPT also says it's supported.

Then you need compile the app locally using the right compile-time flags.

Since its a niche CPU, I wont be adding it into our compilers. It will increase the space the binaries use. No offense.

I understand, not biggie.

But, at least according to Chat-GPT, the problem isn't actually caused by llama.cpp, but by 'Oniguruma'. The dependency chain is LettuceAI -> tokenizers = "0.20" -> onig (Rust bindings for oniguruma) -> oniguruma (C library). I haven't taken a closer look, but onig is an optional dependency.

If it helps, I could try fixing that problem locally over the coming days and then post an update here, if that really was the culprit.

[MegalithOfficial]

Tokenizers library is not under my control. Its used for dynamic memory's embedding runtime.

If the issue caused by it, i suggest contacting their developers. Because for me to fix it, I would have to fork multiple repos and have more responsibilities than i supposed to have.

[schlegan]

Tokenizers library is not under my control. Its used for dynamic memory's embedding runtime.

If the issue caused by it, i suggest contacting their developers. Because for me to fix it, I would have to fork multiple repos and have more responsibilities than i supposed to have.

I apologize for the lacking clarity in my explanation: When including tokenizers as a dependency, you can simply turn the onig dependency off in Cargo.toml, as it's an optional dependency.

According to online sources, you would simply write tokenizers = { version = "0.20", default-features = false} instead of tokenizers = { version = "0.20"}. This turn all optional dependencies off, that are on by default. You can turn on selected features on again by listing them explicitly like this tokenizers = { version = "0.20", default-features = false, features = ["http"] }, for example.

No forking required or code changes to other repos required.