-
Notifications
You must be signed in to change notification settings - Fork 2
All issues
Issue creation is restricted in this repository
Issues
is:issue state:open
is:issue state:open
Search results
An OpenAI-compatible server: streaming is the blocker, and a model instance holds one conversation
enhancementNew capabilityNew capabilityStatus: Open.#199 In xoloki/jlib;K-quants have no device path: a 1.7 GB Q4_K_M costs more memory and 2.5x the load of the 2.8 GB q8_0
enhancementNew capabilityNew capabilityStatus: Open.#196 In xoloki/jlib;The attention scratch is still O(seq^2): 627 MB at 2048, 2.4 GB at 4096, against a model advertising 131072
enhancementNew capabilityNew capabilityStatus: Open.#195 In xoloki/jlib;About 100 us a layer, ~21% of a decode step, is in no kernel -- probably dispatch overhead, and unmeasurable on this device
enhancementNew capabilityNew capabilityStatus: Open.#194 In xoloki/jlib;jalpaca draws a character that arrives as byte-fallback tokens one byte at a time
bugSomething is wrongSomething is wrongStatus: Open.#185 In xoloki/jlib;Nothing checks the forward pass: no logit comparison against a reference implementation
enhancementNew capabilityNew capabilityStatus: Open.#184 In xoloki/jlib;Sliding-window attention: Gemma 2 alternates local and global layers, and jlib attends to everything
enhancementNew capabilityNew capabilityStatus: Open.#181 In xoloki/jlib;The reference tokenizer normalises to NFC and jlib does not, so decomposed text tokenizes differently
bugSomething is wrongSomething is wrongStatus: Open.#176 In xoloki/jlib;- Status: Open.#171 In xoloki/jlib;
- Status: Open.#166 In xoloki/jlib;
- Status: Open.#144 In xoloki/jlib;
- Status: Open.#143 In xoloki/jlib;