niansa/libjustlm
1
0
Fork 0
mirror of https://gitlab.com/niansa/libjustlm.git synced 2025-03-06 20:49:17 +01:00
Code Releases Activity

Convert tokens to text correctly in llama

Browse source
This commit is contained in:
niansa 2023-08-31 18:23:55 +02:00
parent 907cea7f9d
commit 3a953ed13a
1 changed files with 2 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
justlm_llama.hpp
View file

@ -231,7 +231,8 @@ public:
LM_COAWAIT window_scroll();
// Get token as string
const std::string_view str = llama_token_get_text(state->ctx, id);
std::string str(14, ' ');
str.resize(llama_token_to_piece(state->ctx, id, str.data(), 14));
// Append string to function result
state->prompt.append(str);