BRZEN
Byte-pair encoding
Texto da Wikipédia (en), licença CC BY-SA. O BETARUBI mostra o verbete inteiro nesta página — a leitura não continua fora do site.
In computing, byte-pair encoding (BPE), or digram coding, is an algorithm, first described in 1994 by Philip Gage, for encoding strings of text into smaller strings by creating and using a translation table. A slightly modified version of the algorithm is used in large language model tokenizers.
