Livt packages

Eccelerators.Tokenization

Version 0.2.0

Bounded BPE and Unigram tokenization, text normalization, and decoding components for Livt hardware.

Published versions (1)

0.2.0

← Browse packages

Install this version

bash

livt add "Eccelerators.Tokenization@0.2.0"

About this version

AuthorsDenis Vasilík

Organization Eccelerators GmbH

LicenseMIT

SourceRepository

ProjectHomepage

Keywordsfpga, tokenization, bpe, unigram, text

This site uses cookies to give you the best possible experience, and by continuing on this site you agree we can save them on your device. We use technical cookies only and do not personalize content or ads. To learn more, follow the link to our privacy statement.