tokenkit

Name: tokenkit
Author: Chris Petersen

Authors: Chris Petersen

License: MIT

Summary: TokenKit provides lightweight, Unicode-aware word-level tokenization with pattern preservation, backed by Rust for performance.

Can't find the visual you need? Build your own here

Downloads

by period

last day

last week

last month

551

total

All

version

0.1.0.pre.1tokenkit

latest version

RubyGem Downloads Over Time