Optimize Viterbi cost calculation by batching word processing

• Introduced a batch processing approach in the Viterbi algorithm to improve performance and cache locality.
• Added buffers to the Lattice struct to store word entries and preceding state costs in a Struct-of-Arrays (SoA) format during processing.
• Implemented add_edges_in_lattice_batched to process multiple candidate words starting at the same position in a single pass over preceding edges.
• Optimized the Mode::Normal path by inlining cost calculations and skipping penalty overhead.
• Reduced memory allocation and pointer chasing by reusing internal buffers within the Lattice structure.
• Achieved significant performance improvements across various scenarios:
  • Up to 37% speedup when using user dictionaries.
  • 13% speedup for long text tokenization with detailed output.
  • 7-9% improvement in standard tokenization scenarios.