[ICCV 2025] DONUT: A Decoder-Only Model for Trajectory Prediction

Efficient encoder-decoder architecture for small language models (≤1B parameters) with cross-architecture knowledge distillation and vision-language capabilities

使用Decoder-only的Transformer进行时序预测，包含SwiGLU和RoPE(Rotary Positional Embedding)，Time series prediction using Decoder-only Transformer, Including SwiGLU and RoPE(Rotary Positional Embedding)

Code for paper "Modality Plug-and-Play: Elastic Modality Adaptation in Multimodal LLMs for Embodied AI"

🔍 Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment

Minimal decoder-only seq2seq pipeline with proper causal masking, teacher forcing, Ignite training loop, and checkpointed inference

SAMPO: Scale-wise Autoregression with Motion Prompt for Generative World Models

ViAG: A Novel Framework for Fine-tuning Answer Generation models ultilizing Encoder-Decoder and Decoder-only Transformers's architecture

A from-scratch implementation of a scaled-down GPT-2 model in PyTorch, trained on the Snappfood dataset for sentiment-controlled Persian text generation.

Developed and pre-trained a 20.39M-parameter Punjabi GPT-style base model from scratch, including corpus preparation, tokenizer training, benchmark evaluation, and text generation, using a cleaned Punjabi corpus and local Apple Silicon GPU acceleration.

Clean-room GPT-2/GPT-3 implementation: tokenizers, architecture blocks, training loop with AdamW + cosine decay, CLI scripts, inference tools, and pytest suite. Covers OpenWebText-10k & WikiText-103 workflows. Designed as an academic reference for understanding and scaling decoder-only transformers

中文至英文序列转导模型。从零严格复现《Attention Is All You Need》在中文→英文机器翻译（Zh→En）上的完整流程。

Criando um modelo Transformer do zero com variações como Multi-Head Attention e Grouped Query Attention em livros de Machado de Assis.

A mini version of GPT implemented on shakespear using BPE

This is a compilation about excercises to learn how to implement a transformer model

A compact, readable GPT-style decoder-only Transformer implemented in pure PyTorch. The goal is to expose the essential architectural pieces with minimal scaffolding so you can train and tinker quickly.

Implementation of the GPT-2 architecture using PyTorch, trained on the TinyStories dataset. Features custom training pipelines on Modal (cloud computing) and integration with the Hugging Face ecosystem.

Auto regressive text generation application using decoder transformer

Decoder-only transformer, simplest character-level tokenization, training and text generation.