Skip to content

kernel: optimize attention kernel performance#377

Merged
guocuimi merged 6 commits intomainfrom
attn_perf
Jan 18, 2025
Merged

kernel: optimize attention kernel performance#377
guocuimi merged 6 commits intomainfrom
attn_perf

Conversation

@guocuimi
Copy link
Collaborator

@guocuimi guocuimi commented Jan 17, 2025

1> use more static dispatch
2> use cutlass::FastDivmod for slot id calculation
3> handle oob for k (head_dim)

@guocuimi guocuimi merged commit 65b3c53 into main Jan 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant