Skip to content

feat: 开始实现 attention #89

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 17 commits into
base: dev
Choose a base branch
from
Draft

feat: 开始实现 attention #89

Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
cbe7c78
feat: 开始实现 attention
YdrMaster Feb 2, 2024
6d4465c
build(kernel): try to use local cccl instead of system
YdrMaster Feb 2, 2024
e54c7db
refactor(kernel): 实现一种不依赖模板参数的 BlockReduce 并用于 softmax
YdrMaster Feb 2, 2024
8e1564a
feat(kernel): 确认供 attention 调用的 softmax 接口
YdrMaster Feb 2, 2024
ccf293f
feat(kernel): 开始实现 softmax
YdrMaster Feb 2, 2024
20a34ac
feat(kernel): 完成不带 kv cache 的简单 attention
YdrMaster Feb 2, 2024
7286459
fix: 整理和改正 attention 构造问题
YdrMaster Feb 2, 2024
d7bbd3b
fix(kernel): 设置 sharedMemory
YdrMaster Feb 2, 2024
9f736e9
test(llm): 添加 attention 前端单测
YdrMaster Feb 2, 2024
4d47e5d
feat(kernel): 封装 attLen 计算
YdrMaster Feb 2, 2024
6211b92
test(kernel): 添加 attention 单测
YdrMaster Feb 2, 2024
1705126
fix(kernel): 解决 attention 访存错误的问题
YdrMaster Feb 2, 2024
f5d9756
fix(kernel): 仍然使用 cub::BlockReduce 并改正 Attention
YdrMaster Feb 2, 2024
e16c679
feat(kernel): 支持 kv cache attention
YdrMaster Feb 2, 2024
bfa8e9f
fix (kernel): 修复attention算子中的concat
PanZezhong1725 Feb 21, 2024
a686d2f
temp
YdrMaster Feb 21, 2024
0d1fce4
temp
YdrMaster Feb 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
temp 
Signed-off-by: YdrMaster <[email protected]>
  • Loading branch information
@YdrMaster
YdrMaster committed Feb 21, 2024
commit 0d1fce4d50b808e6f5b2be9f5ba2a4262c96371b
Loading