Hansung Kim 72b6004e24 flash: Fix online softmax for warp-specialized ...

Note: now that threads_per_threadblock is passed as compile-time
constant, the compiler likes to completely loop unroll which can cause a
lot of stack spills.

todo fix GEMM part.

2024-08-29 21:50:02 -07:00

..

.gitignore

flash: gitignore

2024-08-15 21:04:59 -07:00

common.h

flash: Change kernel arg to contain qkv; strip stimulus gen from host code

2024-08-15 21:03:02 -07:00

half.hpp

Add flash attention kernel skeleton

2024-08-14 20:46:09 -07:00

kernel.cpp

flash: Fix online softmax for warp-specialized

2024-08-29 21:50:02 -07:00

main.cpp

flash: Fix load addr for V tile; test with seqlen=128

2024-08-20 14:34:09 -07:00

Makefile

Add flash attention kernel skeleton

2024-08-14 20:46:09 -07:00