Hansung Kim
c26558bc93
Add fence before rescale
2025-01-28 23:48:02 -08:00
Hansung Kim
7d7cb5f60a
flash: Disable perf loop multiplier
2024-11-10 22:44:02 -08:00
Hansung Kim
4448f31fdc
fence: Fix moving fence to start of loop
...
For unknown reasons, guarding the fence with a tid == 0 branch causes a
TL source ID re-used assertion. Just call the fence from all
thread/warps as a workaround. At least, all threads in a warp will
coalesce into one request.
2024-11-09 22:04:45 -08:00
Hansung Kim
68054689c9
flash: Move fence to start of loop; wrap all MMIO in one tid=0 branch
2024-11-09 20:59:26 -08:00
Hansung Kim
fcd8b0b892
flash: Disable rescale flag check
...
GEMM-II finishes much earlier than softmax for this to be a problem.
2024-11-09 20:37:58 -08:00
Hansung Kim
1c9b022156
flash: Rename nowarpspec to default
2024-11-09 19:58:45 -08:00
Hansung Kim
76a6aaf085
flash: doc update
2024-11-09 19:09:09 -08:00
Hansung Kim
ad75561efe
flash: Reduce fence calls to improve util
2024-11-09 16:44:17 -08:00
Hansung Kim
952b8debbb
flash: Update to use new CISC interface
2024-11-09 16:21:34 -08:00
Hansung Kim
365b1d8e67
flash: Add begin end markers
2024-11-09 10:19:16 -08:00
Hansung Kim
4e087a8aab
flash: Fix loop iteration for gemmini
...
Kernel is software-pipelined around 2 GEMMs and softmax; it requires two
iterations to fully complete a tile.
2024-11-08 16:43:08 -08:00
Hansung Kim
be15cffbf3
flash: Revert to gemmini config, remove DEBUG and unnecessary checks
2024-09-12 14:25:33 -07:00
Hansung Kim
d69707f686
flash: Enable GEMM II fence; Pull 1st KV move out of the loop
2024-09-11 19:24:06 -07:00
Hansung Kim
068d48534e
flash: Swap S1/S0 to avoid GEMM II - softmax bank conflict
...
+ remove spurrious fences to better overlap GEMM I and DMA
2024-09-11 00:55:36 -07:00
Hansung Kim
dc746272fb
flash: Conditionally enable GEMM II fence code, fix tile_k for DEBUG
2024-09-10 22:53:35 -07:00
Hansung Kim
90e03894fc
flash: Add flag in SMEM for dependency check on O
...
TODO: results unverified.
Stalls O rescale until GEMM II finishes.
2024-09-10 13:42:47 -07:00
Hansung Kim
88760596cb
flash: Remove bogus mvout to SMEM code
2024-09-09 17:18:59 -07:00
Hansung Kim
a17edac875
flash: Fix barrier stall with DEBUG
...
Verified for up to P_expected on 2nd iter; O_before_PV is partially
correct
2024-09-09 17:02:05 -07:00
Hansung Kim
b652e25945
flash: Warp-specialize between warp 0 and 1-7
...
Finishes without stalls; No dependency check between O rescale and
GEMM-II.
2024-09-09 16:42:30 -07:00
Hansung Kim
d31c8ffd7d
flash: Fix grid size to hw cluster size
...
Verified fast config, minus the barrier stall at the end.
2024-09-09 15:44:03 -07:00
Hansung Kim
829af5d429
flash: Comment out mvout to smem
...
Verified up to O_before_PV; still stalls without DEBUG
2024-09-09 15:21:49 -07:00
Hansung Kim
ecc800964a
flash: Change smem alloc for less bank conflicts; noskip stc
2024-09-09 13:47:18 -07:00
Hansung Kim
714b9f501e
flash: Restructure to do delayed fence for better concurrency
...
Verified up to O_before_PV of 2nd iteration; O_after_PV needs preload
fix.
FIXME: Stalls at barrier without DEBUG set.
2024-09-08 22:16:17 -07:00
Hansung Kim
6911843a82
flash: Remove unnecessary dmem preload, fix rowmax/rowsum dependency
2024-09-08 21:11:59 -07:00
Hansung Kim
a4dd45bc1b
flash: Replace CISC with RISC
...
spadQuartile in hw does not match spad addresses in kernel; match them
later for optimization.
2024-09-08 20:52:28 -07:00
Hansung Kim
6547e92757
flash: Load Q to both quartiles; preload O for acc
2024-09-08 19:47:55 -07:00
Hansung Kim
8efa6868ea
flash: Restructure for full software pipelining
...
Verified up to P and O before PV; need to fix iteration for V load.
2024-09-08 18:45:32 -07:00
Hansung Kim
cdb8377b62
flash: Do GEMM II in Gemmini; verify 1st iteration
2024-09-08 16:09:06 -07:00
Hansung Kim
3f50ac57ee
flash: use 12bit dma interface
2024-09-08 15:29:56 -07:00
Hansung Kim
c51dc4902d
flash: Fix online softmax for DMA layout
2024-09-07 23:21:28 -07:00
Hansung Kim
2e1485877d
flash: Add Gemmini-accelerated kernel
2024-09-07 22:40:58 -07:00
