Hansung Kim
95b5719847
sgemm_tcore: Split K-dim loop between consumer/producer
...
... so that you don't have to run (warpgroup_id == 0) condition at every
loop iteration which is expensive due to vx_split/join.
2024-06-05 17:15:30 -07:00
Hansung Kim
150f14af25
sgemm_tcore: Use multiple fp regs for GMEM->SMEM
2024-06-05 16:53:39 -07:00
Hansung Kim
e44173c65e
sgemm_tcore: Deconstruct addr calc for GMEM->SMEM
2024-06-05 15:11:01 -07:00
Hansung Kim
ff6e5bf6dc
sgemm_tcore: Deconstruct smem addr calc to reduce reg alloc
2024-06-05 14:50:36 -07:00
Hansung Kim
d8944db369
sgemm_tcore: Double-buffer over K-dimension
...
TODO: Not completely parameterized with DOUBLE_BUFFER yet.
2024-06-04 18:23:27 -07:00
Hansung Kim
18e3653d31
sgemm_tcore: Increase RF data reuse for WMITER/WNITER
...
... by splitting vx_wmma_load to vx_wmma_load_{a,b} and pulling it
out of the innermost loop.
TODO: there's some duplicate address compute being done in the both
functions.
2024-06-03 21:10:42 -07:00
Hansung Kim
4e723c4655
sgemm_tcore: Support two accumulation reg tiles
2024-06-01 01:12:38 -07:00
Hansung Kim
c8d6c56dd9
sgemm_tcore: Split global DMEM load into a function
2024-05-31 17:37:06 -07:00
Hansung Kim
220ee0aa5e
sgemm_tcore: Unroll around WMITER/WNITER
...
This is within a very tight loop so it's worth unrolling at the risk of
stack spills somewhere else.
2024-05-31 17:35:22 -07:00
Hansung Kim
2b5836022d
Also generate kernel.CONFIG.elf
2024-05-26 13:57:07 -07:00
Hansung Kim
1e48bad4f9
sgemm_tcore: Fix AS transpose
2024-05-26 13:51:47 -07:00
Hansung Kim
200fd3e08c
sgemm_tcore: Revert to packed smem alloc
2024-05-25 22:47:59 -07:00
Hansung Kim
bc7bd1a1dd
sgemm_tcore: Write reference C matrix to file
2024-05-25 22:47:15 -07:00
Hansung Kim
b892c22f00
sgemm_tcore: Reflect WMITER/WNITER in threadblock size
2024-05-16 23:31:52 -07:00
Hansung Kim
18ecebddc0
sgemm_tcore: Fix round-down error with CORES_PER_CLUSTER
2024-05-16 21:36:24 -07:00
Hansung Kim
78b2a318c1
sgemm_tcore: Implement A transpose for coalesced smem access
2024-05-16 20:22:15 -07:00
Hansung Kim
8f64fae7a7
sgemm_tcore: Addr gen for local_k; add SIMT-only for reference
2024-05-16 14:11:09 -07:00
Hansung Kim
df1aa62916
sgemm_tcore: Add warptiling parameters
...
FIXME: accumulation is done wrong
2024-05-15 15:23:26 -07:00
Hansung Kim
5de8e7c33a
sgemm_tg: Fix device address to use ELF operands
2024-05-13 23:09:57 -07:00
Hansung Kim
9d2b533d5c
sgemm_tg: Do operand elf stitching for kernel.elf as well
2024-05-13 16:48:13 -07:00
Hansung Kim
09b23ffe87
sgemm_tg: 1-octet 8-lane kernel
2024-05-13 14:52:33 -07:00
Hansung Kim
d848e88f72
sgemm_tcore: Move C from regF->GMEM directly
2024-05-13 14:00:50 -07:00
Hansung Kim
9e60b1834c
sgemm_tcore: Rewrite with sgemm_Wg parametrization
2024-05-13 13:22:06 -07:00
Hansung Kim
5c298c81df
sgemm_tg: Use reg mapping functions
2024-05-12 22:22:54 -07:00
Hansung Kim
6af0c305ea
Fix path to OBJCOPY
2024-05-08 13:27:11 -07:00
Hansung Kim
6ba6a1e2e5
Merge branch 'kernels' into tensor_core
2024-05-08 13:25:31 -07:00
Hansung Kim
7775830814
Hardcode chipyard device addresses
2024-05-07 16:30:30 -07:00
joshua
5bd25985c6
i kinda forgot most of changes
2024-05-04 23:01:47 -07:00
Hansung Kim
a606a9ef42
common.mk: properly handle unspecified CONFIG
2024-04-29 17:14:28 -07:00
Richard Yan
01f4a69ae9
dma mvout, double buffering & other opts
2024-04-28 01:18:51 -07:00
Richard Yan
d21e7b92c7
internal accumulation, forced rematerialization, better unrolling
2024-04-25 15:28:12 -07:00
Richard Yan
a44edf2b65
Merge branch 'kernels' of https://github.com/hansungk/vortex-private into kernels
2024-04-24 22:10:40 -07:00
Richard Yan
6eafa2de54
write operands to elf
2024-04-24 22:09:30 -07:00
Hansung Kim
df881fd69f
Generate separate ELF for radiance
2024-04-24 21:10:21 -07:00
Hansung Kim
793779aa6c
sgemm_wg: 128x128 config
2024-04-24 21:10:21 -07:00
Hansung Kim
689043b45e
Add regression flops
2024-04-24 21:10:21 -07:00
Hansung Kim
6cbfbfb856
sgemm_wg: Output CPU data to binary
2024-04-24 21:10:21 -07:00
Richard Yan
4e9855dc33
highly unrolled a/b load
2024-04-16 22:19:30 -07:00
Richard Yan
449d99f0bb
dram gemm kernel
2024-04-16 17:15:22 -07:00
Richard Yan
99621a0df9
Merge branch 'kernels' of https://github.com/hansungk/vortex-private into kernels
2024-04-15 10:22:19 -07:00
Richard Yan
0bb7aeb45b
add gpu+gemmini gemm kernel
2024-04-15 10:13:37 -07:00
Hansung Kim
37a60b1141
sgemm_wg: Output C result to binary
2024-04-14 12:36:06 -07:00
Hansung Kim
3383b70732
sgemm_wg: Hardcode device address
2024-04-14 12:36:00 -07:00
Hansung Kim
93a00101ae
sgemm_wg: revert to faster params
2024-04-04 21:06:14 -07:00
Hansung Kim
fa2b6e2ad0
sgemm_wg: Explicitly limit unroll to reduce stack spilling
...
This needs to be done case-by-case for different BK/TM/TN combinations and
examining the assembly.
2024-03-29 02:48:29 -07:00
Hansung Kim
537b97eb20
common.mk: Don't clean all *.elf
2024-03-28 20:17:26 -07:00
Hansung Kim
a9b0814211
sgemm_wg: Document tiling parameter constraints
2024-03-28 18:17:00 -07:00
Hansung Kim
9673db4e8c
sgemm_wg: Fix possible divide-by-0
2024-03-28 17:35:47 -07:00
Hansung Kim
9555b790e7
sgemm_wg: ifdef-guard cluster specific code
2024-03-27 22:45:51 -07:00
Hansung Kim
09822764e7
sgemm_wg: Remove software-based barrier implementation
...
Intra-cluster barrier is now implemented in hardware, transparent to the ISA.
2024-03-27 22:43:45 -07:00
