Kernels:

kernels-community
/

paged-attention

Kernel card Files Files and versions

attention

Paged attention kernels from vLLM and mistral.rs.

Performance

Downloads last month: -

kernels

apache-2.0

Supported hardwares new

CUDA 7.58.08.69.010.0

NVIDIA SXM

B200

192GB

NVIDIA SXM

H200

141GB

NVIDIA SXM

H100

80GB

GPU

L40s

48GB

GPU

L40

48GB

GPU

L20

48GB

GPU

L4

24GB

GPU

RTX 6000 Ada

48GB

GPU

RTX 5880 Ada

48GB

RTX

RTX 5000 Ada

32GB

GPU

RTX 4500 Ada

24GB

RTX

RTX 4000 Ada

20GB

RTX

RTX 4000 SFF Ada

20GB

GPU

RTX 2000 Ada

16GB

GPU

RTX A6000

48GB

GPU

RTX A5000

8GB

GPU

RTX A5000 Max-Q

16GB

GPU

RTX A5000 Mobile

16GB

GPU

RTX A4000

16GB

GPU

RTX A4000 Max-Q

8GB

GPU

RTX A4000 Mobile

8GB

GPU

RTX A3000 Mobile

6GB

GPU

RTX A2000

6GB

GPU

RTX A2000 Embedded

4GB

GPU

RTX A2000 Max-Q

4GB

GPU

RTX A2000 Mobile

4GB

GPU

A100

80GB

GPU

A40

48GB

GPU

A30

24GB

GPU

A10

24GB

GPU

A2

16GB

RTX

RTX 4090

24GB

RTX

RTX 4090D

24GB

RTX

RTX 4090 Mobile

16GB

RTX

RTX 4080 SUPER

16GB

RTX

RTX 4080

16GB

RTX

RTX 4080 Mobile

12GB

RTX

RTX 4070

12GB

RTX

RTX 4070 Mobile

8GB

RTX

RTX 4070 Ti

12GB

RTX

RTX 4070 Super

12GB

RTX

RTX 4070 Ti Super

16GB

RTX

RTX 4060

8GB

RTX

RTX 4060 Ti

8GB

RTX

RTX 4090 Laptop

16GB

RTX

RTX 4080 Laptop

12GB

RTX

RTX 4070 Laptop

8GB

RTX

RTX 4060 Laptop

8GB

RTX

RTX 4050 Laptop

6GB

RTX

RTX 3090

24GB

RTX

RTX 3090 Ti

24GB

RTX

RTX 3080

12GB

RTX

RTX 3080 Ti

12GB

RTX

RTX 3080 Mobile

8GB

RTX

RTX 3070

8GB

RTX

RTX 3070 Ti

8GB

RTX

RTX 3070 Ti Mobile

8GB

RTX

RTX 3060 Ti

8GB

RTX

RTX 3060

12GB

GPU

RTX 2080 Ti

11GB

GPU

RTX 2080

8GB

GPU

RTX 2070

8GB

GPU

RTX 2070 SUPER Mobile

8GB

GPU

RTX 2070 SUPER

8GB

RTX

RTX 3060 Mobile

6GB

RTX

RTX 3050 Mobile

4GB

GPU

RTX 2060

6GB

GPU

RTX 2060 12GB

12GB

GPU

RTX 2060 Mobile

6GB

GPU

RTX Titan

24GB

GPU

GTX 1660

6GB

GPU

GTX 1650 Mobile

4GB

NVIDIA T4

T4

16GB

GPU

T10

16GB

Jetson

Jetson AGX Orin 64GB

64GB

Jetson

Jetson AGX Orin 32GB

32GB

Jetson

Jetson Orin NX 16GB

16GB

Jetson

Jetson Orin NX 8GB

8GB

Jetson

Jetson Orin Nano 8GB

8GB

Jetson

Jetson Orin Nano 4GB

4GB

ROCm

AMD Instinct

MI300

192GB

AMD Instinct

MI250

128GB

AMD Instinct

MI210

64GB

AMD Instinct

MI100

32GB

AMD Instinct

MI60

32GB

AMD Instinct

MI50

16GB

AMD Radeon

RX 7900 XTX

24GB

AMD Radeon

RX 7900 XT

20GB

AMD Radeon

RX 7900 GRE

16GB

AMD Radeon

RX 7800 XT

16GB

AMD Radeon

RX 7700 XT

12GB

AMD Radeon

RX 6950 XT

16GB

AMD Radeon

RX 6800

16GB

GPU

Radeon Pro VII

16GB

Torch: 2.10

OS: macoslinux

Arch: x86_64aarch64