Layouts

You could have invented CuTe hierarchical layout (but maybe not the rest of it?) : ezyang’s blog

Ed posted the best intro to CuTe layouts I have seen, by showing how to extrapolate them from PyTorch striding¹.

Well, it turns out, this is exactly how CuTe layouts work! In CuTe, sizes/strides are hierarchical: a size is actually a tree of ints, where the hierarchy denotes internal structure of a dimension that you can address linearly (in fact, everything by default can be addressed in a 1-D linear way, even if its an N-D object.)

Relatedly, Simon Veitner put together a quite visual understanding of layouts. https://veitner.bearblog.dev/intuition-behind-hierarchical-layouts/ – the graphics are helpful once you have the baseline intuition from Ed’s post!

1. If you’re not familiar with striding, Ed’s PyTorch Internals talk/post remains the best intro! ↩︎