Please Publicize

[ehartford]

Can you please publicize this and make it consumable?

If you do not, then nobody really can use it. (except researchers who stumble on it like me)

It needs to either:

1. be merged with the main flash attention repo or
2. be given a unique name (perhaps FlexHeadFA or something) and added to pypi so that we can 'pip install flexheadfa'

This can be a big contribution to open source AI development, if you would make it accessible.

Thank you,
Eric Hartford

[dingo-actual]

☝️

[xiayuqing0622]

@ehartford Thank you for your suggestion, Eric!

We actually created a PR to the main FlashAttention repository several months ago, with the intention of merging it there. However, since the PR has been stalled for quite some time, giving it a unique name and publishing it to PyPI might indeed be a better approach.

We’ll work on this and make it accessible soon. Thank you for highlighting its potential contribution to the open-source AI community!

[ehartford]

Can you please reference the PR to Flash attention?I will try to bring attention to it

Is it this one? Dao-AILab#1166

[xiayuqing0622]

Can you please reference the PR to Flash attention?I will try to bring attention to it

Is it this one? Dao-AILab#1166

Yes, and we've created a related issue to query about this yesterday. Dao-AILab#1358

[ehartford]

Thankfully he made an explicit decision

[xiayuqing0622]

We’ll work on publishing it to PyPI then. :)

[xiayuqing0622]

Hi @ehartford , We’ve just published it to PyPI! You can install it using: pip install flex-head-fa --no-build-isolation （We went with the name you suggested :)). Please note that this version is not yet synced with the latest main FlashAttention repository. We’ll work on updating it soon.

Thank you for your suggestion.

[ehartford]

Thank you! Looking forward to using it!