New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support graph 64-bit edge number #8

Open

jdongca2003 wants to merge 3 commits into AutomataLab:master from jdongca2003:master

jdongca2003 commented Oct 26, 2022

In the master branch the graph edge number is limited to uint32. In this pull request,

change all nodePointer to ull (unsigned long long (64-bit))
change all node data type to uint (typedef unsigned int uint --- 32-bit uint32). Both are defined in globals.hpp. Currently uint32 is still used to save GPU RAM. But it is easy to support 64-bit node data type. Just refine uint
for converter, use stdin as input. This helps to support very large graph where edge list is compressed (clueweb12).

I successfully tested the version on a subset of clueweb12 graph (http://www.lemurproject.org/clueweb12/webgraph.php/) : 27 billion edges and 600 million nodes on nvidia A100 GPU.

jdongca2003 added 3 commits

October 25, 2022 19:04


          support 64-bit edges

a4d0e40


          remove converter.cpp

7613e6b


          update the instruction for graph format conversion

dd2d54e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet