-
Notifications
You must be signed in to change notification settings - Fork 209
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cudf #445
Closed
Closed
Cudf #445
Changes from 12 commits
Commits
Show all changes
463 commits
Select commit
Hold shift + click to select a range
23df5bc
cudf all the way thru, cuda cannot handle nulls so few more ifs
a2640cd
cudf+umap working on numerics
ec151b8
full numeric cudf-- needs hack to plot
0ec412a
full numeric cudf-- needs hack to plot
55c2b07
use rename if 2 columns, otherwise = to columns list
0a42090
umap cudf
1a013a8
untested, adds decorator for cuml and pandas dataframes, standardizin…
c23529b
rst change PlotterBase to plotter
tanmoyio c79fac6
add graphistry.compute to graphistry.rst
tanmoyio fbc33e3
Delete graphistry.compute.rst
tanmoyio 35345ff
Update modules.rst
tanmoyio 811d3e5
add graphistry.compute to toctree
tanmoyio d6806cf
resolve short underline error
tanmoyio 0b301ca
test1: resolve blank line error
tanmoyio 24638cf
test2: resolve blank line error
tanmoyio 9e0c3d2
test3: resolve blank line error
tanmoyio ac15070
doc fix
tanmoyio f53710c
doc fix
tanmoyio a8a9dc1
doc fix
tanmoyio 8618b1a
doc fix
tanmoyio 3a6611d
doc fix
tanmoyio 82d1d3b
doc fix
tanmoyio 3fe4cee
add versioneer
tanmoyio 8199471
add versioneer
tanmoyio d01e2bf
fix(docstr): removed unneccesary indents/sections
dess890 812f5c7
fix(docstr): removed unneccessary spacing
dess890 b88ce77
fix(docstr): fixed unindent
dess890 940be4e
fix(docstr): removed line/spacing
dess890 fdb7d2b
Revert "umap cudf"
17fd316
Revert "umap cudf"
af399f6
revert b4 alex borq
3f3a1f4
adds cudf support and wraps dataframe if engine=cuml
446bdc8
adds safe gpu wrapper to edges as well
271e3e1
merges cleanup branch and resolves discrepencies
96363f5
adds safer handling of cupy/cudf arrays
6dc826a
removes unused code
27c879b
handles pandas via if statement
4f01bde
adds missing engine flag
93cadb1
adds missing _ to output
f479857
begin testing cudf-cu_cat
923ec9c
begin testing cudf-cu_cat
b4e110d
begin cudf_cat for 3x to 10x
b6a60ea
adds handling of g.transform_umap when engine=cuml
84741c8
safe converts X, y before infer_graph method
6aefc49
debugging why node not in df
caf367c
fix typo
40914dc
adds test if node not in df, adds numeric index
b8a7eb5
adds solution to both infer_graph and infer_self_graph functions
b71d8ad
fixes concat between cudf and pd
8be6aa0
fixes bug
c1bd253
merges recent cleanup changes into cudf-alex2 in new branch
7482eb6
bring enc_X to cudf
e1ceb59
coerces previously fit X to pandas if cudf
0dd3ced
seemingly unavoidable cudf edge transforms to pandas
593b31f
merge cudf, cudf-cat, cudf-alex3
73f8c95
incorp cudf-alex3 for cudf-cu_cat
c2ce1df
incorp cudf-alex3 for cudf-cu_cat
ba56292
naive cudf tests
f7b7156
merge clean, add naive tests
0b8028a
fix(docstr): added layout & compute to nav
dess890 299adf1
fix(docstr): removed unneccessary lines
dess890 eaf69a5
forces dbscan to run on cpu only -- once https://github.com/rapidsai/…
8782a6d
adds engine_dbscan flag and if [sklearn ,umap_learn] will allow g.tra…
44351ba
bug fix
c2d70e2
explicit engine_dbscan flags
48fe85f
bug fix
d2611a3
adds pandas coercion before infer_graph
d4030bc
bug fix
42e8eb3
bug fix
2027c35
bug fix
13d9173
bug fix
d288597
bug fix
dfba41c
bug fix
fa2a6ba
lint
ea6ce34
lint
118e36d
adds modified test for dbscan params
911529c
feat(iframe): added a graph homepg, can be removed
dess890 2c9dc78
adds NVIDIA GTC demo that installs from branch
e1f64c5
removes print statements and changes default umap settings
c9249c0
typoo
6e85a37
merges cudf-alex3
c583814
moves feature_engine resolve to logical order
a5a626a
changes umap spread parameter default to 1
c5dc84d
refactors core featurize and umap engines so that cudf and pd are con…
69659f8
fixes typo and disambiguates between umap_engine and umap_engine_
aedc81e
lazy cudf import (thx alex)
465d486
lazy cudf import (thx alex)
87d706c
changed umap spread to 1
silkspace 08ad02c
lazy cudf import, pin |torch for now
521dc5b
lazy cudf import, pin |torch for now
ba99c40
resolve f_engine
5b55f24
adds cudf conversion inside _featurize_* calls
5e0a1ce
adds print
23da7fc
pulls cudf X into pandas for FAISS indexing
720c7f7
merge cudf + cudf_cat_alex2
8b13311
placeholder cu_cat setup
434256c
tweaks needed for gpu cu_cat
0807e76
fix(docs): removed iframe (for now)
dess890 d25faf8
fix: cuml umap and tests fix
tanmoyio d9987de
lint: flake8 typo
tanmoyio 168af4b
fix: _dgl_graph fix
tanmoyio fee5452
typo
tanmoyio 60d3b97
test: remove xfail test_dgl_utils
tanmoyio 4d74ade
test: remove StartTime test_dgl_utils temp
tanmoyio aaf275f
pinned pandas
tanmoyio fd423a2
alex2/3 cucat req
5f2cb69
more tests
tanmoyio 408ee52
stable
tanmoyio c9d1c95
Merge branch 'master' into cudf-final
tanmoyio 762b2d2
fix(modules.rst) testing to ci warnings
dess890 e0abb95
fix(conf.py); added plugins to nitpick
dess890 6ebb74a
fix(modules): added title
dess890 63d665b
fix(modules.rst): added title for ci testing
dess890 18e0bef
doc fix
tanmoyio ee5be00
Merge branch 'cudf-final' of https://github.com/graphistry/pygraphist…
tanmoyio 6a5bec9
fix(rst) docs fixes for CI passing
dess890 a58f279
umap trick for cudf dfs
tanmoyio 827ae22
ignore args type
tanmoyio b95400e
feat(rst) added badges
dess890 cb10f3c
fix(plotter): adding plotter to menu (will update)
dess890 10907c3
plotterbase to plotter
tanmoyio 9b5c3db
merge
tanmoyio dcf60ac
addStyle to add_style
tanmoyio 75f11b0
fix(rst): revert plotter changes for ci test
dess890 953cccc
all addStyle to add_style
tanmoyio edc1f6d
resolve conflicts
tanmoyio 8d64481
fix(plotter): expanding menu
dess890 5900e2f
fix(docst) added umap to articles
dess890 0850778
feat(docst) added photo for home pg
dess890 c501bca
Merge branch 'navbar-fixes' of github.com:graphistry/pygraphistry int…
dess890 1234dd7
test add chain in __init__.py
tanmoyio 3b3654f
Merge branch 'navbar-fixes' of https://github.com/graphistry/pygraphi…
tanmoyio c7bc46d
test nitpick
tanmoyio a88961b
test nitpick 2
tanmoyio f8d6ee1
test
tanmoyio 8115020
test 3
tanmoyio e581f97
test 4
tanmoyio bde96b1
test 5
tanmoyio f66856c
test 6
tanmoyio 06f2652
test 7
tanmoyio 254b31b
test 8
tanmoyio 1d9722c
test 9
tanmoyio 073eaec
test 10
tanmoyio 1e82076
test 11
tanmoyio 1a68d74
test 12
tanmoyio 8320dd5
test 13
tanmoyio a1941ba
test 14
tanmoyio 4361ae6
final fix
tanmoyio db8c228
final fix 1
tanmoyio 5e9a577
final fix 2
tanmoyio 52506b1
fix(conf.py): added converter for badges
dess890 7114bc8
fix(conf.py): removed img converter
dess890 224351f
test(conf.py): using only directive for ci testing
dess890 cf25d29
test(conf.py): testing only directive
dess890 b17af60
fix(docstr): removed slack badge
dess890 385a0ad
test(docstr): testing to see if uptime is failing
dess890 7a74dc8
cudf edge reqs
402c544
merge cudf-final
6b0056a
cu_cat refactor
800e2ba
merge conflicts with navbar-fixes
tanmoyio e79a3e6
typo: graphistry.rst compute.cluster
tanmoyio 9c445f1
fix: duplicate entries docs
tanmoyio da34a54
TestFeatureCUMLProcessors
20dc72d
merge cudf_cat to main cudf
ce17b8e
tests: add cudf umap pass through
tanmoyio 5ff14f8
lint: flake8 fixes
tanmoyio 77eb8c2
fix: cudf umap skip
tanmoyio c2d2fcb
need to make cudf import for edges lazy
c703a42
test: cudf tests with docker flag
tanmoyio 006aa7d
delete: .swp files
tanmoyio d634d91
add: test_umap_utils on test-gpu-local.sh
tanmoyio 400b632
fix: revert back to addStyle from add_style
tanmoyio e6fc323
added warnings for predict_links
tanmoyio c300050
fix: test cudf flag
tanmoyio 1305db1
doc: changelog update for _dgl_graph
tanmoyio 7ad6449
passing gpu test_feature_utils
tanmoyio 2395147
additional checks for embed utils
tanmoyio 7334b4b
merge
dcolinmorgan d81932e
merge cudf-cat-final
dcolinmorgan 8e99fe3
cu_cat flag in umap
dcolinmorgan 2e9820c
added tanmoy umap changes
dcolinmorgan e06ce0c
flake8 fix
tanmoyio b7ce57e
Merge branch 'cudf-final' of https://github.com/graphistry/pygraphist…
tanmoyio 8f5a40f
mypy ignore hyperdask
tanmoyio 1660774
mypy ignore _version.py
tanmoyio 7574e97
temp fix for cudf objects in embed
tanmoyio a9017fa
lint
dcolinmorgan 65ce26b
lint
dcolinmorgan ab7fd8e
lint
dcolinmorgan d3d3071
type: ignore cu_cat import
dcolinmorgan 88adafc
type: ignore cu_cat import
dcolinmorgan 63f6044
base_extras_heavy[cu_cat]
dcolinmorgan 1887a82
base_extras_heavy[cu_cat]
dcolinmorgan 8ec6c6e
base_extras_heavy[cu-cat]
dcolinmorgan 6f85ee1
base_extras_heavy[cu-cat]
dcolinmorgan 4acc4d8
base_extras_heavy[cu-cat]
dcolinmorgan 0e0ae32
long_version_py ignore type
dcolinmorgan 7ec91f7
egg-0.02.0
dcolinmorgan d78628b
egg-0.02.0
dcolinmorgan 429f6a3
rm egg
dcolinmorgan 6d6ae25
fix: cu_cat missing stubs ignore
tanmoyio 4204713
skip feature_utils cudf tests
tanmoyio f1a0b2c
Merge branch 'cudf-final' of https://github.com/graphistry/pygraphist…
tanmoyio 037975e
sklearn FunctionTransformer, no lazy cuml import
dcolinmorgan 2631564
sklearn FunctionTransformer, no lazy cuml import
dcolinmorgan 71cfcd5
some fixes for cpu checks(gpu issues are still there)
tanmoyio 9755e44
flake: fix
tanmoyio a03a294
merge with cudf-final embed
dcolinmorgan 19e9f6c
merge with cudf-final embed
dcolinmorgan 9d673b4
assert cudf not import
dcolinmorgan 03a4042
assert cudf not import
dcolinmorgan 071f1c1
lazy not assert
dcolinmorgan dd38945
Update embed_utils.py
tanmoyio e96bf01
migrate check_cudf to embed_utils.py
tanmoyio 827984d
Update embed_utils.py
tanmoyio a95371a
Merge branch 'cudf-cat-final' into cudf
dcolinmorgan a7e28cd
Merge branch 'cudf-final' into cudf
dcolinmorgan 13f6b7e
lint
dcolinmorgan 339779b
lint
dcolinmorgan ff213fb
lint
dcolinmorgan 084395a
lint
dcolinmorgan 3068e6a
merge cudf-final
dcolinmorgan 70b50d9
lint
dcolinmorgan 79cafba
Merge branch 'cudf-cat-final' into cudf
dcolinmorgan 06a691b
lazy cudf import
dcolinmorgan 4b779ac
lazy cudf import
dcolinmorgan 9b25eec
Merge branch 'cudf-cat-final' into cudf
dcolinmorgan c1a0cca
lint
dcolinmorgan f853472
better lazy cudf import
dcolinmorgan 8e05dd4
Merge branch 'cudf-cat-final' into cudf
dcolinmorgan b6b148b
lazy merge
dcolinmorgan 17f0af6
lint
dcolinmorgan 2a5c879
lint
dcolinmorgan 80ba095
functiontransform cuml import
dcolinmorgan 6709bea
functiontransform cuml import
dcolinmorgan 901846c
functiontransform cuml import
dcolinmorgan 118ea80
functiontransform cuml import
dcolinmorgan f1ee230
functiontransform cuml import
dcolinmorgan 7fc02e2
use dirty_cat superVec for torch/etc, except if cu_cat
dcolinmorgan d38f469
use dirty_cat superVec for torch/etc, except if cu_cat
dcolinmorgan 6436067
use dirty_cat superVec for torch/etc, except if cu_cat
dcolinmorgan 25573ea
sklearn functiontransformer & MLB
dcolinmorgan dee5ad4
sklearn functiontransformer & MLB
dcolinmorgan cac6cc4
all preprocess back to sklearn
dcolinmorgan 3757b10
import FT again
dcolinmorgan 523d180
rewrite g_n_t
dcolinmorgan 97b725d
revert g_n_t
dcolinmorgan 7ab97a4
import FT in get_numeric_transform
dcolinmorgan aba0c55
import FT in get_numeric_transform
dcolinmorgan fb96400
latest release opt-in install
dcolinmorgan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is
emb
-- isn't it acudf.DataFrame
already? this might just be aemb.rename(columns={..})
... or nothing? I'm not sure of what/why hereThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
very strange... getmodule does say it is a cudf df, but rename does not work on emb since its a cupy
following works but not pretty enough im guessing:
emb.columns=[config.X, config.Y]
emb.index=index
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what do
isinstance(emb, cudf.DataFrame)
andtype(emb)
say?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True, <class 'cudf.core.dataframe.DataFrame'>
can use rename when just 2 columns