Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from tensorflow:master #169

Open
wants to merge 1,187 commits into
base: master
Choose a base branch
from

Conversation

pull[bot]
Copy link

@pull pull bot commented Dec 8, 2024

See Commits and Changes for more details.


Created by pull[bot] (v2.0.0-alpha.1)

Can you help keep this open source service alive? 💖 Please sponsor : )

@pull pull bot added the ⤵️ pull label Dec 8, 2024
tensorflower-gardener and others added 29 commits January 4, 2025 00:44
PiperOrigin-RevId: 711972819
PiperOrigin-RevId: 711972858
PiperOrigin-RevId: 711979713
PiperOrigin-RevId: 711982874
PiperOrigin-RevId: 711990180
PiperOrigin-RevId: 711990181
PiperOrigin-RevId: 711990863
PiperOrigin-RevId: 711990919
PiperOrigin-RevId: 711991091
PiperOrigin-RevId: 711992189
PiperOrigin-RevId: 711997285
PiperOrigin-RevId: 711997299
PiperOrigin-RevId: 712021736
PiperOrigin-RevId: 712079394
PiperOrigin-RevId: 712079403
PiperOrigin-RevId: 712088917
PiperOrigin-RevId: 712179206
PiperOrigin-RevId: 712193767
PiperOrigin-RevId: 712323268
PiperOrigin-RevId: 712341978
junwhanahn and others added 30 commits January 8, 2025 20:49
`xla::PjRtLayout` was designed as an abstract class so that it leaves options to represent layouts without depending on `xla::Layout`. In reality, `xla::PjRtXlaLayout` is the only concrete layout representation that will exist in the foreseeable future, and the lack of a proper type-erased layout creation interface forces everyone to use unsafe downcast to access the underlying layout. This causes an unnecessary code bloat without much extensibility because too many downcasts practically prevent new layout representations from being easily introduced.

This CL folds `xla::PjRtXlaLayout` into `xla::PjRtLayout` and make `xla::PjRtLayout` a non-abstract class. Like `xla::Shape` that is used pervasively in PjRt, this CL makes layouts a concrete type based on `xla::Layout`. The benefit is that it simplifies many callers that use PjRt layouts: `xla::GetXlaLayoutUnsafe()` is now replaced with the `pjrt_layout->xla_layout()` accessor, no more `down_cast`/`dynamic_cast` to access `xla::PjRtXlaLayout`, etc.

`xla::ifrt::BasicStringArrayLayout` was the only other implementation of `xla::PjRtLayout` and this is now removed. Since string arrays are supported only in IFRT and not in PjRt, its layout representation should also live only in IFRT. Since no one depends on string array layouts, this CL simply removes its implementation so that we can add a proper one once a proper IFRT layout type is added.

PiperOrigin-RevId: 713516368
PiperOrigin-RevId: 713524077
PiperOrigin-RevId: 713524368
There is only one call to `TfLiteDelegateCopyFromBufferHandleInternal`,
which passes in `t` for the `tensor` parameter and `t->delegate` for the `delegate` parameter, so inside this function, `tensor->delegate` and `delegate` are equivalent expressions that evaluate to the same value.  But referencing `delegate` rather
than `tensor->delegate` is simpler and more readable here, and makes the nullness check match the dereference on the following line, and is more consistent with the other functions in this file.  So this change modifies the code to use `delegate`
rather than `tensor->delegate`.

PiperOrigin-RevId: 713528157
PiperOrigin-RevId: 713534685
PiperOrigin-RevId: 713549118
PiperOrigin-RevId: 713561687
PiperOrigin-RevId: 713563161
PiperOrigin-RevId: 713569059
PiperOrigin-RevId: 713571453
PiperOrigin-RevId: 713572893
PiperOrigin-RevId: 713575031
…FirstTargetDimToMoveShardingTiles`.

`GetFirstTargetDimToMoveShardingTiles` can be used for moving the sharding tiles from a source dimension to a target dimension when the source dimension and target dimension are different and the size of target dimension is divisible by the merged tile size. This util function will be used in the dimensions that need replication in the partitioner.

This cl has no behavior change. We will use this util function to support
1. Concat dimension in concat operations
2. Slice dimensions in dynamic-slice operations

PiperOrigin-RevId: 713588209
Imported from GitHub PR openxla/xla#21166

Copybara import of the project:

--
b939d5aea471e4b267a806b19102b6d56a7abe0a by Ilia Sergachev <[email protected]>:

[DOC] Fix a link in the documentation.

Merging this change closes #21166

PiperOrigin-RevId: 713589150
Imported from GitHub PR openxla/xla#21175

Copybara import of the project:

--
caaf17448ae8dade929d728852093ec82384337b by Ilia Sergachev <[email protected]>:

[DOC] Fix a mistype.

Merging this change closes #21175

PiperOrigin-RevId: 713594132
PiperOrigin-RevId: 713597347
PiperOrigin-RevId: 713597458
PiperOrigin-RevId: 713601879
PiperOrigin-RevId: 713622968
PiperOrigin-RevId: 713628270
PiperOrigin-RevId: 713628733
PiperOrigin-RevId: 713630341
PiperOrigin-RevId: 713633408
… obsolete when reorderValues function was removed. We still want to keep the test and remove the patch.

PiperOrigin-RevId: 713638728
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.