Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] main from kubeflow:main #69

Merged
merged 10 commits into from
Jun 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/ui-bff-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: UI - BFF - Build
on:
push:
branches:
- 'main'
pull_request:
paths-ignore:
- 'LICENSE*'
- 'DOCKERFILE*'
- '**.gitignore'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.22.2'

- name: Clean
working-directory: clients/ui/bff
run: make clean

- name: Build
working-directory: clients/ui/bff
run: make build

- name: Check if there are uncommitted file changes
working-directory: clients/ui/bff
run: |
clean=$(git status --porcelain)
if [[ -z "$clean" ]]; then
echo "Empty git status --porcelain: $clean"
else
echo "Uncommitted file changes detected: $clean"
git diff
exit 1
fi
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
# Test binary, built with `go test -c`
*.test

__debug*

# Output of the go coverage tool, specifically when used with LiteIDE
*.out

Expand Down
41 changes: 41 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,19 @@ and now you can substitute `gmake` every time the make command is mentioned in g
Several options of docker engines are available for Mac.
Having Docker installed is also helpful for Testcontainers.

### Podman

[Podman](https://podman.io/) also supports Rosetta (Apple specific) emulation which is handy since the Google MLMD project dependency is x86 specific.

We recommend setting up the Podman machine with root privileges,
and setting the environment variable

```sh
export TESTCONTAINERS_RYUK_PRIVILEGED=true
```

when running TestContainer-based Model Registry Python tests (for more information, see [here](https://pypi.org/project/testcontainers/#:~:text=TESTCONTAINERS_RYUK_PRIVILEGED)).

### Colima

Colima offers Rosetta (Apple specific) emulation which is handy since the Google MLMD project dependency is x86 specific.
Expand Down Expand Up @@ -202,3 +215,31 @@ Then with the given setup MLMD is already installed inside the DevContainer:
At this point Poetry is already installed as well and can be used to build and run test of the Model Registry Python client.

<!-- to be continued with explanation of this "hack": https://github.com/tarilabs/ml-metadata-remote#readme -->

# FAQ

## Error `docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/containers ...`

This happens on Mac OSX when running Testcontainers-based Python tests, and [Ryuk](https://github.com/testcontainers/moby-ryuk) does not have the correct access priviledges.

```
-------------------------------------------- Captured stderr setup ---------------------------------------------
Pulling image testcontainers/ryuk:0.7.0
Container started: 4af385c2d670
---------------------------------------------- Captured log setup ----------------------------------------------
WARNING root:config.py:59 DOCKER_AUTH_CONFIG is experimental, see testcontainers/testcontainers-python#566
INFO testcontainers.core.container:container.py:88 Pulling image testcontainers/ryuk:0.7.0
INFO testcontainers.core.container:container.py:101 Container started: 4af385c2d670
=========================================== short test summary info ============================================
ERROR tests/store/test_wrapper.py::test_get_undefined_artifact_type_id - docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/containers/4af385c2d67067875f6f0...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================================== 1 error in 0.50s ===============================================
```

Solution:
- Use `export TESTCONTAINERS_RYUK_PRIVILEGED=true` (we recommend this option)
- Use `export TESTCONTAINERS_RYUK_DISABLED=true` to disable Ryuk entirely; the tests are configured to close all container resources, but we do not recommend this option

For more information:
- section [Docker engine](https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md#docker-engine) in this document
- https://pypi.org/project/testcontainers/#:~:text=the%20database%20version.-,Configuration,-Env%20Variable
27 changes: 26 additions & 1 deletion clients/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,30 @@ There are caveats to be noted when using this method:
)
```

## Advanced use-cases

### Using Model Registry Python Client with newer Python versions (>=3.11)

> [!CAUTION]
> The mechanism described in this section is a temporary workaround and likely will never be supported.
> This workaround is ONLY applicable if your Python/Notebook project does NOT make use of MLMD directly or indirectly.

<!-- a longer-term plan to address this ties to the investigations to rebase this client on top of MR REST api directly,
so to avoid having to wrap the MLMD Wheel. See more: https://github.com/kubeflow/model-registry/pull/59 -->

This project _currently_ depends for internal implementations on the Google's [MLMD Python library](https://pypi.org/project/ml-metadata/).
Due to this dependency, this project supports [only the Python versions](https://github.com/kubeflow/model-registry/blob/8d77c13100c6cc5a9465d4293403114a3576fdd7/clients/python/pyproject.toml#L14) which are also available for the MLMD library (see more [here](https://pypi.org/project/ml-metadata/#files)).

As a workaround, **only IF your Python/Notebook project does NOT make use of MLMD directly or indirectly**,
you could opt-in to make use of a non-constrained variant of the MLMD dependency supporting _only_ remote gRPC calls (and not constrained by specific Python versions or architectures):

```
!pip install "https://github.com/opendatahub-io/ml-metadata/releases/download/v1.14.0%2Bremote.1/ml_metadata-1.14.0+remote.1-py3-none-any.whl" # need a Python 3.11 compatible version
!pip install --no-deps --ignore-requires-python --pre "model-registry" # ignore dependencies because of the above override
```

You can read more about this use-case, in the [Remote-only packaging of MLMD Python lib](https://github.com/kubeflow/model-registry/blob/main/docs/remote_only_packaging_of_MLMD_Python_lib.md) document.

## Development

Common tasks, such as building documentation and running tests, can be executed using [`nox`](https://github.com/wntrblm/nox) sessions.
Expand All @@ -110,13 +134,14 @@ Use `nox -l` to list sessions and execute them using `nox -s [session]`.

### Running Locally on Mac M1 or M2 (arm64 architecture)

If you want run tests locally you will need to set up a colima develeopment environment using the instructions [here](https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md#colima)
If you want run tests locally you will need to set up a development environment, including docker engine; we recommend following the instructions [here](https://github.com/kubeflow/model-registry/blob/main/CONTRIBUTING.md#docker-engine).

You will also have to change the package source to one compatible with ARM64 architecture. This can be actioned by uncommenting lines 14 or 15 in the pyproject.toml file. Run the following command after you have uncommented the line.

```sh
poetry lock
```

Use the following commands to directly run the tests with individual test output. Alternatively you can use the nox session commands above.

```sh
Expand Down
Loading
Loading