Nvidia Triton for OpenShift Source to Image (s2i)

This repo shows how to package a ML model with a Nvidia Triton server container.

Quickstart

Note

In this example Triton server is configured to serve models from:/models

Run Triton - Polling mode (PVC storage)

Note

You can copy models from local storage via oc cp to a PVC or ephemeral storage

oc apply -k gitops/overlays/triton

# change project / ns context
oc project demo-triton

# get pod name
POD=$(oc get pod -l app=triton-server -o custom-columns=POD:.metadata.name --no-headers)

# copy model into /models
oc cp examples/source-builder/models/simple ${POD}:/models/

Run Triton - Use an Init Container to setup model

oc apply -k gitops/overlays/triton-init

Run Triton - Load models from S3 bucket at runtime

Note

Modify triton-s3-models secret with s3 credentials

oc apply -k gitops/overlays/triton-s3

Run Builder Demos

oc apply -k gitops/

More Info

See examples for more details

Links

• Triton Walkthrough