Skip to content

Commit

Permalink
Update docs for 1.18.0 (#373)
Browse files Browse the repository at this point in the history
Updating docker links & version references
  • Loading branch information
kzawora-intel authored Oct 8, 2024
1 parent b4db57d commit 4183a07
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 10 deletions.
2 changes: 1 addition & 1 deletion Dockerfile.hpu
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM vault.habana.ai/gaudi-docker/1.17.0/ubuntu22.04/habanalabs/pytorch-installer-2.3.1:latest
FROM vault.habana.ai/gaudi-docker/1.18.0/ubuntu22.04/habanalabs/pytorch-installer-2.4.1:latest

COPY ./ /workspace/vllm

Expand Down
10 changes: 5 additions & 5 deletions README_GAUDI.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Requirements
- OS: Ubuntu 22.04 LTS
- Python: 3.10
- Intel Gaudi accelerator
- Intel Gaudi software version 1.17.0
- Intel Gaudi software version 1.18.0

To verify that the Intel Gaudi software was correctly installed, run:

Expand All @@ -45,8 +45,8 @@ for more details.
Use the following commands to run a Docker image:

``` {.console}
$ docker pull vault.habana.ai/gaudi-docker/1.17.0/ubuntu22.04/habanalabs/pytorch-installer-2.3.1:latest
$ docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --net=host --ipc=host vault.habana.ai/gaudi-docker/1.17.0/ubuntu22.04/habanalabs/pytorch-installer-2.3.1:latest
$ docker pull vault.habana.ai/gaudi-docker/1.18.0/ubuntu22.04/habanalabs/pytorch-installer-2.4.1:latest
$ docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --net=host --ipc=host vault.habana.ai/gaudi-docker/1.18.0/ubuntu22.04/habanalabs/pytorch-installer-2.4.1:latest
```

Build and Install vLLM
Expand Down Expand Up @@ -159,10 +159,10 @@ variable), and `--enforce-eager` flag.


> [!WARNING]
> In 1.17.0, all modes utilizing `PT_HPU_LAZY_MODE=0` are highly
> In 1.18.0, all modes utilizing `PT_HPU_LAZY_MODE=0` are highly
> experimental and should be only used for validating functional
> correctness. Their performance will be improved in the next releases.
> For obtaining the best performance in 1.17.0, please use HPU Graphs, or
> For obtaining the best performance in 1.18.0, please use HPU Graphs, or
> PyTorch lazy mode.
Bucketing mechanism
Expand Down
8 changes: 4 additions & 4 deletions docs/source/getting_started/gaudi-installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Requirements
- OS: Ubuntu 22.04 LTS
- Python: 3.10
- Intel Gaudi accelerator
- Intel Gaudi software version 1.17.0
- Intel Gaudi software version 1.18.0

To verify that the Intel Gaudi software was correctly installed, run:

Expand All @@ -45,8 +45,8 @@ Use the following commands to run a Docker image:

.. code:: console
$ docker pull vault.habana.ai/gaudi-docker/1.17.0/ubuntu22.04/habanalabs/pytorch-installer-2.3.1:latest
$ docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --net=host --ipc=host vault.habana.ai/gaudi-docker/1.17.0/ubuntu22.04/habanalabs/pytorch-installer-2.3.1:latest
$ docker pull vault.habana.ai/gaudi-docker/1.18.0/ubuntu22.04/habanalabs/pytorch-installer-2.4.1:latest
$ docker run -it --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --net=host --ipc=host vault.habana.ai/gaudi-docker/1.18.0/ubuntu22.04/habanalabs/pytorch-installer-2.4.1:latest
Build and Install vLLM
---------------------------
Expand Down Expand Up @@ -156,7 +156,7 @@ Currently in vLLM for HPU we support four execution modes, depending on selected
- PyTorch lazy mode

.. warning::
In 1.17.0, all modes utilizing ``PT_HPU_LAZY_MODE=0`` are highly experimental and should be only used for validating functional correctness. Their performance will be improved in the next releases. For obtaining the best performance in 1.17.0, please use HPU Graphs, or PyTorch lazy mode.
In 1.18.0, all modes utilizing ``PT_HPU_LAZY_MODE=0`` are highly experimental and should be only used for validating functional correctness. Their performance will be improved in the next releases. For obtaining the best performance in 1.18.0, please use HPU Graphs, or PyTorch lazy mode.


Bucketing mechanism
Expand Down

0 comments on commit 4183a07

Please sign in to comment.