Skip to content

Commit 70f629a

Browse files
authored
Update README.md
1 parent eaff7a8 commit 70f629a

File tree

1 file changed

+13
-6
lines changed

1 file changed

+13
-6
lines changed

docker/README.md

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
# Get model from Hugging Face
22
`python3 ./hug_model.py`
33

4-
You should now have a model in the current directory and model.bin symlinked to it for the subsequent Docker build and copy step. e.g.
4+
You should now have a model in the current directory and `model.bin` symlinked to it for the subsequent Docker build and copy step. e.g.
55
```
66
docker $ ls -lh *.bin
7-
-rw-rw-r-- 1 user user 4.8G May 23 18:30 llama-7b.ggmlv3.q5_1.bin
7+
-rw-rw-r-- 1 user user 4.8G May 23 18:30 <downloaded-model-file>.q5_1.bin
88
lrwxrwxrwx 1 user user 24 May 23 18:30 model.bin -> <downloaded-model-file>.q5_1.bin
99
```
10-
- Note #1: Make sure you have enough disk space to d/l the model. As the model is then copied into the image you will need at least
10+
**Note #1:** Make sure you have enough disk space to d/l the model. As the model is then copied into the image you will need at least
1111
**TWICE** as much disk space as the size of the model:
1212

1313
| Model | Quantized size |
@@ -17,16 +17,23 @@ lrwxrwxrwx 1 user user 24 May 23 18:30 model.bin -> <downloaded-model-file>.q5
1717
| 30B | 25 GB |
1818
| 65B | 50 GB |
1919

20-
- Note #2: If you want to pass or tune additional parameters, customise `./start_server.sh` before running `docker build ...`
20+
**Note #2:** If you want to pass or tune additional parameters, customise `./start_server.sh` before running `docker build ...`
2121

22-
# Use OpenBLAS (No NVidia GPU, defaults to `python:3-slim-bullseye` Docker base image)
22+
# Install Docker Server
23+
24+
**Note #3:** This was tested with Docker running on Linux. If you can get it working on Windows or MacOS, please update this README with a PR!
25+
26+
[Install Docker Engine](https://docs.docker.com/engine/install)
27+
28+
# Use OpenBLAS
29+
No NVidia GPU, defaults to `python:3-slim-bullseye` Docker base image and OpenBlAS:
2330
## Build:
2431
`docker build --build-arg -t openblas .`
2532
## Run:
2633
`docker run --cap-add SYS_RESOURCE -t openblas`
2734

2835
# Use CuBLAS
29-
Requires NVidia GPU and Docker NVidia support (see https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)
36+
Requires NVidia GPU and Docker NVidia support (see [container-toolkit/install-guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html))
3037
## Build:
3138
`docker build --build-arg IMAGE=nvidia/cuda:12.1.1-devel-ubuntu22.04 -t opencuda .`
3239
## Run:

0 commit comments

Comments
 (0)