mirror of
https://github.com/DS4SD/docling.git
synced 2025-12-08 20:58:11 +00:00
docs: More GPU results and improvements in the example docs (#2674)
* add more results and improve the example docs Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * 5070 windows timing Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> * add reference for cpu-only Signed-off-by: Michele Dolfi <dol@zurich.ibm.com> --------- Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
This commit is contained in:
17
docs/examples/gpu_standard_pipeline.py
vendored
17
docs/examples/gpu_standard_pipeline.py
vendored
@@ -1,3 +1,20 @@
|
||||
# %% [markdown]
|
||||
#
|
||||
# What this example does
|
||||
# - Run a conversion using the best setup for GPU for the standard pipeline
|
||||
#
|
||||
# Requirements
|
||||
# - Python 3.9+
|
||||
# - Install Docling: `pip install docling`
|
||||
#
|
||||
# How to run
|
||||
# - `python docs/examples/gpu_standard_pipeline.py`
|
||||
#
|
||||
# This example is part of a set of GPU optimization strategies. Read more about it in [GPU support](../../usage/gpu/)
|
||||
#
|
||||
# ## Example code
|
||||
# %%
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
import time
|
||||
|
||||
29
docs/examples/gpu_vlm_pipeline.py
vendored
29
docs/examples/gpu_vlm_pipeline.py
vendored
@@ -1,3 +1,32 @@
|
||||
# %% [markdown]
|
||||
#
|
||||
# What this example does
|
||||
# - Run a conversion using the best setup for GPU using VLM models
|
||||
#
|
||||
# Requirements
|
||||
# - Python 3.10+
|
||||
# - Install Docling: `pip install docling`
|
||||
# - Install vLLM: `pip install vllm`
|
||||
#
|
||||
# How to run
|
||||
# - `python docs/examples/gpu_vlm_pipeline.py`
|
||||
#
|
||||
# This example is part of a set of GPU optimization strategies. Read more about it in [GPU support](../../usage/gpu/)
|
||||
#
|
||||
# ### Start models with vllm
|
||||
#
|
||||
# ```console
|
||||
# vllm serve ibm-granite/granite-docling-258M \
|
||||
# --host 127.0.0.1 --port 8000 \
|
||||
# --max-num-seqs 512 \
|
||||
# --max-num-batched-tokens 8192 \
|
||||
# --enable-chunked-prefill \
|
||||
# --gpu-memory-utilization 0.9
|
||||
# ```
|
||||
#
|
||||
# ## Example code
|
||||
# %%
|
||||
|
||||
import datetime
|
||||
import logging
|
||||
import time
|
||||
|
||||
Reference in New Issue
Block a user