mirrors/docling
1
0
Fork 0
mirror of https://github.com/DS4SD/docling.git synced 2025-12-08 20:58:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: Describe examples (#2262)

Browse Source 
* Update .py examples with clearer guidance,
update out of date imports and calls

Signed-off-by: Mingxuan Zhao <43148277+mingxzhao@users.noreply.github.com>

* Fix minimal.py string error, fix ruff format error

Signed-off-by: Mingxuan Zhao <43148277+mingxzhao@users.noreply.github.com>

* fix more CI issues

Signed-off-by: Mingxuan Zhao <43148277+mingxzhao@users.noreply.github.com>

---------

Signed-off-by: Mingxuan Zhao <43148277+mingxzhao@users.noreply.github.com>
This commit is contained in:
Mingxuan Zhao
2025-09-16 10:00:38 -04:00
committed by GitHub
parent 0e95171dd6
commit ff351fd40c
21 changed files with 608 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
docs/examples/export_multimodal.py vendored
View File

@@ -1,3 +1,34 @@
# %% [markdown]
# Export multimodal page data (image bytes, text, segments) to a Parquet file.
#
# What this example does
# - Converts a PDF and assembles per-page multimodal records: image, cells, text, segments.
# - Normalizes records to a pandas DataFrame and writes a timestamped `.parquet` in `scratch/`.
#
# Prerequisites
# - Install Docling and `pandas`. Optional: `datasets` and `Pillow` for the commented demo.
#
# How to run
# - From the repo root: `python docs/examples/export_multimodal.py`.
# - Output parquet is written to `scratch/`.
#
# Key options
# - `IMAGE_RESOLUTION_SCALE`: page rendering scale (1 ~ 72 DPI).
# - `PdfPipelineOptions.generate_page_images`: keep page images for export.
#
# Requirements
# - Writing Parquet requires an engine such as `pyarrow` or `fastparquet`
# (`pip install pyarrow` is the most common choice).
#
# Input document
# - Defaults to `tests/data/pdf/2206.01062.pdf`. Change `input_doc_path` as needed.
#
# Notes
# - The commented block at the bottom shows how to load the Parquet with HF Datasets
# and reconstruct images from raw bytes.
# %%
import datetime
import logging
import time
@@ -23,10 +54,8 @@ def main():
input_doc_path = data_folder / "pdf/2206.01062.pdf"
output_dir = Path("scratch")
# Important: For operating with page images, we must keep them, otherwise the DocumentConverter
# will destroy them for cleaning up memory.
# This is done by setting AssembleOptions.images_scale, which also defines the scale of images.
# scale=1 correspond of a standard 72 DPI image
# Keep page images so they can be exported to the multimodal rows.
# Use PdfPipelineOptions.images_scale to control the render scale (1 ~ 72 DPI).
pipeline_options = PdfPipelineOptions()
pipeline_options.images_scale = IMAGE_RESOLUTION_SCALE
pipeline_options.generate_page_images = True