Get your documents ready for gen AI
Go to file
Christoph Auer aed772ab33
feat: Threaded PDF pipeline (#1951)
* Initial async pdf pipeline

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* UpstreamAwareQueue

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Refactoring into async pipeline primitives and graph

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Cleanups and safety improvements

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Better threaded PDF pipeline

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Pin docling-ibm-models

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Remove unused args

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Add test

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Revise pipeline

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Unload doc backend

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Revert "Unload doc backend"

This reverts commit 01066f0b6e.

* Remove redundant method

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Update threaded test

Signed-off-by: Ubuntu <ubuntu@ip-172-31-30-253.eu-central-1.compute.internal>

* Stop accumulating docs in test run

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fix: don't starve on docs with > max_queue_size pages

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fix: don't starve on docs with > max_queue_size pages

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* DCO Remediation Commit for Christoph Auer <cau@zurich.ibm.com>

I, Christoph Auer <cau@zurich.ibm.com>, hereby add my Signed-off-by to this commit: fa71cde950
I, Ubuntu <ubuntu@ip-172-31-30-253.eu-central-1.compute.internal>, hereby add my Signed-off-by to this commit: d66da87d96

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fix: python3.9 compat

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Option to enable threadpool with doc_batch_concurrency setting

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Clean up unused code

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fix settings defaults expectations

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Use released docling-ibm-models

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Remove ignores for typing/linting

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Signed-off-by: Ubuntu <ubuntu@ip-172-31-30-253.eu-central-1.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-30-253.eu-central-1.compute.internal>
2025-07-26 11:49:37 +02:00
.actor fix(integration): update the Apify Actor integration (#1619) 2025-05-21 02:47:55 +02:00
.github docs: update readme and add ASR example (#1836) 2025-06-23 18:55:16 +02:00
docling feat: Threaded PDF pipeline (#1951) 2025-07-26 11:49:37 +02:00
docs docs: add chat with dosu (#1984) 2025-07-24 11:07:36 +02:00
tests feat: Threaded PDF pipeline (#1951) 2025-07-26 11:49:37 +02:00
.gitattributes chore: exclude data from GH Linguist (#1671) 2025-05-28 15:42:34 +02:00
.gitignore ci: Add Github Actions (#4) 2024-07-16 13:05:04 +02:00
.pre-commit-config.yaml feat: simplify dependencies, switch to uv (#1700) 2025-06-03 15:18:54 +02:00
CHANGELOG.md chore: bump version to 2.42.2 [skip ci] 2025-07-24 10:21:10 +00:00
CITATION.cff chore: Update repository URL in CITATION.cff (#1363) 2025-04-14 06:57:04 +02:00
CODE_OF_CONDUCT.md docs: Linux Foundation AI & Data (#1183) 2025-03-19 09:05:57 +01:00
CONTRIBUTING.md feat: simplify dependencies, switch to uv (#1700) 2025-06-03 15:18:54 +02:00
Dockerfile chore: properly clean up apt temporary files in Dockerfile (#1223) 2025-03-25 11:10:09 +01:00
LICENSE chore: fix placeholders in license (#63) 2024-09-06 17:10:07 +02:00
MAINTAINERS.md docs: Linux Foundation AI & Data (#1183) 2025-03-19 09:05:57 +01:00
mkdocs.yml docs: enrich existing DoclingDocument (#1969) 2025-07-22 16:20:15 +02:00
pyproject.toml feat: Threaded PDF pipeline (#1951) 2025-07-26 11:49:37 +02:00
README.md docs: add chat with dosu (#1984) 2025-07-24 11:07:36 +02:00
uv.lock feat: Threaded PDF pipeline (#1951) 2025-07-26 11:49:37 +02:00

Docling

Docling

DS4SD%2Fdocling | Trendshift

arXiv Docs PyPI version PyPI - Python Version uv Ruff Pydantic v2 pre-commit License MIT PyPI Downloads Docling Actor Chat with Dosu OpenSSF Best Practices LF AI & Data

Docling simplifies document processing, parsing diverse formats — including advanced PDF understanding — and providing seamless integrations with the gen AI ecosystem.

Features

  • 🗂️ Parsing of multiple document formats incl. PDF, DOCX, PPTX, XLSX, HTML, WAV, MP3, images (PNG, TIFF, JPEG, ...), and more
  • 📑 Advanced PDF understanding incl. page layout, reading order, table structure, code, formulas, image classification, and more
  • 🧬 Unified, expressive DoclingDocument representation format
  • ↪️ Various export formats and options, including Markdown, HTML, DocTags and lossless JSON
  • 🔒 Local execution capabilities for sensitive data and air-gapped environments
  • 🤖 Plug-and-play integrations incl. LangChain, LlamaIndex, Crew AI & Haystack for agentic AI
  • 🔍 Extensive OCR support for scanned PDFs and images
  • 👓 Support of several Visual Language Models (SmolDocling)
  • 🎙️ Support for Audio with Automatic Speech Recognition (ASR) models
  • 💻 Simple and convenient CLI

Coming soon

  • 📝 Metadata extraction, including title, authors, references & language
  • 📝 Chart understanding (Barchart, Piechart, LinePlot, etc)
  • 📝 Complex chemistry understanding (Molecular structures)

Installation

To use Docling, simply install docling from your package manager, e.g. pip:

pip install docling

Works on macOS, Linux and Windows environments. Both x86_64 and arm64 architectures.

More detailed installation instructions are available in the docs.

Getting started

To convert individual documents with python, use convert(), for example:

from docling.document_converter import DocumentConverter

source = "https://arxiv.org/pdf/2408.09869"  # document per local path or URL
converter = DocumentConverter()
result = converter.convert(source)
print(result.document.export_to_markdown())  # output: "## Docling Technical Report[...]"

More advanced usage options are available in the docs.

CLI

Docling has a built-in CLI to run conversions.

docling https://arxiv.org/pdf/2206.01062

You can also use 🥚SmolDocling and other VLMs via Docling CLI:

docling --pipeline vlm --vlm-model smoldocling https://arxiv.org/pdf/2206.01062

This will use MLX acceleration on supported Apple Silicon hardware.

Read more here

Documentation

Check out Docling's documentation, for details on installation, usage, concepts, recipes, extensions, and more.

Examples

Go hands-on with our examples, demonstrating how to address different application use cases with Docling.

Integrations

To further accelerate your AI application development, check out Docling's native integrations with popular frameworks and tools.

Get help and support

Please feel free to connect with us using the discussion section.

Technical report

For more details on Docling's inner workings, check out the Docling Technical Report.

Contributing

Please read Contributing to Docling for details.

References

If you use Docling in your projects, please consider citing the following:

@techreport{Docling,
  author = {Deep Search Team},
  month = {8},
  title = {Docling Technical Report},
  url = {https://arxiv.org/abs/2408.09869},
  eprint = {2408.09869},
  doi = {10.48550/arXiv.2408.09869},
  version = {1.0.0},
  year = {2024}
}

License

The Docling codebase is under MIT license. For individual model usage, please refer to the model licenses found in the original packages.

LF AI & Data

Docling is hosted as a project in the LF AI & Data Foundation.

IBM ❤️ Open Source AI

The project was started by the AI for knowledge team at IBM Research Zurich.