mirror of
https://github.com/DS4SD/docling.git
synced 2025-07-24 19:14:23 +00:00
2f72167ff6
* Draft implementation of Doctag backend Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> * Updated VLM pipeline doctags to docling conversion, now properly supports lists Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> * preparing to migrate to new doctags deserializer Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> * re-using DocTagsDocument.from_doctags_and_image_pairs Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> * satisfying mypy and other checks Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> * Added support for force_backend_text parameter Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> * removed unnecessary transformation Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> * Cleaned up Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> * Update tests Signed-off-by: Christoph Auer <cau@zurich.ibm.com> * Updated readme Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> --------- Signed-off-by: Maksym Lysak <mly@zurich.ibm.com> Signed-off-by: Christoph Auer <cau@zurich.ibm.com> Co-authored-by: Maksym Lysak <mly@zurich.ibm.com> Co-authored-by: Christoph Auer <cau@zurich.ibm.com>
156 lines
6.6 KiB
Markdown
156 lines
6.6 KiB
Markdown
<p align="center">
|
|
<a href="https://github.com/docling-project/docling">
|
|
<img loading="lazy" alt="Docling" src="https://github.com/docling-project/docling/raw/main/docs/assets/docling_processing.png" width="100%"/>
|
|
</a>
|
|
</p>
|
|
|
|
# Docling
|
|
|
|
<p align="center">
|
|
<a href="https://trendshift.io/repositories/12132" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12132" alt="DS4SD%2Fdocling | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
|
</p>
|
|
|
|
[](https://arxiv.org/abs/2408.09869)
|
|
[](https://docling-project.github.io/docling/)
|
|
[](https://pypi.org/project/docling/)
|
|
[](https://pypi.org/project/docling/)
|
|
[](https://python-poetry.org/)
|
|
[](https://github.com/psf/black)
|
|
[](https://pycqa.github.io/isort/)
|
|
[](https://pydantic.dev)
|
|
[](https://github.com/pre-commit/pre-commit)
|
|
[](https://opensource.org/licenses/MIT)
|
|
[](https://pepy.tech/projects/docling)
|
|
[](https://apify.com/vancura/docling)
|
|
|
|
Docling simplifies document processing, parsing diverse formats — including advanced PDF understanding — and providing seamless integrations with the gen AI ecosystem.
|
|
|
|
## Features
|
|
|
|
* 🗂️ Parsing of [multiple document formats][supported_formats] incl. PDF, DOCX, XLSX, HTML, images, and more
|
|
* 📑 Advanced PDF understanding incl. page layout, reading order, table structure, code, formulas, image classification, and more
|
|
* 🧬 Unified, expressive [DoclingDocument][docling_document] representation format
|
|
* ↪️ Various [export formats][supported_formats] and options, including Markdown, HTML, and lossless JSON
|
|
* 🔒 Local execution capabilities for sensitive data and air-gapped environments
|
|
* 🤖 Plug-and-play [integrations][integrations] incl. LangChain, LlamaIndex, Crew AI & Haystack for agentic AI
|
|
* 🔍 Extensive OCR support for scanned PDFs and images
|
|
* 🥚 Support of Visual Language Models ([SmolDocling](https://huggingface.co/ds4sd/SmolDocling-256M-preview))
|
|
* 💻 Simple and convenient CLI
|
|
|
|
### Coming soon
|
|
|
|
* 📝 Metadata extraction, including title, authors, references & language
|
|
* 📝 Chart understanding (Barchart, Piechart, LinePlot, etc)
|
|
* 📝 Complex chemistry understanding (Molecular structures)
|
|
|
|
## Installation
|
|
|
|
To use Docling, simply install `docling` from your package manager, e.g. pip:
|
|
```bash
|
|
pip install docling
|
|
```
|
|
|
|
Works on macOS, Linux and Windows environments. Both x86_64 and arm64 architectures.
|
|
|
|
More [detailed installation instructions](https://docling-project.github.io/docling/installation/) are available in the docs.
|
|
|
|
## Getting started
|
|
|
|
To convert individual documents, use `convert()`, for example:
|
|
|
|
```python
|
|
from docling.document_converter import DocumentConverter
|
|
|
|
source = "https://arxiv.org/pdf/2408.09869" # document per local path or URL
|
|
converter = DocumentConverter()
|
|
result = converter.convert(source)
|
|
print(result.document.export_to_markdown()) # output: "## Docling Technical Report[...]"
|
|
```
|
|
|
|
More [advanced usage options](https://docling-project.github.io/docling/usage/) are available in
|
|
the docs.
|
|
|
|
## Documentation
|
|
|
|
Check out Docling's [documentation](https://docling-project.github.io/docling/), for details on
|
|
installation, usage, concepts, recipes, extensions, and more.
|
|
|
|
## Examples
|
|
|
|
Go hands-on with our [examples](https://docling-project.github.io/docling/examples/),
|
|
demonstrating how to address different application use cases with Docling.
|
|
|
|
## Integrations
|
|
|
|
To further accelerate your AI application development, check out Docling's native
|
|
[integrations](https://docling-project.github.io/docling/integrations/) with popular frameworks
|
|
and tools.
|
|
|
|
## Apify Actor
|
|
|
|
<a href="https://apify.com/vancura/docling?fpr=docling"><img src="https://apify.com/ext/run-on-apify.png" alt="Run Docling Actor on Apify" width="176" height="39" /></a>
|
|
|
|
You can run Docling in the cloud without installation using the [Docling Actor](https://apify.com/vancura/docling?fpr=docling) on Apify platform. Simply provide a document URL and get the processed result:
|
|
|
|
```bash
|
|
apify call vancura/docling -i '{
|
|
"options": {
|
|
"to_formats": ["md", "json", "html", "text", "doctags"]
|
|
},
|
|
"http_sources": [
|
|
{"url": "https://vancura.dev/assets/actor-test/facial-hairstyles-and-filtering-facepiece-respirators.pdf"},
|
|
{"url": "https://arxiv.org/pdf/2408.09869"}
|
|
]
|
|
}'
|
|
```
|
|
|
|
The Actor stores results in:
|
|
|
|
* Processed document in key-value store (`OUTPUT_RESULT`)
|
|
* Processing logs (`DOCLING_LOG`)
|
|
* Dataset record with result URL and status
|
|
|
|
Read more about the [Docling Actor](.actor/README.md), including how to use it via the Apify API and CLI.
|
|
|
|
## Get help and support
|
|
|
|
Please feel free to connect with us using the [discussion section](https://github.com/docling-project/docling/discussions).
|
|
|
|
## Technical report
|
|
|
|
For more details on Docling's inner workings, check out the [Docling Technical Report](https://arxiv.org/abs/2408.09869).
|
|
|
|
## Contributing
|
|
|
|
Please read [Contributing to Docling](https://github.com/docling-project/docling/blob/main/CONTRIBUTING.md) for details.
|
|
|
|
## References
|
|
|
|
If you use Docling in your projects, please consider citing the following:
|
|
|
|
```bib
|
|
@techreport{Docling,
|
|
author = {Deep Search Team},
|
|
month = {8},
|
|
title = {Docling Technical Report},
|
|
url = {https://arxiv.org/abs/2408.09869},
|
|
eprint = {2408.09869},
|
|
doi = {10.48550/arXiv.2408.09869},
|
|
version = {1.0.0},
|
|
year = {2024}
|
|
}
|
|
```
|
|
|
|
## License
|
|
|
|
The Docling codebase is under MIT license.
|
|
For individual model usage, please refer to the model licenses found in the original packages.
|
|
|
|
## IBM ❤️ Open Source AI
|
|
|
|
Docling has been brought to you by IBM.
|
|
|
|
[supported_formats]: https://docling-project.github.io/docling/usage/supported_formats/
|
|
[docling_document]: https://docling-project.github.io/docling/concepts/docling_document/
|
|
[integrations]: https://docling-project.github.io/docling/integrations/
|