mirrors/docling
1
0
Fork 0
mirror of https://github.com/DS4SD/docling.git synced 2025-07-27 12:34:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
511 Commits 36 Branches 112 Tags 287 MiB
661f7c9780
Commit Graph

511 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Peter Staar
661f7c9780 fixed the pipeline for Phi4 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-16 15:55:49 +02:00
Peter Staar
d41b856961 finalising last points for vlms support 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-16 12:39:26 +02:00
Peter Staar
fc61258273 merged with main 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-15 07:46:06 +02:00
Peter Staar
e2c95d09bc need to get Phi4 working again ... 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-15 07:32:55 +02:00
Peter Staar
15a8f328c2 added pipeline_model_specializations file 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-15 05:27:16 +02:00
Peter Staar
7c67d2b2fe fixed the MyPy 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-14 17:51:43 +02:00
github-actions[bot]
aeb0716bbb chore: bump version to 2.32.0 [skip ci] 2025-05-14 14:28:21 +00:00
Vinay R Damodaran
3a04f2a367
feat: Improve parallelization for remote services API calls (#1548) 
* Provide the option to make remote services call concurrent

Signed-off-by: Vinay Damodaran <vrdn@hey.com>

* Use yield from correctly?

Signed-off-by: Vinay Damodaran <vrdn@hey.com>

* not do amateur hour stuff

Signed-off-by: Vinay Damodaran <vrdn@hey.com>

---------

Signed-off-by: Vinay Damodaran <vrdn@hey.com>
 2025-05-14 15:47:55 +02:00
jimkarag02
9f8b479f17
fix(ocr): orig field in TesseractOcrCliModel as str (#1553) 
fix: ensure orig and text are both strings in TesseractOcrCliModel

Signed-off-by: Dimitris Karagatslis <dimo9.dk@gmail.com>
 2025-05-14 15:05:52 +02:00
Panos Vagenas
9f28abf061
docs: add advanced chunking & serialization example (#1589) 
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
 2025-05-14 14:35:07 +02:00
Peter Staar
a3716b1961 refactoring minimal_vlm_pipeline 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-14 13:57:32 +02:00
Alex Sokolov
2efb7a7c06
fix(settings): fix nested settings load via environment variables (#1551) 
Signed-off-by: Alexander Sokolov <alsokoloff@gmail.com>
 2025-05-14 13:42:10 +02:00
Peter Staar
7c97b494ec added the VlmPredictionToken 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-14 12:23:46 +02:00
Elwin
12dab0a1e8
feat: support image/webp file type (#1415) 
* support image/webp file type

Signed-off-by: Elwin <61868295+hzhaoy@users.noreply.github.com>
Signed-off-by: Elwin <hzywong@gmail.com>

* docs: add webp image format in supported_formats.md

Signed-off-by: Elwin <61868295+hzhaoy@users.noreply.github.com>
Signed-off-by: Elwin <hzywong@gmail.com>

* test: add a test case for `image/webp` file

Signed-off-by: Elwin <hzywong@gmail.com>

* style: apply styling

Signed-off-by: Elwin <hzywong@gmail.com>

* test: update test case of converting `image/webp` file with more ocr engines

Signed-off-by: Elwin <hzywong@gmail.com>

* style: apply styling

Signed-off-by: Elwin <hzywong@gmail.com>

* rename test file

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: Elwin <61868295+hzhaoy@users.noreply.github.com>
Signed-off-by: Elwin <hzywong@gmail.com>
Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Co-authored-by: Michele Dolfi <dol@zurich.ibm.com>
 2025-05-14 09:47:28 +02:00
Peter Staar
f159075b67 pixtral 12b runs via MLX and native transformers 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-14 07:39:20 +02:00
Peter Staar
054e01d8b3 added the formulate_prompt 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-14 06:26:16 +02:00
Peter Staar
4c0bc61e54 refactoring the download_model 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-14 05:31:54 +02:00
Peter Staar
3407955a47 all working, now serious refacgtoring necessary 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-13 18:23:55 +02:00
github-actions[bot]
23238c241f chore: bump version to 2.31.2 [skip ci] 2025-05-13 10:09:19 +00:00
Marco Fargetta
4046d0b2f3
fix: AsciiDoc header identification (#1562) (#1563) 
Fix regular expression to identify header lines in AsciiDoc avoiding to
match defined blocks.

Signed-off-by: Marco Fargetta <mfargett@redhat.com>
 2025-05-13 11:17:26 +02:00
Michele Dolfi
8baa85a49d
fix: restrict click version and update lock file (#1582) 
* fix click dependency and update lock file

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* Update test GT

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Co-authored-by: Christoph Auer <cau@zurich.ibm.com>
 2025-05-13 10:40:08 +02:00
Peter Staar
96862bd326 refactoring the VLM part 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-13 10:01:37 +02:00
Peter Staar
ee01e3cff0 Merge branch 'main' into dev/add-other-vlm-models 2025-05-13 06:08:26 +02:00
Peter Staar
7fbe021359 working on vlm's 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-13 06:07:11 +02:00
Peter Staar
77eb21b235 got microsoft/Phi-4-multimodal-instruct to work 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-12 13:37:03 +02:00
Peter Staar
68747e3cad fixed the transformers 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-12 13:08:33 +02:00
github-actions[bot]
0d0fa6cbe3 chore: bump version to 2.31.1 [skip ci] 2025-05-12 09:44:26 +00:00
Michele Dolfi
127e38646f
fix: add smoldocling in download utils (#1577) 
add smoldocling in download utils

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
 2025-05-12 10:48:07 +02:00
Peter Staar
bd2d01f0ac Merge branch 'main' into dev/add-other-vlm-models 2025-05-12 08:52:52 +02:00
Oleg Lavrovsky
844babb390
docs: update links in data_prep_kit (#1559) 
Update data_prep_kit.md

The links were broken, since the repository was renamed. I also noticed that PDF2Parquet is now referred to as Docling2Parquet.

Signed-off-by: Oleg Lavrovsky <31819+loleg@users.noreply.github.com>
 2025-05-11 20:38:25 +02:00
Peter Staar
18e1ec4df2 feat: adding new vlm-models support 
Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-05-11 09:30:10 +02:00
Cesar Berrospi Ramis
776e7ecf9a
fix(HTML): handle row spans in header rows (#1536) 
* chore(HTML): log the stacktrace of errors

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

* fix(HTML): handle row headers like in pivot tables

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

---------

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
 2025-05-09 15:14:32 +02:00
Panos Vagenas
3220a592e7
docs: add serialization docs, update chunking docs (#1556) 
* docs: add serializers docs, update chunking docs

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

* update notebook to improve MD table rendering

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>

---------

Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
 2025-05-08 21:43:01 +02:00
DavidLee
f1658edbad
fix: mime error in document streams (#1523) 
Update document.py

edit got file mime error

Signed-off-by: DavidLee <yongsheng_li@foxmail.com>
 2025-05-06 09:30:46 +02:00
Michele Dolfi
7c705739f9
fix: usage of hashlib for FIPS (#1512) 
fix usage of hashlib for FIPS

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
 2025-05-02 15:03:29 +02:00
Panos Vagenas
de56523974
chore: format JSON test files to enable comparison (#1511) 
Signed-off-by: Panos Vagenas <pva@zurich.ibm.com>
 2025-05-02 10:52:18 +02:00
Ihar Hrachyshka
b147331f2a
chore: restore typing hint for self.script_readers (#1500) 
With future annotations, typing hints resolution is always deferred.

https://peps.python.org/pep-0563/

Signed-off-by: Ihar Hrachyshka <ihar.hrachyshka@gmail.com>
 2025-04-30 20:33:27 +02:00
Ben Browning
4ab7e9ddfb
fix: Guard against attribute errors in TesseractOcrModel __del__ (#1494) 
This moves the initialization of the `reader` and `script_readers`
attributes to before we attempt to import tesserocr, so that when later
accessing these attributes in the garbage collection method `__del__`
the attributes exist.

This requires changing the typing of the `script_readers` dict value to
`Any` because we cannot yet reference its actual strong type, since it's
a tesserocr value.

This prevents throwing an exception during garbage collection for
cases where the TesseractOcrModel instance didn't properly initialize,
like when it throws an `ImportError` during its initializer.

Signed-off-by: Ben Browning <bbrownin@redhat.com>
 2025-04-30 17:51:33 +02:00
Zach Cox
cc453961a9
fix: enable cuda_use_flash_attention2 for PictureDescriptionVlmModel (#1496) 
fix: enable use_cuda_flash_attention2 for PictureDescriptionVlmModel

Signed-off-by: Zach Cox <zach.s.cox@gmail.com>
 2025-04-30 08:02:52 +02:00
Peter W. J. Staar
976e92e289
fix: updated the time-recorder label for reading order (#1490) 
* fix: updated the time-recorder label for reading order

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

* reformatted code

Signed-off-by: Peter Staar <taa@zurich.ibm.com>

---------

Signed-off-by: Peter Staar <taa@zurich.ibm.com>
 2025-04-29 13:02:53 +02:00
Michele Dolfi
d8959c6b19
chore: update dependencies in lock file (#1458) 
update lock: h11 vuln and torch update

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
 2025-04-28 08:52:46 +02:00
nkh0472
a097ccd8d5
chore: typo fix (#1465) 
* typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

* chore: typo fix

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>

---------

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>
 2025-04-28 08:52:09 +02:00
Emmanuel Ferdman
3afbe6c969
docs: update supported formats guide (#1463) 
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
 2025-04-28 08:51:54 +02:00
Maxim Lysak
94d66a0765
fix: Incorrect scaling of TableModel bboxes when do_cell_matching is False (#1459) 
fixing double scaling in case of do_cell_matching is False

Signed-off-by: Maksym Lysak <mly@zurich.ibm.com>
Co-authored-by: Maksym Lysak <mly@zurich.ibm.com>
 2025-04-25 12:34:12 +02:00
github-actions[bot]
c67133dde4 chore: bump version to 2.31.0 [skip ci] 2025-04-25 08:28:25 +00:00
Ryan Lin
a2fbbba9f7
feat: add tutorial using Milvus and Docling for RAG pipeline (#1449) 
* feat: add milvus rag with docling tutorial

Signed-off-by: Ryan Lin <linjinhong@yandex.com>

* chore: run pre-commit

Signed-off-by: Ryan Lin <linjinhong@yandex.com>

* feat: add RAG with Milvus example to mkdocs

Signed-off-by: Ryan Lin <linjinhong@yandex.com>

---------

Signed-off-by: Ryan Lin <linjinhong@yandex.com>
 2025-04-25 09:12:35 +02:00
Michele Dolfi
976431ed7f
chore: update locked deps (#1442) 
update deps

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
 2025-04-23 14:59:31 +02:00
Cesar Berrospi Ramis
ed20124544
fix(html): handle address, details, and summary tags (#1436) 
* fix(html): handle 'address' tag

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

* fix(html): handle 'details' tag

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

---------

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
 2025-04-23 09:30:59 +02:00
nkh0472
c2470ed216
docs: Fix wrong output format in example code (#1427) 
fix: wrong output format

Signed-off-by: nkh0472 <67589323+nkh0472@users.noreply.github.com>
 2025-04-22 12:32:55 +02:00
Michele Dolfi
64918a81ac
docs: Add OpenSSF Best Practices badge (#1430) 
* docs: add openssf badge

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

* add badge to docs

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>

---------

Signed-off-by: Michele Dolfi <dol@zurich.ibm.com>
 2025-04-22 11:23:28 +02:00