feat: Add DoclingParseV4 backend, using high-level docling-parse API (#905)

* Add DoclingParseV3 backend implementation

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Use docling-core with docling-parse types

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fixes and test updates

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fix streams

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fix streams

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Reset tests

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* update test cases

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* update test units

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Add back DoclingParse v1 backend, pipeline options

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Update locks

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* fix: update docling-core to 2.22.0

Update dependency library docling-core to latest release 2.22.0
Fix regression tests and ground truth files

Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

* Ground-truth files updated

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Update tests, use TextCell.from_ocr property

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Text fixes, new test data

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Rename docling backend to v4

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Test all backends, fixes

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Reset all tests to use docling-parse v1 for now

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* Fixes for DPv4 backend init, better test coverage

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

* test_input_doc use default backend

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>

---------

Signed-off-by: Christoph Auer <cau@zurich.ibm.com>
Signed-off-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>
Co-authored-by: Cesar Berrospi Ramis <75900930+ceberam@users.noreply.github.com>

tests/data/groundtruth/docling_v2/pa20010031492.md

@@ -1,6 +1,6 @@
 # Assay reagent
 
-## ABSTRACT
+### ABSTRACT
 
 A cell-derived assay reagent prepared from cells which have been killed by treatment with an antibiotic selected from the bleomycin-phleomycin family of antibiotics but which retain a signal-generating metabolic activity such as bioluminescence. 
 
@@ -74,9 +74,9 @@ FIG. 7: Control cells, stationary phase.
 
 FIG. 8: Zeocin™ treated cells, stationary phase.
 
-## EXAMPLE 1
+### EXAMPLE 1
 
-## (A) Inactivation of Bioluminescent E. coil Method
+### (A) Inactivation of Bioluminescent E. coil Method
 
 1. Bioluminescent genetically modified E. coil strain HB101 (E. coli HB101 made bioluminescent by transformation with a plasmid carrying the lux operon of Vibrio fischeri constructed by the method of Shaw and Kado, as described in Biotechnology 4: 560-564) were grown from a frozen stock in 5 ml of low salt medium (LB (5 g/ml NaCl)+glycerol+MgSO₄) for 24 hours.
 
@@ -90,7 +90,7 @@ FIG. 8: Zeocin™ treated cells, stationary phase.
 
 FIG. 1 shows the effect of Zeocin™ treatment on the light output and viable count (per ml) of recombinant bioluminescent E. coil. Zeocin™ was added to a final concentration of 1.5 mg/ml at time zero. The number of viable cells in the culture was observed to decrease with increasing contact cells with Zeocin™, the culture being completely inactivated after 3 hours. The light output from the culture was observed to decrease gradually with increasing Zeocin™ contact time.
 
-## (B) Production of Assay Reagent
+### (B) Production of Assay Reagent
 
 Five hours after the addition of Zeocin™ or water the remaining bacterial cells in the Zeocin™ treated and control cultures were harvested by the centrifugation, washed (to remove traces of Zeocin™ from the Zeocin™ treated culture), re-centrifuged and resuspended in cryoprotectant to an OD₆₃₀ of 0.25. 200 μl aliquots of the cells in cryoprotectant were dispensed into single shot vials, and freeze dried. Freeze dried samples of the Zeocin™ treated cells and control cells were reconstituted in 0.2M sucrose to form assay reagents and the light output of the assay reagents measured at various times after reconstitution.
 
@@ -98,9 +98,9 @@ The light output from assay reagent prepared from cells exposed to 1.5 mg/ml Zeo
 
 FIG. 2 shows the light output from five separate vials of reconstituted Zeocin™ treated assay reagent inactivated according to the method of Example 1(A) and processed into assay reagent as described in Example 1(B). Reconstitution solution was added at time zero and thereafter light output was observed to increase steadily before stabilising out at around 15 minutes after reconstitution. All five vials were observed to give similar light profiles after reconstitution.
 
-## EXAMPLE 2
+### EXAMPLE 2
 
-## Sensitivity of Zeocin™ Treated Assay Reagent to Toxicant Method
+### Sensitivity of Zeocin™ Treated Assay Reagent to Toxicant Method
 
 1. Bioluminescent genetically modified E. coil strain HB101 (E. coli HB101 made bioluminescent by transformation with a plasmid carrying the lux operon of vibrio fischeri constructed by the method of Shaw and Kado, as described in Biotechnology 4: 560-564) was grown in fermenter as a batch culture in low salt medium (LB(5 g/ml NaCl)+glycerol+MgSO₄).
 
@@ -160,9 +160,9 @@ Table 1: Sensitivity of the different assay reagents to ZnSo₄ expressed as EC
 
 The results of the toxicity assays indicate that Zeocin™ treatment does not significantly affect the sensitivity of a recombinant bioluminescent E. coli derived assay reagent to ZnSO₄. Similar results could be expected with other toxic substances which have an effect on signal-generating metabolic activities.
 
-## EXAMPLE 3
+### EXAMPLE 3
 
-## Method to Determine Viable Count
+### Method to Determine Viable Count
 
 1. Samples of bacterial culture to be assayed for viable count were centrifuged at 10,000 rpm for 5 minutes to pellet the bacterial cells.
 
@@ -174,7 +174,7 @@ The results of the toxicity assays indicate that Zeocin™ treatment does not si
 
 5. The number of bacterial colonies present for each of the three aliquots at each of the serial dilutions were counted and the values averaged. Viable count was calculated per ml of bacterial culture.
 
-## CLAIMS
+### CLAIMS
 
 1. A method of making a non-viable preparation of prokaryotic or eukaryotic cells, which preparation has a signal-generating metabolic activity, which method comprises contacting a viable culture of said cells having signal-generating metabolic activity with an antibiotic selected from the bleomycin/phleomycin family of antibiotics.