mirrors/docling
1
0
Fork 0
mirror of https://github.com/DS4SD/docling.git synced 2025-07-30 14:04:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Specify encoding when writing output file to avoid errors when default target encoding doesn't have all characters

Browse Source 
Specify encoding when writing output file to avoid errors when default target encoding doesn't have all characters. utf8 seems like the most universal and supported encoding. Otherwise, the cli fails with encoding errors when input file contains unicode text (basically most files nowadays) and the target system has default encoding set to some one-byte charset like cp1252

Signed-off-by: Johnny Salazar <cepera.ang@gmail.com>
This commit is contained in:
Johnny Salazar 2024-11-03 18:21:19 +07:00 committed by GitHub
parent 244ca69cfd
commit 97aef61667
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docling/cli/main.py
View File

@ -90,28 +90,28 @@ def export_documents(
# Export Deep Search document JSON format:
if export_json:
fname = output_dir / f"{doc_filename}.json"
with fname.open("w") as fp:
with fname.open("w", encoding="utf8") as fp:
_log.info(f"writing JSON output to {fname}")
fp.write(json.dumps(conv_res.document.export_to_dict()))
# Export Text format:
if export_txt:
fname = output_dir / f"{doc_filename}.txt"
with fname.open("w") as fp:
with fname.open("w", encoding="utf8") as fp:
_log.info(f"writing Text output to {fname}")
fp.write(conv_res.document.export_to_markdown(strict_text=True))
# Export Markdown format:
if export_md:
fname = output_dir / f"{doc_filename}.md"
with fname.open("w") as fp:
with fname.open("w", encoding="utf8") as fp:
_log.info(f"writing Markdown output to {fname}")
fp.write(conv_res.document.export_to_markdown())
# Export Document Tags format:
if export_doctags:
fname = output_dir / f"{doc_filename}.doctags"
with fname.open("w") as fp:
with fname.open("w", encoding="utf8") as fp:
_log.info(f"writing Doc Tags output to {fname}")
fp.write(conv_res.document.export_to_document_tokens())