page 2 blended with page 1 on output via append

[kimberv]

I need to read in a fillable form pdf, fill it out, then generate result pdf. this is largely working, except that page 2 of the pdf (which contains only text) doesn't get properly appended. It's text contents are laid over the top of a copy of page 1.

Environment

Which environment were you using when you encountered the problem?

$ python -m platform
macOS-11.6.5-x86_64-i386-64bit

$ python -c "import pypdf;print(pypdf.__version__)"
3.3.0

Code + PDF

This is a minimal, complete example that shows the issue:

from pypdf import PdfReader, PdfWriter

reader = PdfReader('cms1500.pdf')
writer = PdfWriter()
writer.append(reader)

with open('copy.pdf', "wb") as output_stream:
    writer.write(output_stream)

Pdf source is
form-cms1500.pdf - a standard pdf in healthcare. There should be no problem with you using it in test suites.

Results

Outcome from using writer.append:

Outcome from using writer.append(reader, [1])

NOTE: Outcome from using writer.clone_document_from_reader does produce expected output BUT i can't use it as it causes other problems (namely after clone writer.pages items appear to be DictionaryObject and not PageObject):

[kimberv]

I'd love any insight on the above issue or if you need any additional information.

[pubpub-zz]

I've started to have a look and I'm very confused : the page content shown no different when copied into the new PDF.
The big difference when using clone_document_from_reader() you are copying some fields/object from "/Root".
I have not been able to identify yet where's the fields modifying the behavior

Meanwhile, I've prepared a PR for returning PageObject when using clone_document_from_reader(), can you check it

PS : Found the field which is changing the behavior: /OCProperties. To be understood... 🤔

[pubpub-zz]

@kimberv, you looked in a hurry... Can you provide update?

[kimberv]

I can confirm that this change allows me to use writer.clone_document_from_reader(reader) as expected, and the resulting pdf has a "clean" page 2! I look forward to this patch being part of a release! (to be clear to any curious reader though - append(reader) still has the above problem for this pdf)

[pubpub-zz]

As sais, the difference of behavior between page1 and 2 is due to the "/OCProperties" within "/Root". There is no indirect object making link with the pages : Append() will not be able to copy these data

[MartinThoma]

The fix was just merged and will be part of the pypdf>3.4.1 release this weekend (26.02.2023)