Skip to content

Notebooks failing in readthedocs build does not trigger red status #799

@AdamGleave

Description

@AdamGleave

Bug description

Running notebooks is currently failing in the readthedocs build but as a warning only so CI wrongly looks green.

Steps to reproduce

From the logs, there are a variety of failures that are warnings only:

/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/3_train_gail.ipynb: WARNING: Executing notebook failed: CellTimeoutError [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/3_train_gail.ipynb: WARNING: Notebook exception traceback saved in: /home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/_readthedocs/html/reports/tutorials/3_train_gail.err.log [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/4_train_airl.ipynb: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/4_train_airl.ipynb: WARNING: Notebook exception traceback saved in: /home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/_readthedocs/html/reports/tutorials/4_train_airl.err.log [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/5a_train_preference_comparisons_with_cnn.ipynb: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/5a_train_preference_comparisons_with_cnn.ipynb: WARNING: Notebook exception traceback saved in: /home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/_readthedocs/html/reports/tutorials/5a_train_preference_comparisons_with_cnn.err.log [mystnb.exec]
/home/docs/checkouts/readthedocs.org/user_builds/imitation/checkouts/latest/docs/tutorials/8a_train_sqil_sac.ipynb: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec]

Suggest we either make these warnings fatal, or unskip testing the notebooks in tests/test_examples.py so we test them on Linux again.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions