Skip to content

Conversation

@malkaed
Copy link

@malkaed malkaed commented May 26, 2022

When running workers, Activity task worker and Decision task worker throw an asyncio.CancelledError error the first time they are started(not sure why), in python 3.7, the retry decorator excepts that error using except Exception and trys to restart the worker which solves the issue.
In python 3.8, asyncio.CancelledError became a subclass of BaseException instead of Exception, so the error is no longer excepted and is raised instead, causing the workers to not start.
The previous commit solved this issue for the Activity task worker, but not for the Decision task worker(which failed for our case), this PR solves both issues by adding handling of the asyncio.CancelledError error to the retry decorator so that it would behave exactly as it did in python 3.7.
Handling of asyncio.CancelledError is separated from handling of Exception to not call logger.error().

@NullOsama NullOsama requested a review from tolivedxb May 26, 2022 19:03
Copy link
Collaborator

@NullOsama NullOsama left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was working fine on local, but I got lots of unhandled exceptions during testing those changes on stg. See link and link. In the previous commit the log.error was omitted, So maybe you can catch the asyncio.CancelledError in a different block and just sleep initial time.

@malkaed malkaed merged commit aab27d8 into py310 May 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants