`AsyncBearerTokenCredentialPolicy` depends on `asyncio`

[jhogklint-dk]

• Package Name: azure-synapse-artifacts, azure-identity
• Package Version: 0.18.0, 1.15.0
• Operating System: Arch Linux 6.5.5-arch1-1
• Python Version: Python 3.11.5

Describe the bug
AsyncBearerTokenCredentialPolicy depends on asyncio which causes race conditions when the library is used with trio. _authentication_async.py#L41.

When a single event loop task authenticates, the asyncio.Lock never waits, so no problem is seen. When two tasks happen to do it at the same time, getting the asyncio event loop fails.

To Reproduce
Steps to reproduce the behavior:

1. pip install azure-synapse-artifacts==0.18.0 azure-identity==1.15.0 trio==0.22.2
2. Replace the *** in the below file and then run it.

import trio
from datetime import datetime
from azure.core.pipeline.transport import TrioRequestsTransport
from azure.identity.aio import ClientSecretCredential
from azure.synapse.artifacts.aio import ArtifactsClient
from azure.synapse.artifacts.models import RunFilterParameters

async def list_runs(client):
    await client.pipeline_run.query_pipeline_runs_by_workspace(
        RunFilterParameters(
            last_updated_after=datetime.now(),
            last_updated_before=datetime.now(),
        )
    )

async def parent():
    async with ArtifactsClient(
        credential=ClientSecretCredential(
            client_id="***",
            client_secret="***",
            tenant_id="***",
            transport=TrioRequestsTransport(),
        ),
        endpoint="***",
        transport=TrioRequestsTransport(),
    ) as client, trio.open_nursery() as nursery:
        nursery.start_soon(list_runs, client)
        nursery.start_soon(list_runs, client)
trio.run(parent)

3. Gives the following error

Traceback

Traceback (most recent call last):
  File "/tmp/asyncio_policy.py", line 34, in <module>
    trio.run(parent)
  File "/tmp/.venv/lib/python3.11/site-packages/trio/_core/_run.py", line 2093, in run
    raise runner.main_task_outcome.error
  File "/tmp/asyncio_policy.py", line 20, in parent
    async with ArtifactsClient(
  File "/tmp/.venv/lib/python3.11/site-packages/trio/_core/_run.py", line 881, in __aexit__
    raise combined_error_from_nursery
  File "/tmp/asyncio_policy.py", line 11, in list_runs
    await client.pipeline_run.query_pipeline_runs_by_workspace(
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/tracing/decorator_async.py", line 77, in wrapper_use_tracer
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.venv/lib/python3.11/site-packages/azure/synapse/artifacts/aio/operations/_pipeline_run_operations.py", line 147, in query_pipeline_runs_by_workspace
    pipeline_response: PipelineResponse = await self._client._pipeline.run(  # pylint: disable=protected-access
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 221, in run
    return await first_node.send(pipeline_request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 2 more times]
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/pipeline/policies/_redirect_async.py", line 73, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/pipeline/policies/_retry_async.py", line 179, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/pipeline/policies/_authentication_async.py", line 92, in send
    await await_result(self.on_request, request)
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/pipeline/_tools_async.py", line 58, in await_result
    return await result
           ^^^^^^^^^^^^
  File "/tmp/.venv/lib/python3.11/site-packages/azure/core/pipeline/policies/_authentication_async.py", line 56, in on_request
    async with self._lock:
  File "/usr/lib/python3.11/asyncio/locks.py", line 15, in __aenter__
    await self.acquire()
  File "/usr/lib/python3.11/asyncio/locks.py", line 106, in acquire
    fut = self._get_loop().create_future()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'create_future'

Expected behavior
I expect both tasks to not use asyncio, so that they both execute to completion.

Screenshots
N/A

Additional context
I am able to work around it with a custom policy but would like native support for it. Preferably without the need to specify credential and scopes in the application code as that is already given to/handled by ArtifactsClient and ArtifactsClientConfiguration.

class TrioAsyncBearerTokenCredentialPolicy(AsyncBearerTokenCredentialPolicy):
    def __init__(self, credential: AsyncTokenCredential, *scopes: str, **kwargs: Any) -> None:
        super().__init__(credential, *scopes, **kwargs)
        self._lock = trio.Lock()

Thanks

[kristapratico]

@jhogklint-dk thanks for your detailed issue, we'll take a look and get back to you soon.

[xiangyan99]

Thanks for reporting the issue.

Yes. This is a real problem and we are working on it.

Will keep you updated in the issue.