Adding push/pull phase

[vishnuchalla]

Description

Adding push/pull phase to only test push pull which a given user login details. This will be helpful to test both quay.io and quay-operator as well

Testing

Tested and verified by testing it on quay.io staging environment details shared by @SeanZhao-redhat

Input ENVs in the config

        env:
          - name: QUAY_HOST
            value: "stage_url"
          - name: QUAY_OAUTH_TOKEN
            value: "TOKEN"
          - name: QUAY_ORG
            value: "testorg"
          - name: ES_HOST
            value: "ES_URL"
          - name: ES_PORT
            value: "443"
          - name: PYTHONUNBUFFERED
            value: "0"
          - name: ES_INDEX
            value: "quay-vegeta"
          - name: PUSH_PULL_IMAGE
            value: "IMAGE_URL"
          - name: PUSH_PULL_ES_INDEX
            value: "quay-push-pull"
          - name: PUSH_PULL_NUMBERS
            value:  "5"
          - name: TARGET_HIT_SIZE
            value: "10"
          - name: CONCURRENCY
            value: "15"
          - name: TEST_NAMESPACE
            value: "quay-ptt"
          - name: "TEST_PHASES"
            value: "push_pull"
          - name: QUAY_USERNAME
            value: "USERNAME" 
          - name: QUAY_PASSWORD
            value: "PASSWORD"  

Output logs

vchalla@vchalla-thinkpadp1gen2:~/quay-performance-scripts$ oc logs pod/quay-perf-test-orchestrator-hfqdl
/tmp/main.py:662: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
  date=datetime.datetime.utcnow().isoformat(),
INFO:root:Running Quay Scale & Performance Tests	date=2024-09-23T19:57:09.966400 host=https://stage.quay.io test_uuid=260b587a-c528-4435-ac31-74db597e612c organization=seanorg num_users=10 num_repos=11 num_teams=10 target_hit_size=10 concurrency=15 repos_with_tags_sizes=(5,) total_tags=5 pull_push_batch_size=400
/tmp/main.py:698: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
  start_time = datetime.datetime.utcnow()
INFO:root:Starting image push/pulls (UTC): 2024-09-23 19:58:09.966724
INFO:root:Queued 5 tags to be created
INFO:root:Queued 5 tags to be pulled
INFO:root:Created Job: test-registry-pushquay-qe
INFO:root:Job test-registry-pushquay-qe has been completed.
INFO:root:Created Job: test-registry-pullquay-qe
INFO:root:Job test-registry-pullquay-qe has been completed.
/tmp/main.py:701: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
  end_time = datetime.datetime.utcnow()
INFO:root:Ending image push/pulls (UTC): 2024-09-23 20:00:10.049355

[Marcusk19]

Is it possible to split this further into separate push/pull tests? For things like the registry proxy only pulls are supported. If not, would we be able to run the push_pull test against the registry proxy and expect to see all image pushes failing?

[vishnuchalla]

would we be able to run the push_pull test against the registry proxy and expect to see all image pushes failing?

Yes

[Marcusk19]

Could we pass in a set of tags to pull explicitly via environment variables? i.e. have something like TAGS="registry.stage.redhat.io/ubi9/ubi,registry.stage.redhat.io/ubi9/podman". It looks like the tags are based on what the tests can push to quay, but for the registry proxy it won't be able to push any images

[vishnuchalla]

This ask seems very specific to proxy. I would say, use this PR to upload images without proxy and then pull them using a custom loop script (which should be pretty simple to write) by enabling the proxy if that makes sense

[Marcusk19]

lgtm