Added multiprocessing for image push/pulls based on input rate#53
Added multiprocessing for image push/pulls based on input rate#53vishnuchalla merged 6 commits intoquay:masterfrom vishnuchalla:dev
Conversation
tests.py
Outdated
| 'start_time': start_time, | ||
| 'end_time': end_time, | ||
| 'failures': failures, | ||
| 'successes': successes, |
There was a problem hiding this comment.
consider names like success_count and failure_count to avoid confusion
There was a problem hiding this comment.
Sure. Updated as suggested.
| def parallel_process(user, **kwargs): | ||
| common_args = kwargs | ||
| # Container Operations | ||
| redis_client.delete('tags_to_push'+"-".join(user.split("_"))) # avoid stale data |
There was a problem hiding this comment.
are the parallel operations only for push/pull?
There was a problem hiding this comment.
No. For rest of the APIS, we hit endpoints using vegeta. So the concurrency is automatically handled by the --rate option in the tool. Since we execute podman commands for push/pull we have to explicitly implement parallel processing for them as in the above code.
| logger.info("Created Job: %s", resp.metadata.name) | ||
|
|
||
|
|
||
| def parallel_process(user, **kwargs): |
There was a problem hiding this comment.
can you write a small description on how this function works and how you are using redis etc
There was a problem hiding this comment.
Sure. Added the description. Please let me know if you have any further questions on functionality.
2 participants
Description
Added multiprocessing for image push/pulls using the input rate specified.
If we have n users, n push/pull jobs will be created with given rate rps, each uploading and downloading 100 images.
Testing
Tested on self managed openshift cluster.