Skip to content

Utility/example for deleting empty project versions older than <d> days#174

Closed
OffBy0x01 wants to merge 9 commits intomasterfrom
ar-calder/#173
Closed

Utility/example for deleting empty project versions older than <d> days#174
OffBy0x01 wants to merge 9 commits intomasterfrom
ar-calder/#173

Conversation

@OffBy0x01
Copy link
Collaborator

  • Added new example for deleting empty project versions older than a given number of days
  • Made datetime conversion utilities more robust
  • Swapped own-implementations for builtin functions where appropriate

Was needing to write something like this anyway, though #173 gave me a little extra motivation :)

Note: I've not been able to fully test this script - please give it a go and let me know if everything looks right.

@OffBy0x01 OffBy0x01 requested a review from skiyooka May 14, 2021 21:38
yield curr_date
curr_date += delta

def min_iso8601():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I am all for deleting code, are these methods that you just recently added e.g. this year? Generally, deleting or changing method signatures are considered as breaking backwards-compatibility in an API.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of these were in HubInstance - the only use they had was in demo_client, though I'll update that to use the two new methods.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay I'm fine with deleting them.


for project in bd.get_resource('projects'):
# skip projects younger than max age
if to_datetime(project.get('createdAt')) > max_age: continue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm getting an exception here: TypeError: can't compare offset-naive and offset-aware datetimes

Copy link
Collaborator Author

@OffBy0x01 OffBy0x01 May 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A possible solution could be to add something like this to utils:

// from dateutil.tz import tzlocal
def to_local_datetime(date):
    return to_datetime(date).astimezone(tz=tzlocal())   

def get_local_datetime():
    return datetime.now(tz=tzlocal())

Normally I'd avoid creating wrappers for stl functions but in this case I don't think it is obvious how to get a timezone aware datetime given datetime.now() and even datetime.utcnow() are timezone naive. Thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In sage_version_activity_to_csv.py I used: from dateutil.parser import isoparse
Would that work?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take a look at that tonight

@skiyooka
Copy link
Contributor

You may find https://github.com/blackducksoftware/sage/blob/master/sage_version_activity_to_csv.py interesting as it creates a .csv with one line per project version. The .csv includes all columns I can think of containing decision criteria for deletion. The intended pipeline is to run Sage, then the above script, then filter_activity.py to trim the list to the stuff one just wants to delete.

@OffBy0x01 OffBy0x01 closed this Sep 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants