Skip to content

Implemented functionality to calculate all local scores using Realm #213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Givikap120 wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Implemented functionality to calculate all local scores using Realm #213

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
62294c9
ported from master
Givikap120 Aug 13, 2024
8ec4ed2
Update LazerCalculationSettings.cs
Givikap120 Aug 13, 2024
1c81765
fixed CI
Givikap120 Aug 13, 2024
ce15637
CI fix again
Givikap120 Aug 13, 2024
ed1d7f6
Update ProfileScreen.cs
Givikap120 Aug 13, 2024
eef465b
updated the text and disabled RX filtering
Givikap120 Aug 14, 2024
dd2123e
fixed scorev1 simulation not working
Givikap120 Aug 22, 2024
9acd859
added HR check
Givikap120 Aug 28, 2024
5f912bf
removed arbitrary filtering
Givikap120 Aug 31, 2024
a2b4f3b
removed bindables array
Givikap120 Aug 31, 2024
6d082ac
changed error to unknown
Givikap120 Aug 31, 2024
417c6de
changed hashing
Givikap120 Sep 1, 2024
2b28193
renamed `currentUser`
Givikap120 Sep 2, 2024
dc560e3
added functionality to calculate offline profile
Givikap120 Sep 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
added functionality to calculate offline profile
  • Loading branch information
@Givikap120
Givikap120 committed Sep 2, 2024
commit dc560e3b955044f8bc4a8f15ca25bde848a019e1
28 changes: 21 additions & 7 deletions PerformanceCalculatorGUI/Screens/ProfileScreen.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
using PerformanceCalculatorGUI.Configuration;
using System.IO;
using osu.Framework.Platform;
using osu.Game.Screens.Play;

namespace PerformanceCalculatorGUI.Screens
{
Expand Down Expand Up @@ -411,9 +412,24 @@ private void calculateProfileFromLazer(string username)
{
Schedule(() => loadingLayer.Text.Value = "Getting user data...");

var player = await apiManager.GetJsonFromApi<APIUser>($"users/{username}/{ruleset.Value.ShortName}");
APIUser player = null;

try
{
player = await apiManager.GetJsonFromApi<APIUser>($"users/{username}/{ruleset.Value.ShortName}");

currentUserNicknames = [player.Username, .. player.PreviousUsernames, player.Id.ToString()];

currentUserNicknames = [player.Username, .. player.PreviousUsernames, player.Id.ToString()];
} catch {
notificationDisplay.Display(new Notification("Unable to find player on the servers, using local name..."));

player = new APIUser
{
Username = username
};

currentUserNicknames = [username];
}

Schedule(() =>
{
Expand All @@ -430,11 +446,8 @@ private void calculateProfileFromLazer(string username)

if (token.IsCancellationRequested)
return;

var plays = new List<ProfileScore>();

var rulesetInstance = ruleset.Value.CreateInstance();

var lazerPath = configManager.GetBindable<string>(Settings.LazerFolderPath).Value;

if (lazerPath == string.Empty)
Expand All @@ -444,6 +457,7 @@ private void calculateProfileFromLazer(string username)
}

var storage = gameHost.GetStorage(lazerPath);

var realmAccess = new RealmAccess(storage, @"client.realm");
Copy link
Member

@stanriders stanriders Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tools should never use lazer's database directly - please copy to a local directory and use that instead

Copy link
Contributor

@minisbett minisbett Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then please add the least add buttons to copy lazers data into a local copy and be able to sync it

Copy link
Member

@stanriders stanriders Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't, it should be syncronized automatically at the start of profile calculation.

Copy link
Contributor

@minisbett minisbett Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh thats an option too, but why should it be cloned instead of accessing Lazer directly in the first place? Issues if Lazer is running at the same time?

Copy link
Contributor Author

@Givikap120 Givikap120 Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tools should never use lazer's database directly - please copy to a local directory and use that instead

copying 30+ gigs of data? sounds very bad

Copy link
Contributor

@smoogipoo smoogipoo Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either that or maybe have the user select the client.db?

Furthermore, I wonder if you'd be better off splitting into two processes:

  1. Select a db file, copy it locally, recalculate scores given whatever parameters, write values back to the DB.
  2. Select a db file, display scores.

You could implement (1) as a CLI command that writes the DB to an output path.

dotnet run -- recalc-lazer-db /path/to/input/client.ream /path/to/output/client.realm [--options]

Copy link
Contributor Author

@Givikap120 Givikap120 Sep 1, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a good way to make copied client.realm to contain ONLY replay files
not only this will be more convenient - it will be also faster

Copy link
Contributor

@smoogipoo smoogipoo Sep 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Faster how? You said it's a 50MB file? That takes 2 seconds on a 2000s era HDD, and is practically instantaneous on anything faster.

I suggest you don't go pre-optimising.

Copy link
Contributor Author

@Givikap120 Givikap120 Sep 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not about copying
The first 28 seconds of 3 minutes calculation was spent to collect all scores from the database, what is quite big portion of the time
Source:
https://www.youtube.com/watch?v=KEYa7Y-UJCw

Copy link
Contributor

@smoogipoo smoogipoo Sep 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a problem with the structure of your code, because you're using Detach(). You should restructure so you don't have to do that, such as using the Live<> class or otherwise.

Copy link
Member

@stanriders stanriders Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RealmAccess definitely should not be created here - having it in a Task is bound to have threading-related issues

Copy link
Contributor

@smoogipoo smoogipoo Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be fine as long as it's constrained to a single thread. If you're passing the realm or objects retrieved within it that are not .Detach()ed between threads, then there'll be an issue.

Copy link
Member

@stanriders stanriders Aug 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It already crashes on someone's linux machine because of cross-thread access, I think it'd just be safer to use it through safeguards instead of hoping that TPL wouldn't do something funny


var realmScores = getRealmScores(realmAccess);
Expand Down Expand Up @@ -472,7 +486,7 @@ private void calculateProfileFromLazer(string username)
if (token.IsCancellationRequested)
return;

Schedule(() => loadingLayer.Text.Value = $"Calculating {player.Username}'s scores... {currentScoresCount} / {totalScoresCount}");
Schedule(() => loadingLayer.Text.Value = $"Calculating {username}'s scores... {currentScoresCount} / {totalScoresCount}");

if (score.BeatmapInfo == null)
continue;
Expand Down Expand Up @@ -568,7 +582,7 @@ private List<List<ScoreInfo>> getRealmScores(RealmAccess realm)

Schedule(() => loadingLayer.Text.Value = "Filtering scores...");

realmScores.RemoveAll(x => !currentUserNicknames.Contains(x.User.Username) // Wrong username
realmScores.RemoveAll(x => !currentUserNicknames.Any(nickname => nickname.Equals(x.User.Username, StringComparison.OrdinalIgnoreCase)) // Wrong username
|| x.BeatmapInfo == null // No map for score
|| x.Passed == false || x.Rank == ScoreRank.F // Failed score
|| x.Ruleset.OnlineID != ruleset.Value.OnlineID // Incorrect ruleset
Expand Down