Skip to content

[Refactor] Refactor common code between all benchmark commands #1611

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

[Refactor] Refactor common code between all benchmark commands #1611

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
505b5e7
[Refactor] Refactor common code between all benchmark commands
ruflin Dec 21, 2023
3de300c
remove obsolete methods
ruflin Dec 21, 2023
7bea775
add note about more common fields that colud use refactoring
ruflin Dec 21, 2023
e6c57b6
Merge branch 'main' into benchrunner-refactor
ruflin Jan 15, 2024
e5d444a
cleanup imporots
ruflin Jan 15, 2024
25d73b2
Merge branch 'main' into benchrunner-refactor
ruflin Jan 22, 2024
dab5303
remove todo
ruflin Jan 23, 2024
7548ee2
Merge branch 'main' into benchrunner-refactor
ruflin Jan 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add note about more common fields that colud use refactoring
  • Loading branch information
@ruflin
ruflin committed Dec 21, 2023
commit 7bea775e6e12dd2a5a419c66d8b634f32825ba3d
3 changes: 2 additions & 1 deletion internal/benchrunner/runners/system/scenario.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,13 @@ type scenario struct {
Corpora corpora `config:"corpora" json:"corpora"`
}

// TODO: Why is this and next one slightly different from the common fields?
// TODO: Why is this slightly different from the common fields?
Copy link
Contributor Author

@ruflin ruflin Jan 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aspacca You might know more here? And the one below?

Copy link

@paolafrancesca paolafrancesca Jan 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the data required for both data_stream and corpora configuration is different between system benchmarks and rally/stream ones.

specifically: rally/stream does not require data_stream.vars and corpora.input_service

since I "duplicated" both structs used to unmarshal the yaml configuration in the rally/stream packages that I added, I removed the fields that were not required.

it is fine to have this two structs only in the common package as well, with all the fields currently defined in the system package (ie: the "version" we have in this file). the packages where the fields are not required will just not access them. there's no impact on the specs from the underlying structs that are used to unmarshal the yaml.

the only caveat is that what can happen is that someone could add data_stream.vars or corpora.input_service in a configuration where they are not required and provide a value that cannot be unmarshalled. in this case we cannot simply ignore/not access the not required fields, the user has to remove them.

Copy link
Contributor Author

@ruflin ruflin Jan 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, I for now remove the 2 TODO mentions, we can always follow up on this.

type dataStream struct {
Name string `config:"name" json:"name"`
Vars map[string]interface{} `config:"vars" json:"vars"`
}

// TODO: Why is this slightly different from the common fields?
type corpora struct {
Generator *common.Generator `config:"generator" json:"generator"`
InputService *inputService `config:"input_service" json:"input_service"`
Expand Down