Skip to content

[system tests] Add new settings to wait for documents being ingested #2429

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Feb 27, 2025
Merged

[system tests] Add new settings to wait for documents being ingested #2429

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
c6819e8
Add assert option: min_count
mrodm Feb 21, 2025
e1a9c8d
Add assert option: fields present
mrodm Feb 21, 2025
cb1f66c
Update docs
mrodm Feb 24, 2025
c9242aa
Add option to wait for N seconds without new docs
mrodm Feb 24, 2025
0890fe9
Skip check if there are no docs
mrodm Feb 24, 2025
724d496
Rename assert setting
mrodm Feb 24, 2025
cf0fece
Allow to run several assert at the same time
mrodm Feb 24, 2025
0fc7868
Add function for assert.hit_count
mrodm Feb 24, 2025
1675367
Just look for fields not found previously
mrodm Feb 25, 2025
eddb8e9
Add test package for assert.fields_present
mrodm Feb 25, 2025
d552144
Rename assert condition
mrodm Feb 25, 2025
ef9862b
Update comments assert struct fields
mrodm Feb 25, 2025
273e4d6
Rename package and add two system tests
mrodm Feb 25, 2025
37277fc
Remove check between hit_count and min_count
mrodm Feb 25, 2025
571330e
Add check for maximum value of assert.hit_count/min_count
mrodm Feb 26, 2025
b37f7d2
Remove ingestion_idle_time setting
mrodm Feb 26, 2025
249dc83
Update assert checks and min count condition
mrodm Feb 26, 2025
58b4ca3
Remove debug message
mrodm Feb 26, 2025
492f6ca
Add false positive test package for assert.fields_present
mrodm Feb 26, 2025
a81cbdf
Update error message about hits not found
mrodm Feb 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update assert checks and min count condition
  • Loading branch information
@mrodm
mrodm committed Feb 26, 2025
commit 249dc831b9ba5e883674080318ff870239e2d321
13 changes: 9 additions & 4 deletions internal/testrunner/runners/system/tester.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1542,11 +1542,11 @@ func (r *tester) waitForDocs(ctx context.Context, config *testConfig, dataStream
waitForDataTimeout = config.WaitForDataTimeout
}

if config.Assert.HitCount >= elasticsearchQuerySize {
if config.Assert.HitCount > elasticsearchQuerySize {
return nil, fmt.Errorf("invalid value for assert.hit_count (%d): it must be lower of the maximum query size (%d)", config.Assert.HitCount, elasticsearchQuerySize)
}

if config.Assert.MinCount >= elasticsearchQuerySize {
if config.Assert.MinCount > elasticsearchQuerySize {
return nil, fmt.Errorf("invalid value for assert.min_count (%d): it must be lower of the maximum query size (%d)", config.Assert.MinCount, elasticsearchQuerySize)
}

Expand Down Expand Up @@ -1625,8 +1625,13 @@ func (r *tester) waitForDocs(ctx context.Context, config *testConfig, dataStream
return true
}()

// By default, config.Assert.MinCount is zero
assertMinCount := hits.size() > config.Assert.MinCount
assertMinCount := func() bool {
if config.Assert.MinCount > 0 {
return hits.size() >= config.Assert.MinCount
}
// By default at least one document
return hits.size() > 0
}()

return assertFieldsPresent && assertMinCount && assertHitCount, nil
}, 1*time.Second, waitForDataTimeout)
Expand Down