[SPARK-18868][FLAKY-TEST] Deflake StreamingQueryListenerSuite: single listener, check trigger...

[brkyvz]

What changes were proposed in this pull request?

Use recentProgress instead of lastProgress and filter out last non-zero value. Also add eventually to the latest assertQuery similar to first assertQuery

How was this patch tested?

Ran test 1000 times

[SparkQA]

Test build #70163 has finished for PR 16287 at commit cedaafd.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #70202 has finished for PR 16287 at commit 32d1ca8.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #70207 has finished for PR 16287 at commit 50d1e44.

• This patch fails RAT tests.
• This patch merges cleanly.
• This patch adds no public classes.

[brkyvz]

retest this please

[SparkQA]

Test build #70203 has finished for PR 16287 at commit d4bbb45.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[brkyvz]

@zsxwing @tdas This passed 1,000 times on 2 jenkins tests. The latest test started tipping over after 1,100 retries

[SparkQA]

Test build #70211 has finished for PR 16287 at commit fc2bc34.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zsxwing]

LGTM. Merging to master and 2.1.

[SparkQA]

Test build #70212 has finished for PR 16287 at commit 95824f3.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #70208 has finished for PR 16287 at commit 50d1e44.

• This patch fails from timeout after a configured wait of `250m`.
• This patch merges cleanly.
• This patch adds no public classes.