Skip to content

Conversation

@wenfang6
Copy link
Collaborator

@wenfang6 wenfang6 commented Oct 18, 2019

What changes were proposed in this pull request?

This PR solves the problem that MySQL data source can't partition. The current code don't load partition information of MySQL into memory when cache.level is 1 . The PR fixes the bug.

How was this patch tested?

No UT.

@beliefer
Copy link
Collaborator

mysql -> MySQL

@beliefer
Copy link
Collaborator

The PR fixs the bug. -> The PR fixes the bug .

@wenfang6
Copy link
Collaborator Author

The PR fixs the bug. -> The PR fixes the bug .

Thanks for your correction

}
if (partitionsParameters.nonEmpty) {
specialProperties += ((s"${dsName}.${dbName}.${tbName}", partitionsParameters))
}
Copy link
Collaborator

@beliefer beliefer Oct 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  Option(partitionsMap(dbName)(tbName)).foreach{ m =>
    specialProperties += ((s"${dsName}.${dbName}.${tbName}", m))
  }

tablePartitionsMap.get.get(tbName).foreach { m =>
specialProperties += ((s"${dsName}.${dbName}.${tbName}", m))
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  Option(partitionsMap(dbName)).map{ tablePartitionsMap =>
    Option(tablePartitionsMap(tbName)).foreach { m =>
      specialProperties += ((s"${dsName}.${dbName}.${tbName}", m))
    }
  }

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your advice, but the code can't pass.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@beliefer beliefer changed the title [CORE] Fix mysql datasource can't partition [JDBC] Fix mysql datasource can't partition Oct 22, 2019
@beliefer beliefer added the bug Something isn't working label Oct 22, 2019
beliefer pushed a commit that referenced this pull request Oct 22, 2019
This PR solves the problem that MySQL data source can't partition. The current code don't load partition information of MySQL into memory when `cache.level` is 1 . The PR fixes the bug.
beliefer pushed a commit that referenced this pull request Oct 22, 2019
This PR solves the problem that MySQL data source can't partition. The current code don't load partition information of MySQL into memory when `cache.level` is 1 . The PR fixes the bug.
@beliefer
Copy link
Collaborator

Thanks! Merged to master and branch-0.6

@beliefer beliefer closed this Oct 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants