Skip to content

feat(ingest): expose additional types to Python via codegen #2712

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shirshanka merged 9 commits into from
Jun 17, 2021
Merged

feat(ingest): expose additional types to Python via codegen #2712

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
55aac8d
codegen audit event schemas too
hsheth2 May 6, 2021
2219c23
Merge branch 'master' into schema_merging
hsheth2 Jun 16, 2021
de927d4
refactor codegen py
hsheth2 Jun 16, 2021
abe6b5f
add kafka example recipe
hsheth2 Jun 17, 2021
3004b50
fixes for union schema
hsheth2 Jun 17, 2021
b7b1edc
Merge branch 'master' into schema_merging
hsheth2 Jun 17, 2021
0000698
rebuild models
hsheth2 Jun 17, 2021
2b6f633
fix newlines
hsheth2 Jun 17, 2021
dbc6a98
prune unused logic
hsheth2 Jun 17, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix newlines
  • Loading branch information
@hsheth2
hsheth2 committed Jun 17, 2021
commit 2b6f6333d6f6dbc38656f80b6f4d5136ce8b0b8b
3 changes: 1 addition & 2 deletions metadata-ingestion/scripts/avro_codegen.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ def suppress_checks_in_file(filepath: Union[str, Path]) -> None:

def _load_schema(schema_name: str) -> str:
return (pathlib.Path(__file__).parent / f"{schema_name}.avsc").read_text()

"""
individual_schema_method = """
@functools.lru_cache(maxsize=None)
Expand All @@ -93,7 +92,7 @@ def get{schema_name}Schema() -> str:


def make_load_schema_methods(schemas: Iterable[str]) -> str:
return load_schema_method + "\n".join(
return load_schema_method + "".join(
individual_schema_method.format(schema_name=schema) for schema in schemas
)

Expand Down
2 changes: 0 additions & 2 deletions metadata-ingestion/src/datahub/metadata/schemas/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,10 @@
def _load_schema(schema_name: str) -> str:
return (pathlib.Path(__file__).parent / f"{schema_name}.avsc").read_text()


@functools.lru_cache(maxsize=None)
def getMetadataChangeEventSchema() -> str:
return _load_schema("MetadataChangeEvent")


@functools.lru_cache(maxsize=None)
def getMetadataAuditEventSchema() -> str:
return _load_schema("MetadataAuditEvent")
Expand Down