Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
fix(docs): fix some broken links in the docs
  • Loading branch information
hsheth2 committed Jul 8, 2021
commit c0a0aa2b30cbe5d2fa7941d52540cf0a99ac9a9a
2 changes: 1 addition & 1 deletion docs-website/generateDocsDir.ts
Original file line number Diff line number Diff line change
Expand Up @@ -305,7 +305,7 @@ function markdown_sanitize_and_linkify(content: string): string {
// Link to issues/pull requests.
content = content.replace(
/#(\d+)\b/g,
"[#$1](https://github.com/linkedin/datahub/issues/$1)"
"[#$1](https://github.com/linkedin/datahub/pull/$1)"
);

return content;
Expand Down
2 changes: 1 addition & 1 deletion docs/how/build-metadata-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ public interface GraphBuilder<SNAPSHOT extends RecordTemplate> {
[RecordTemplate]: https://github.com/linkedin/rest.li/blob/master/data/src/main/java/com/linkedin/data/template/RecordTemplate.java
[GenericRecord]: https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/generic/GenericRecord.java
[DAO]: https://en.wikipedia.org/wiki/Data_access_object
[Pegasus]: https://linkedin.github.io/rest.li/DATA-Data-Schema-and-Templates
[Pegasus]: https://linkedin.github.io/rest.li/pdl_schema
[relationship]: ../what/relationship.md
[entity]: ../what/entity.md
[aspect]: ../what/aspect.md
Expand Down
4 changes: 2 additions & 2 deletions docs/links.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Articles & Talks

## Articles
## Related Articles
* [DataHub: A Generalized Metadata Search & Discovery Tool](https://engineering.linkedin.com/blog/2019/data-hub)
* [Open sourcing DataHub: LinkedIn’s metadata search and discovery platform](https://engineering.linkedin.com/blog/2020/open-sourcing-datahub--linkedins-metadata-search-and-discovery-p)
* [Data Catalogue — Knowing your data](https://medium.com/albert-franzi/data-catalogue-knowing-your-data-15f7d0724900)
* [LinkedIn Datahub Application Architecture Quick Understanding](https://medium.com/@liangjunjiang/linkedin-datahub-application-architecture-quick-understanding-a5b7868ee205)
* [LinkIn Datahub Metadata Ingestion Scripts Unofficical Guide](https://medium.com/@liangjunjiang/linkin-datahub-etl-unofficical-guide-7c3949483f8b)
* [Datahub - RPubs](https://rpubs.com/Priya_Shaji/dataHub)
* [A Dive Into Metadata Hubs](https://www.holistics.io/blog/a-dive-into-metadata-hubs/)
* [How LinkedIn, Uber, Lyft, Airbnb and Netflix are Solving Data Management and Discovery for Machine Learning Solutions](https://towardsdatascience.com/how-linkedin-uber-lyft-airbnb-and-netflix-are-solving-data-management-and-discovery-for-machine-9b79ee9184bb)
* [How LinkedIn, Uber, Lyft, Airbnb and Netflix are Solving Data Management and Discovery for Machine Learning Solutions](https://www.kdnuggets.com/2019/08/linkedin-uber-lyft-airbnb-netflix-solving-data-management-discovery-machine-learning-solutions.html)
* [Data Discovery in 2020](https://medium.com/@torokyle/data-discovery-in-2020-3c907383caa0)
* [Work-Bench Snapshot: The Evolution of Data Discovery & Catalog](https://medium.com/work-bench/work-bench-snapshot-the-evolution-of-data-discovery-catalog-2f6c0425616b)
* [In-house Data Discovery platforms](https://datastrategy.substack.com/p/in-house-data-discovery-platforms)
Expand Down
14 changes: 7 additions & 7 deletions docs/modeling/extending-the-metadata-model.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
---
title: Extending the metadata model
sidebar_label: Extending the metadata model
slug: /metadata-modeling/extending-the-metadata-model
---

# How do I extend the Metadata model?
# Extending the Metadata Model

You can extend the metadata model by either creating a new Entity or extending an existing one. Unsure if you need to
create a new entity or add an aspect to an existing entity? Read [metadata-model](./metadata-model.md) to understand
Expand All @@ -15,9 +13,9 @@ Dashboard Entity. If you want to extend an existing Entity, you can skip directl

At a high level, an entity is made up of

1) a union of Aspects, or bundles of related metadata,
2) a Key Aspect, which uniquely identifies an instance of an entity,
3) A snapshot, which pairs a group of aspects with a serialized key, or urn.
1. a union of Aspects, or bundles of related metadata,
2. a Key Aspect, which uniquely identifies an instance of an entity,
3. A snapshot, which pairs a group of aspects with a serialized key, or urn.

## Defining an Entity

Expand Down Expand Up @@ -62,7 +60,9 @@ record DashboardKey {

The Urn representation of the Key shown above would be:

```urn:li:dashboard:(<tool>,<id>)```
```
urn:li:dashboard:(<tool>,<id>)
```

Because they are aspects, keys need to be annotated with an @Aspect annotation, This instructs DataHub that this struct can be a part of.

Expand Down
2 changes: 1 addition & 1 deletion docs/what/mxe.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The models defined in [snapshot](snapshot.md) and [delta](delta.md) are used to

We also model MXEs using [PDL](https://linkedin.github.io/rest.li/pdl_schema) and rely on the [pegasus gradle plugin](https://linkedin.github.io/rest.li/setup/gradle#generateavroschema) to convert them into [AVSC](https://avro.apache.org/docs/current/spec.html). However, we also need to rename all the namespaces of the generated AVSC to avoid namespace clashes for projects that depend on both the PDL models and MXEs.

As the AVSC and PDL model schemas are 100% compatible, it’d be very easy to convert the in-memory representation from one to another using [Pegasus’ DataTranslator](https://linkedin.github.io/rest.li/DATA-Data-Schema-and-Templates#translating-data-to-and-from-avro).
As the AVSC and PDL model schemas are 100% compatible, it’d be very easy to convert the in-memory representation from one to another using [Pegasus’ DataTranslator](https://linkedin.github.io/rest.li/avro_translation).

## Metadata Change Event (MCE)

Expand Down