Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
acf1cdc
Initial setup of Observability OTEL demo using Opensearch as store
YANG-DB Apr 26, 2023
c842150
Initial setup of Observability OTEL demo using Opensearch as store
YANG-DB Apr 26, 2023
b83ea99
Merge remote-tracking branch 'origin/opensearch-observability-otel' i…
YANG-DB Apr 26, 2023
9850688
Initial setup of Observability OTEL demo using Opensearch as store
YANG-DB May 1, 2023
0db3223
Merge branch 'main' into opensearch-observability-otel
YANG-DB May 2, 2023
bf41276
add curl to the basic docker O/S
YANG-DB May 3, 2023
4292136
Merge pull request #1 from open-telemetry/main
YANG-DB May 3, 2023
f0600be
add monitoring & opinionated assets for this demo
YANG-DB May 4, 2023
418d196
Merge remote-tracking branch 'origin/opensearch-observability-otel' i…
YANG-DB May 4, 2023
b0db9fc
export F/E & Load-Generator port services
YANG-DB May 8, 2023
9629fb7
Merge pull request #2 from open-telemetry/main
YANG-DB May 8, 2023
ba709f1
fix jaeger service name
YANG-DB May 9, 2023
daedfab
adding nginx as frontend proxy, fluent-bit.conf as log emitter
YANG-DB May 10, 2023
d58ed46
adding nginx & fluent-bit docs
YANG-DB May 10, 2023
a8e3f5d
Merge pull request #3 from open-telemetry/main
YANG-DB May 15, 2023
b6a018a
adding integrationbs docker-compose service for assets loading into o…
YANG-DB May 15, 2023
6031c17
Merge remote-tracking branch 'origin/opensearch-observability-otel' i…
YANG-DB May 15, 2023
92240ba
adding datasource for query prometheus
YANG-DB May 15, 2023
12d52cb
adding architecture.md including images
YANG-DB May 15, 2023
85fefce
add images and service links
YANG-DB May 16, 2023
73e2680
add jaeger-agent container name
YANG-DB May 16, 2023
f6067c6
add data-prepper support for traces ingestion & serviceMap creation
YANG-DB May 16, 2023
24d278c
fix auth issues for data-prepper
YANG-DB May 16, 2023
7410478
add documentation for different ingestion capabilities
YANG-DB May 16, 2023
d8a2abf
Merge pull request #4 from YANG-DB/otel-opensearch-data-prep
YANG-DB May 16, 2023
fe655cc
Initial setup of Observability OTEL demo using Opensearch as store
YANG-DB Apr 26, 2023
0be11cd
Initial setup of Observability OTEL demo using Opensearch as store
YANG-DB May 1, 2023
f54a011
add curl to the basic docker O/S
YANG-DB May 3, 2023
90c9a61
add monitoring & opinionated assets for this demo
YANG-DB May 4, 2023
379372a
[shippingservice] update rust version and dependencies (#865)
julianocosta89 May 3, 2023
6ae5c80
[load generator] Bump loagen dependencies (#869)
julianocosta89 May 3, 2023
a69595a
[grafana] fix demo dashboard to be compatible with spanmetrics connec…
fatsheep9146 May 3, 2023
1c28220
export F/E & Load-Generator port services
YANG-DB May 8, 2023
42537be
[kafka] remove KRaft mode support workarounds (#880)
styblope May 8, 2023
8bd25ad
enabling batch span processor metrics for quoteservice (#878)
brettmc May 8, 2023
2cc7484
Generate random errors in cartservice (#824)
alanwest May 8, 2023
54e3b6f
fix jaeger service name
YANG-DB May 9, 2023
daf0d08
adding nginx as frontend proxy, fluent-bit.conf as log emitter
YANG-DB May 10, 2023
88e0171
adding nginx & fluent-bit docs
YANG-DB May 10, 2023
d292741
adding integrationbs docker-compose service for assets loading into o…
YANG-DB May 15, 2023
6f55bd8
[currencyservice] Fix OTel C++ build and update OTel version (#886)
julianocosta89 May 12, 2023
193552f
adding datasource for query prometheus
YANG-DB May 15, 2023
febdfe1
adding architecture.md including images
YANG-DB May 15, 2023
904ed80
add images and service links
YANG-DB May 16, 2023
9cf4326
add jaeger-agent container name
YANG-DB May 16, 2023
c14aa11
add data-prepper support for traces ingestion & serviceMap creation
YANG-DB May 16, 2023
a55d74f
fix auth issues for data-prepper
YANG-DB May 16, 2023
6032e3e
add documentation for different ingestion capabilities
YANG-DB May 16, 2023
5954bdf
Merge remote-tracking branch 'origin/opensearch-observability-otel' i…
YANG-DB May 18, 2023
4d1d72e
add assets manager flast app for loading the integration assets
YANG-DB May 18, 2023
91f7a0e
remove experimental assets mgr app
YANG-DB May 18, 2023
78b38e6
Opensearch otel demo observability tutorial (#5)
YANG-DB May 23, 2023
fe8a551
Merge pull request #2 from open-telemetry/main
YANG-DB May 26, 2023
b422bc7
Update local-config.md
YANG-DB May 26, 2023
4431d7b
Update README.md
YANG-DB May 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
adding architecture.md including images
Signed-off-by: YANGDB <[email protected]>
  • Loading branch information
YANG-DB committed May 18, 2023
commit febdfe1cc29daa58dd3a69e06b15d4e4d9914f59
3 changes: 2 additions & 1 deletion .github/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ The next instructions are similar and use the same docker compose file.
docker-compose up -d
```

## Explore and analyze the data With OpenSearch Observability
### Explore and analyze the data With OpenSearch Observability
Review revised OpenSearch [Observability Architecture](architecture.md)

### Service map
![Service map](https://docs.aws.amazon.com/images/opensearch-service/latest/developerguide/images/ta-dashboards-services.png)
Expand Down
73 changes: 73 additions & 0 deletions .github/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Opensearch OTEL Demo Architecture
This document will review the OpenSearch architecture for the [OTEL demo](https://opentelemetry.io/docs/demo/) and will review how to use the new Observability capabilities
implemented into OpenSearch.

[OTEL DEMO](https://opentelemetry.io/docs/demo/architecture/) Describes the list of services that are composing the Astronomy Shop.

They are combined of:
- [Accounting](https://opentelemetry.io/docs/demo/services/accounting/)
- [Ad](https://opentelemetry.io/docs/demo/services/ad/)
- [Cart](https://opentelemetry.io/docs/demo/services/cart/)
- [Checkout](https://opentelemetry.io/docs/demo/services/checkout/)
- [Currency](https://opentelemetry.io/docs/demo/services/currency/)
- [Email](https://opentelemetry.io/docs/demo/services/email/)
- [Feature Flag](https://opentelemetry.io/docs/demo/services/feature-flag/)
- [Fraud Detection](https://opentelemetry.io/docs/demo/services/fraud-detection/)
- [Frontend](https://opentelemetry.io/docs/demo/services/frontend/)
- [Frontend Nginx Proxy](../src/nginx-otel/README.md) *(replacement for _Frontend-Proxy_)*
- [Kafka](https://opentelemetry.io/docs/demo/services/kafka/)
- [Load Generator](https://opentelemetry.io/docs/demo/services/load-generator/)
- [Payment](https://opentelemetry.io/docs/demo/services/payment/)
- [Product Catalog](https://opentelemetry.io/docs/demo/services/product-catalog/)
- [Quote](https://opentelemetry.io/docs/demo/services/quote/)
- [Recommendation](https://opentelemetry.io/docs/demo/services/recommendation/)
- [Shipping](https://opentelemetry.io/docs/demo/services/shipping/)
- [Fluent-Bit](../src/fluent-bit/README.md) *(nginx's otel log exported)*
- [Integrations](../src/integrations/README.md) *(pre-canned OpenSearch assets)*


---

## Purpose
The purpose of this demo is to demonstrate the different capabilities of OpenSearch Observability to investigate and reflect your system.

- Integrations - the integration service is a list of pre-canned assets that are loaded in a combined manner to allow users the ability
for simple and automatic way to discover and review their services topology.

These integrations contain the following assets:
- components & index template mapping
- datasources
- data-stream & indices
- queries
- dashboards

Once they are loaded, the user can imminently review his OTEL demo services and dashboards that reflect the system state.
- [Nginx Dashboard](../src/integrations/display/nginx-logs-dashboard-new.ndjson) - reflects the Nginx Proxy server that routes all the network communication to/from the frontend
- [Prometheus datasource](../src/integrations/datasource/prometheus.json) - reflects the connectivity to the prometheus metric storage that allows us to federate metrics analytics queries
- [Logs Datastream](../src/integrations/indices/data-stream.json) - reflects the data-stream used by nginx logs ingestion and dashboards representing a well-structured [log schema](../src/integrations/mapping-templates/logs.mapping)

Once these assets are loaded - the user can start reviewing its Observability dashboards and traces

![Nginx Dashboard](img/nginx_dashboard.png)

![Prometheus Metrics](img/prometheus_federated_metrics.png)

![Trace Analytics](img/trace_analytics.png)

![Service Maps](img/services.png)

![Traces](img/traces.png)


---

### **Scenarios**

How can you solve problems with OpenTelemetry? These scenarios walk you through some pre-configured problems and show you how to interpret OpenTelemetry data to solve them.

- Generate a Product Catalog error for GetProduct requests with product id: OLJCESPC7Z using the Feature Flag service
- Discover a memory leak and diagnose it using metrics and traces. Read more

### **Reference**
Project reference documentation, like requirements and feature matrices [here](https://opentelemetry.io/docs/demo/#reference)

Binary file added .github/img/nginx_dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/img/prometheus_federated_metrics.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/img/services.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/img/trace_analytics.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .github/img/traces.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.