diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 000000000..7cec353e9 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,147 @@ +# AWS Reference Implementation + +Thank you for your interest in contributing to the AWS Reference Implementation! This guide will help you get started with contributing to this Internal Developer Platform (IDP) reference implementation for AWS. + +## Getting Started + +### Prerequisites + +Before contributing, ensure you have the following tools installed: + +- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) +- [kubectl](https://kubernetes.io/docs/tasks/tools/) +- [yq](https://mikefarah.gitbook.io/yq/v3.x) +- [helm](https://helm.sh/docs/intro/install/) +- [Git](https://git-scm.com/downloads) + +### Development Environment Setup + +1. Fork the repository to your GitHub organization +2. Clone your fork locally: + ```bash + git clone https://github.com/your-org/reference-implementation-aws.git + cd reference-implementation-aws + ``` +3. Set up your development environment following the [Getting Started](README.md#getting-started) guide + +## How to Contribute + +### Reporting Issues + +- Use GitHub Issues to report bugs or request features +- Search existing issues before creating new ones +- Provide detailed information including: + - Steps to reproduce + - Expected vs actual behavior + - Environment details (AWS region, EKS version, etc.) + - Relevant logs or error messages + +### Making Changes + +1. Create a feature branch: + ```bash + git checkout -b feature/your-feature-name + ``` + +2. Make your changes following the project structure: + - **Addons**: Add new addons in `packages/` directory + - **Scripts**: Update installation/management scripts in `scripts/` + - **Documentation**: Update docs in `docs/` directory + - **Templates**: Add Backstage templates in `templates/` + - **Examples**: Add usage examples in `examples/` + +3. Test your changes: + ```bash + # Test installation + ./scripts/install.sh + + # Verify addons are healthy + kubectl get applications -n argocd + ``` + +4. Commit your changes: + ```bash + git add . + git commit -m "feat: add new addon for X" + ``` + +5. Push and create a pull request: + ```bash + git push origin feature/your-feature-name + ``` + +## Project Structure + +``` +├── cluster/ # EKS cluster configurations (eksctl/terraform) +├── docs/ # Documentation and images +├── examples/ # Usage examples and demos +├── packages/ # Helm charts and addon configurations +├── private/ # GitHub App credentials (templates) +├── scripts/ # Installation and management scripts +├── templates/ # Backstage templates +└── config.yaml # Main configuration file +``` + +## Adding New Addons + +To add a new addon: + +1. Create a directory in `packages/your-addon/` +2. Add `values.yaml` with Helm chart configuration +3. Update `packages/addons/values.yaml` to include your addon +4. Add documentation in `docs/` if needed +5. Test the addon installation + +## Code Standards + +- Follow existing code style and patterns +- Use meaningful commit messages (conventional commits preferred) +- Update documentation for any user-facing changes +- Ensure scripts are executable and include proper error handling +- Test changes in a real EKS environment when possible + +## Pull Request Guidelines + +- Keep PRs focused on a single feature or fix +- Include tests or validation steps +- Update relevant documentation +- Reference related issues in PR description +- Ensure CI checks pass + +## Testing + +Before submitting: + +1. Test the full installation flow: + ```bash + ./scripts/install.sh + ``` + +2. Verify all addons are healthy: + ```bash + kubectl get applications -n argocd + ``` + +3. Test cleanup process: + ```bash + ./scripts/uninstall.sh + ``` + +## Documentation + +- Update README.md for user-facing changes +- Add or update documentation in `docs/` directory +- Include examples in `examples/` directory when applicable +- Update configuration tables and addon lists as needed + +## Getting Help + +- Check existing [documentation](docs/) +- Review [troubleshooting guide](docs/troubleshooting.md) +- Open an issue for questions or problems +- Join community discussions in GitHub Discussions + +## License + +By contributing, you agree that your contributions will be licensed under the same license as the project. \ No newline at end of file diff --git a/docs/examples/app-with-aws-resources.md b/docs/examples/app-with-aws-resources.md deleted file mode 100644 index e69de29bb..000000000 diff --git a/docs/examples/demo.md b/docs/examples/demo.md deleted file mode 100644 index 4bfe86704..000000000 --- a/docs/examples/demo.md +++ /dev/null @@ -1,43 +0,0 @@ -## Backstage template generation - -Using the CNOE CLI, you can generate Backstage template input fields from Terraform modules, Kubernetes CRDs, and Crossplane XRDs. - -### Terraform (imperative) - -For example, to generate input field from a terraform module in the Data on EKS repository: - -```bash -git clone https://github.com/awslabs/data-on-eks.git /tmp/data-on-eks - -cnoe template tf -i /tmp/data-on-eks/analytics/terraform/spark-k8s-operator -o examples/template-generation -t examples/template-generation/data-on-eks.yaml -p '.spec.parameters[0].properties.tfVars' - -``` -The above command takes the terraform module available at `/tmp/data-on-eks/analytics/terraform/spark-k8s-operator` then inserts generated backstage fields into a partially configured template at `examples/template-generation/data-on-eks.yaml`. This partially configure template contains fields that are common to all modules in the Data on EKS repository. - -You can view what fields are generated by using `diff` and `yq`: - -```bash -diff <(yq -P 'sort_keys(..)' -o=props examples/template-generation/spark-k8s-operator.yaml) <(yq -P 'sort_keys(..)' -o=props examples/template-generation/data-on-eks.yaml) -``` - -The rest of this document assumes you've used the spark-k8s-operator module to generate input fields. - -Once the template is hydrated, it is ready to use. You can push it to a private git repository within your organization or push it to a public repository. -You can then register the template in the Backstage UI (`Create...` > `Register Existing Component`) or update your Backstage configuration. The main configuration file is a ConfigMap in the backstage namespace with the name `backstage-config`. - -You can use the template to create a EKS cluster with optimization for data workload using terraform. The example uses a custom Argo Workflows template to run the install script, add a IAM role to the admin group in the cluster, create a Backstage service account, then registers the cluster to Backstage for use. - -The Argo Workflows run is associated with the Backstage entity. So you can check on its status within Backstage. If you'd like to see detailed information about the run, you can click on it to go to the Argo Workflows UI. - -Once the new cluster is ready, you can submit a Spark Job to the new cluster through Backstage. - -### Terraform (declarative) - -You can also use the terraform controller or Crossplane terraform provider to deploy terraform modules. Examples of this will be included in the future. - - -### Crossplane and ACK - -When using Crossplane, the bulk of work is done by Crossplane providers. You can use GitOps tools like ArgoCD to deploy these manifests to the cluster using Backstage scaffolder templates. Examples of this will be included in the future. - - diff --git a/docs/installation_with_idpbuilder.md b/docs/installation_with_idpbuilder.md index ea1499632..ef0d6126d 100644 --- a/docs/installation_with_idpbuilder.md +++ b/docs/installation_with_idpbuilder.md @@ -1,3 +1,4 @@ +# Installation using IDPBuilder ## Installation Flow Diagram This diagram illustrates the high-level installation flow for the CNOE AWS Reference Implementation using **`idpbuilder`**. It shows how the **`idpbuilder`** and local environment interact with AWS resources to deploy and configure the platform on an EKS cluster. diff --git a/docs/examples/spark/pi.yaml b/examples/spark/pi.yaml similarity index 99% rename from docs/examples/spark/pi.yaml rename to examples/spark/pi.yaml index 39b7202ae..448b01aa1 100644 --- a/docs/examples/spark/pi.yaml +++ b/examples/spark/pi.yaml @@ -42,4 +42,3 @@ spec: - key: spark-spot-ca operator: Exists effect: NoSchedule - diff --git a/docs/examples/template-generation/data-on-eks.yaml b/examples/template-generation/data-on-eks.yaml similarity index 100% rename from docs/examples/template-generation/data-on-eks.yaml rename to examples/template-generation/data-on-eks.yaml