Skip to content

Magic Eden Crossplane Configurations

This site documents how Magic Eden models and provisions AWS infrastructure using Crossplane. We package opinionated Compositions and XRDs (Composite Resource Definitions) into versioned Configuration packages that platform teams and application owners can install and consume in-cluster.

What is Crossplane at Magic Eden?

Crossplane lets us describe cloud resources as Kubernetes APIs. We define higher-level composite resources (XRs) like S3Bucket, DynamoDBTable, Cache, etc., and implement them using providers for AWS. Compositions encapsulate best practices, defaults, tagging, and guardrails so users declare intent while the platform handles the complicated bits.

Why Crossplane?

Crossplane allows platform teams to expose safe, high‑level APIs (XRs) for common infrastructure while keeping cloud‑specific details and guardrails in one place.

Key benefits

  • Standard, audited defaults and tags across accounts and regions
  • Namespaced, application‑scoped resources with environment‑aware settings
  • Reusable packages with semantic versions and automated releases

Repository structure

At a glance:

.
├─ compositions/
│  ├─ <composition>/
│  │  ├─ apis/                 # XRD (xrd.yaml) and Composition (composition.yaml)
│  │  ├─ examples/             # Example XR manifests
│  │  ├─ kcl/                  # KCL used by the function‑kcl pipeline for rendering
│  │  ├─ scripts/              # Helper scripts (e.g., render)
│  │  ├─ tests/                # Local render scaffolding and fixtures
│  │  └─ crossplane.yaml       # Configuration package manifest
├─ configurations/             # Top‑level Configuration package descriptors
├─ k8s/configuration-repos/    # ECR repos where packages are pushed (used by CI)
├─ .github/workflows/          # Release pipelines and shared reusable workflow
└─ docs/                       # This documentation site (MkDocs)

Available compositions

Name Description Version
auroradb-observer Observe existing Aurora PostgreSQL clusters auroradb-observer
dynamodb Manage DynamoDB tables with sensible defaults, TTL, streams, and autoscaling dynamodb
ecr-repository ECR repositories with optional KMS encryption, policies, lifecycle rules ecr-repository
external-secrets IAM and Secrets Manager setup for External Secrets Operator (ESO) external-secrets
kms-key Manage AWS KMS keys and aliases with rotation and policy kms-key
pod-identity EKS Pod Identities for Kubernetes ServiceAccounts pod-identity
s3-bucket S3 buckets (general/directory), policies, versioning, encryption s3-bucket
cache ElastiCache ReplicationGroups (Valkey by default, Redis supported) cache

Each composition page includes examples and links to the XRD and Composition. To preview rendered managed resources locally, use the render.sh script found in each composition’s scripts directory.

Using the packages

1) Ensure Crossplane and required functions/providers are installed in your cluster. 2) Install a package by referencing its OCI image from ECR (our CI publishes new versions automatically). Example package descriptors live in configurations/. 3) Apply the composition’s XRD (installed with the package), then create an XR in your namespace using the example under compositions/<name>/examples/xr.yaml. 4) Provide environment defaults via EnvironmentConfig (region, tags, subnet group names, etc.). Our Compositions read these using function‑environment‑configs.

EnvironmentConfig

EnvironmentConfig lets you centralize common settings like region, commonTags, or VPC‑related names. The function‑environment‑configs step injects these values during composition rendering.

Local development and testing

  • Render locally with crossplane render:
    • Use compositions/<name>/scripts/render.sh which wires in tests/functions.yaml and tests/extra-resources.yaml (EnvironmentConfig).
  • Update KCL or Composition YAML and re‑render to validate outputs before opening a PR.

Releases and versioning

  • Each composition is released independently via GitHub Actions using a reusable workflow. Images are pushed to our Amazon ECR under crossplane/<composition>.
  • Versions follow semver and are automatically computed from existing ECR tags.
  • After a release‑all run, README badges are updated to reflect the latest versions.

Conventions and guardrails

  • Namespaced managed resources where supported (aws.m.upbound.io groups)
  • Environment‑driven defaults with user overrides
  • managementPolicies support where applicable
  • CEL validations in XRDs to fail fast on invalid combinations

Where to start

  • Browse compositions/ to see available XRs and examples
  • Check configurations/ for installable package manifests
  • Use examples/xr.yaml in each composition to create your first XR

If you have questions or suggestions, open an issue or PR in github.com/magiceden/crossplane.