AeThex-Engine-Core/docs/ARCHITECTURE_OVERVIEW.md

26 KiB

AeThex Engine - Architecture Overview

Complete technical architecture of the AeThex Engine ecosystem.


System Overview

AeThex is a complete game development platform consisting of three core components:

┌────────────────────────────────────────────────────────────────┐
│                    AeThex Platform                              │
├────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌─────────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │  AeThex Engine  │  │  Studio IDE  │  │ Cloud Services   │  │
│  │                 │  │              │  │                  │  │
│  │  • Game Runtime │  │  • Browser   │  │  • Auth          │  │
│  │  • Editor       │  │  • Collab    │  │  • Multiplayer   │  │
│  │  • Cloud SDK    │  │  • Live View │  │  • Saves         │  │
│  │  • AI Module    │  │  • Assets    │  │  • Analytics     │  │
│  └─────────────────┘  └──────────────┘  └──────────────────┘  │
│          ↕                   ↕                    ↕            │
│       C++/GDScript      TypeScript/React       Node.js/Go      │
│                                                                 │
└────────────────────────────────────────────────────────────────┘

1. AeThex Engine (Core)

Based on Godot Engine 4.3-stable with AeThex-specific modules.

Architecture Layers

┌────────────────────────────────────────────┐
│         Game Code (GDScript/C#)            │  ← Developer's game
├────────────────────────────────────────────┤
│      AeThex Cloud SDK (GDScript)           │  ← Cloud API
├────────────────────────────────────────────┤
│     AeThex Modules (C++)                   │  ← Cloud/AI/Studio
│  • aethex_cloud  • aethex_ai               │
│  • aethex_studio • aethex_analytics        │
├────────────────────────────────────────────┤
│   Godot Core Engine (C++)                  │  ← Game engine
│  • Rendering  • Physics  • Audio           │
│  • Networking • Scripting • Scene System   │
├────────────────────────────────────────────┤
│     Platform Layer (OS-specific)           │  ← Windows/Linux/Mac
└────────────────────────────────────────────┘

Module Structure

Core Godot Modules (Unchanged)

Located in engine/modules/:

  • gltf/ - GLTF import/export
  • noise/ - Procedural noise
  • regex/ - Regular expressions
  • text_server_* - Text rendering
  • websocket/ - WebSocket support
  • multiplayer/ - Base networking

AeThex Custom Modules

Located in engine/modules/:

aethex_cloud/ - Cloud services integration

aethex_cloud/
├── register_types.{h,cpp}     # Module registration
├── aethex_cloud.{h,cpp}       # Main singleton
├── auth/                      # Authentication
│   ├── auth_manager.{h,cpp}
│   └── session_manager.{h,cpp}
├── saves/                     # Cloud saves
│   ├── cloud_save_manager.{h,cpp}
│   └── conflict_resolver.{h,cpp}
├── multiplayer/               # Multiplayer backend
│   ├── multiplayer_manager.{h,cpp}
│   ├── matchmaking.{h,cpp}
│   └── room_manager.{h,cpp}
├── analytics/                 # Event tracking
│   ├── analytics_manager.{h,cpp}
│   └── crash_reporter.{h,cpp}
└── api/                       # HTTP/WebSocket clients
    ├── http_client.{h,cpp}
    └── websocket_client.{h,cpp}

aethex_ai/ - AI assistant integration

aethex_ai/
├── register_types.{h,cpp}
├── aethex_ai.{h,cpp}          # AI singleton
├── assistant/                 # In-game assistant
│   ├── ai_assistant.{h,cpp}
│   └── code_generator.{h,cpp}
└── llm/                       # LLM integration
    ├── openai_client.{h,cpp}
    └── anthropic_client.{h,cpp}

aethex_studio/ - Studio IDE bridge

aethex_studio/
├── register_types.{h,cpp}
├── studio_bridge.{h,cpp}      # IDE communication
├── live_reload.{h,cpp}        # Hot reload
├── remote_debug.{h,cpp}       # Debug bridge
└── asset_sync.{h,cpp}         # Asset hot-swapping

Build System

SCons-based build:

# Build targets
scons platform=linuxbsd target=editor      # Linux editor
scons platform=windows target=template_release  # Windows export template
scons platform=web target=template_release      # Web export

Key build files:

  • engine/SConstruct - Main build script
  • engine/platform/*/detect.py - Platform detection
  • engine/modules/*/config.py - Module configuration

Singleton Architecture

AeThex exposes functionality through global singletons:

# Automatically available in all scripts
AeThexCloud.connect_to_cloud()
AeThexAuth.login_email("user@example.com", "pass")
AeThexSaves.save_game("slot1", data)
AeThexMultiplayer.create_room("room-123")
AeThexAnalytics.track_event("level_complete", {})
AeThexAI.ask_assistant("How do I add jumping?")
AeThexStudio.reload_scene()

Singleton registration:

// In register_types.cpp
Engine::get_singleton()->add_singleton(
    Engine::Singleton("AeThexCloud", AeThexCloud::get_singleton())
);

2. Studio IDE

Web-based collaborative IDE for AeThex development.

Technology Stack

Frontend:              Backend:
• React 18            • Node.js + Express
• TypeScript          • WebSocket (ws)
• Monaco Editor       • PostgreSQL
• TailwindCSS         • JWT auth
• Vite build          • File system API

Architecture

┌────────────────────────────────────────────────────────┐
│                    Browser (Frontend)                   │
│  ┌──────────────┐  ┌──────────────┐  ┌─────────────┐  │
│  │ Code Editor  │  │ Scene Tree   │  │ Game View   │  │
│  │ (Monaco)     │  │              │  │ (iframe)    │  │
│  └──────────────┘  └──────────────┘  └─────────────┘  │
│         ↓                  ↓                 ↓          │
│  ┌────────────────────────────────────────────────┐   │
│  │         WebSocket Manager                      │   │
│  └────────────────────────────────────────────────┘   │
└────────────────────────────────────────────────────────┘
                        ↓ WebSocket
┌────────────────────────────────────────────────────────┐
│                  Studio Backend (Node.js)              │
│  ┌────────────┐  ┌─────────────┐  ┌────────────────┐  │
│  │ File API   │  │ Live Reload │  │ Collaboration  │  │
│  └────────────┘  └─────────────┘  └────────────────┘  │
│  ┌────────────────────────────────────────────────┐   │
│  │      Project Manager (PostgreSQL)              │   │
│  └────────────────────────────────────────────────┘   │
└────────────────────────────────────────────────────────┘
                        ↓ Engine Bridge
┌────────────────────────────────────────────────────────┐
│            AeThex Engine (Local Process)               │
│  ┌────────────────────────────────────────────────┐   │
│  │    Studio Bridge Module (C++)                  │   │
│  └────────────────────────────────────────────────┘   │
└────────────────────────────────────────────────────────┘

Key Features

1. Live Reload

  • File watcher monitors changes
  • WebSocket pushes updates to engine
  • Engine reloads modified resources
  • No manual restart needed

2. Collaboration

  • Real-time code editing (CRDT-based)
  • Shared cursor positions
  • Voice/text chat
  • Project permissions

3. Asset Management

  • Drag-and-drop uploads
  • Asset browser
  • Texture preview
  • Model viewer

4. Debug Tools

  • Breakpoints
  • Variable inspection
  • Console output
  • Performance profiler

File Structure

tools/studio/
├── frontend/                  # React app
│   ├── src/
│   │   ├── components/       # UI components
│   │   │   ├── Editor/       # Code editor
│   │   │   ├── SceneTree/    # Scene hierarchy
│   │   │   └── GameView/     # Game preview
│   │   ├── hooks/            # React hooks
│   │   ├── services/         # API clients
│   │   │   ├── websocket.ts  # WS connection
│   │   │   └── engine.ts     # Engine bridge
│   │   └── App.tsx           # Main app
│   └── package.json
├── backend/                   # Node.js server
│   ├── src/
│   │   ├── routes/           # API routes
│   │   ├── services/         # Business logic
│   │   ├── websocket/        # WS handlers
│   │   └── index.js          # Entry point
│   └── package.json
└── README.md

3. Cloud Services

Scalable backend services for multiplayer, saves, and analytics.

Microservices Architecture

                    ┌──────────────────┐
                    │  API Gateway     │
                    │  (Load Balancer) │
                    └────────┬─────────┘
                             │
        ┌────────────────────┼────────────────────┐
        ↓                    ↓                    ↓
┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│  Auth        │    │  Saves       │    │ Multiplayer  │
│  Service     │    │  Service     │    │  Service     │
│              │    │              │    │              │
│  Node.js     │    │  Go/Rust     │    │  C++/Rust    │
│  Express     │    │  S3 storage  │    │  WebSocket   │
└──────────────┘    └──────────────┘    └──────────────┘
        ↓                    ↓                    ↓
┌──────────────┐    ┌──────────────┐    ┌──────────────┐
│ PostgreSQL   │    │  Object      │    │  Redis       │
│ (Users)      │    │  Storage     │    │  (Sessions)  │
└──────────────┘    └──────────────┘    └──────────────┘

Service Responsibilities

Auth Service (Node.js + PostgreSQL)

  • User registration/login
  • JWT token generation
  • OAuth providers (Google, GitHub, Discord)
  • Session management
  • Password reset

Saves Service (Go + S3)

  • Cloud save storage
  • Versioning
  • Conflict resolution
  • Compression
  • Synchronization

Multiplayer Service (C++/Rust + WebSocket)

  • Real-time relay
  • Matchmaking
  • Room management
  • State synchronization
  • Voice chat (WebRTC)

Analytics Service (Python + ClickHouse)

  • Event ingestion
  • Real-time analytics
  • Crash reporting
  • Query API

Database Schema

PostgreSQL (Relational):

  • users - User accounts
  • sessions - Active sessions
  • oauth_connections - OAuth links
  • projects - Studio projects
  • collaborators - Project permissions

S3 (Object Storage):

  • saves/{user_id}/{game_id}/{slot}.json.gz
  • assets/{game_id}/{asset_id}.{ext}
  • dlc/{game_id}/{version}/...

Redis (Cache):

  • session:{token} - Session cache
  • user:{id} - User profile cache
  • leaderboard:{game_id}:{type} - Leaderboard cache
  • room:{id} - Active room state

ClickHouse (Analytics):

  • events - Analytics events
  • sessions - User sessions
  • crashes - Crash reports

Communication Protocols

1. Engine ↔ Cloud Services

Protocol: HTTPS + WebSocket

Authentication:

Authorization: Bearer <jwt-token>

REST API (HTTPS):

POST   /api/v1/auth/login
GET    /api/v1/saves/:user_id
POST   /api/v1/analytics/event
GET    /api/v1/leaderboard/:game_id

WebSocket (Real-time):

// Client → Server
{
  "type": "join_room",
  "room_id": "ABCD-1234",
  "player_data": {...}
}

// Server → Client
{
  "type": "player_joined",
  "player_id": "67890",
  "player_data": {...}
}

2. Engine ↔ Studio IDE

Protocol: WebSocket (local network)

Connection: ws://localhost:9003/bridge

Messages:

// Studio → Engine
{
  "type": "reload_scene",
  "scene_path": "res://main.tscn"
}

// Engine → Studio
{
  "type": "scene_loaded",
  "success": true,
  "scene_path": "res://main.tscn"
}

// File change notification
{
  "type": "file_changed",
  "path": "res://player.gd",
  "content": "extends CharacterBody2D\n..."
}

3. Studio Frontend ↔ Backend

Protocol: WebSocket + REST

WebSocket for:

  • Live file updates
  • Collaboration (cursors, edits)
  • Real-time logs
  • Game preview sync

REST for:

  • File operations (create/delete)
  • Project management
  • User authentication
  • Asset uploads

Data Flow Examples

Example 1: Player Login

┌─────────┐         ┌─────────┐         ┌──────────┐
│  Game   │         │ Engine  │         │  Cloud   │
└────┬────┘         └────┬────┘         └─────┬────┘
     │                   │                    │
     │ login_email()     │                    │
     ├──────────────────>│                    │
     │                   │ POST /api/v1/auth/ │
     │                   │  login             │
     │                   ├───────────────────>│
     │                   │                    │
     │                   │  { token, user }   │
     │                   │<───────────────────┤
     │ { success: true } │                    │
     │<──────────────────┤                    │
     │                   │                    │

Example 2: Multiplayer Game

 Player 1       Player 2        Cloud Server
    │               │                 │
    │ create_room() │                 │
    ├──────────────────────────────>  │
    │               │ Room: ABCD-1234 │
    │<────────────────────────────────┤
    │               │                 │
    │               │ join_room()     │
    │               ├────────────────>│
    │               │                 │
    │ player_joined │                 │
    │<────────────────────────────────┤
    │               │                 │
    │ send_state()  │                 │
    ├──────────────────────────────>  │
    │               │ player_state    │
    │               │<────────────────┤
    │               │                 │

Example 3: Cloud Save Sync

  Game          Engine         Cloud
    │              │              │
    │ save_game()  │              │
    ├─────────────>│              │
    │              │ Compress     │
    │              │ Checksum     │
    │              │ Encrypt      │
    │              │              │
    │              │ POST /saves  │
    │              ├─────────────>│
    │              │              │ [Store S3]
    │              │              │ [Update DB]
    │              │              │
    │              │ {success}    │
    │              │<─────────────┤
    │ {success}    │              │
    │<─────────────┤              │
    │              │              │

Deployment Architecture

Development Environment

localhost:
  ├─ Engine (compiled binary)
  ├─ Studio Frontend :3000
  ├─ Studio Backend :9002
  ├─ Studio Bridge :9003
  └─ Local PostgreSQL :5432

Production Environment

┌─────────────────────────────────────────────┐
│              Cloudflare CDN                 │
│  • Static assets • DDoS protection          │
└───────────────────┬─────────────────────────┘
                    ↓
┌─────────────────────────────────────────────┐
│          Load Balancer (AWS ALB)            │
└───────────────────┬─────────────────────────┘
                    ↓
┌───────────────┬───────────────┬─────────────┐
│  Service 1    │  Service 2    │  Service N  │
│  (k8s pod)    │  (k8s pod)    │  (k8s pod)  │
└───────────────┴───────────────┴─────────────┘
        ↓               ↓               ↓
┌──────────────────────────────────────────────┐
│  Databases: PostgreSQL, Redis, ClickHouse    │
└──────────────────────────────────────────────┘

Infrastructure:

  • Kubernetes - Container orchestration
  • PostgreSQL - Primary database
  • Redis - Caching & sessions
  • S3/MinIO - Object storage
  • ClickHouse - Analytics
  • Cloudflare - CDN & DDoS protection

Security Architecture

Authentication Flow

1. User enters credentials
2. Client hashes password (bcrypt)
3. Send to auth service over TLS
4. Server verifies hash
5. Generate JWT (15 min expiry)
6. Generate refresh token (30 days)
7. Return both tokens
8. Client stores securely
9. Use JWT for API calls
10. Refresh when expired

Data Encryption

In Transit:

  • TLS 1.3 for all HTTPS
  • WSS (WebSocket Secure)
  • Certificate pinning (optional)

At Rest:

  • Database encryption (PostgreSQL)
  • S3 server-side encryption
  • Password hashing (bcrypt/argon2)
  • Sensitive config in secrets manager

End-to-End:

  • Voice chat (WebRTC DTLS/SRTP)
  • Optional game data encryption

Access Control

JWT Claims:

{
  "user_id": "12345",
  "email": "user@example.com",
  "roles": ["player", "developer"],
  "permissions": ["read:saves", "write:saves"],
  "iat": 1234567890,
  "exp": 1234568790
}

Permission Levels:

  • Guest: Limited features, no saves
  • Player: Full game access, cloud saves
  • Developer: Project management, analytics
  • Admin: Full system access

Performance Considerations

Engine Performance

Optimization strategies:

  • Multithreaded rendering
  • Culling & LOD
  • Physics optimization
  • Script compilation (GDScript → bytecode)
  • Resource streaming

Target metrics:

  • 60 FPS on mid-range hardware
  • <100ms input latency
  • <500MB RAM for simple games
  • <5s startup time

Cloud Performance

Target metrics:

  • <100ms API response time (p95)
  • <50ms WebSocket latency
  • 99.9% uptime SLA
  • Support 100k concurrent users

Scaling strategy:

  • Horizontal scaling (add more pods)
  • Database read replicas
  • CDN for static assets
  • Redis caching
  • Connection pooling

Monitoring & Observability

Engine Telemetry

Collected metrics:

  • FPS & frame time
  • Memory usage
  • Draw calls
  • Physics objects
  • Network bandwidth

Crash reporting:

  • Stack traces
  • Engine version
  • OS/hardware info
  • Last user actions

Cloud Monitoring

Tools:

  • Prometheus - Metrics collection
  • Grafana - Dashboards
  • Loki - Log aggregation
  • Jaeger - Distributed tracing
  • Sentry - Error tracking

Dashboards:

  • System health
  • API performance
  • User analytics
  • Cost tracking

Development Workflow

Local Development

# 1. Build engine
cd engine
scons platform=linuxbsd target=editor -j8

# 2. Start Studio backend
cd tools/studio/backend
npm install && npm run dev

# 3. Start Studio frontend
cd tools/studio/frontend
npm install && npm run dev

# 4. Start cloud services (Docker Compose)
cd services
docker-compose up -d

# 5. Run engine
cd engine/bin
./aethex.linuxbsd.editor.x86_64

CI/CD Pipeline

GitHub Actions:
  
  on: push
  
  jobs:
    test:
      - Lint code
      - Run unit tests
      - Integration tests
    
    build:
      - Build engine (Linux/Windows/Mac)
      - Build Studio
      - Build Docker images
    
    deploy:
      - Push to container registry
      - Deploy to Kubernetes
      - Run smoke tests

Future Architecture Plans

Planned Features

Engine:

  • Native AI inference (on-device LLMs)
  • Advanced networking (client-side prediction)
  • VR/AR support enhancements
  • Mobile optimization

Studio:

  • Visual scripting
  • Asset store integration
  • Team features (git, reviews)
  • Cloud builds

Cloud:

  • Global CDN expansion
  • Edge computing (regional servers)
  • Advanced analytics (ML insights)
  • Blockchain integration (optional)

Scalability Roadmap

Phase 1: 10k concurrent users Phase 2: 100k concurrent users Phase 3: 1M+ concurrent users

Required:

  • Multi-region deployment
  • Database sharding
  • Microservices scaling
  • Cost optimization

Technical Decisions Log

Why Godot as Base?

Pros:

  • MIT licensed (fully open)
  • Modern architecture
  • Active community
  • GDScript ease of use
  • Good performance

Cons:

  • Less mature than Unity/Unreal
  • Smaller asset ecosystem
  • Needs cloud integration

Decision: Build on Godot, add cloud features

Why Microservices?

Pros:

  • Independent scaling
  • Language flexibility
  • Fault isolation
  • Easier maintenance

Cons:

  • More complex deployment
  • Network overhead
  • Distributed debugging

Decision: Use microservices, manage complexity with K8s

Why Web-based Studio?

Pros:

  • No install required
  • Cross-platform
  • Real-time collaboration
  • Easier updates

Cons:

  • Requires internet
  • Performance overhead
  • File system limitations

Decision: Web-based with local engine bridge


Additional Resources


Next: Explore Getting Started to build your first AeThex game!