AeThex-Labs/AeThex-Engine-Core
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2
AeThex-Engine-Core/docs/ARCHITECTURE_OVERVIEW.md

812 lines
26 KiB
Markdown
Raw Blame History

# AeThex Engine - Architecture Overview
Complete technical architecture of the AeThex Engine ecosystem.
---
## System Overview
AeThex is a complete game development platform consisting of three core components:
```
┌────────────────────────────────────────────────────────────────┐
│ AeThex Platform │
├────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ AeThex Engine │ │ Studio IDE │ │ Cloud Services │ │
│ │ │ │ │ │ │ │
│ │ • Game Runtime │ │ • Browser │ │ • Auth │ │
│ │ • Editor │ │ • Collab │ │ • Multiplayer │ │
│ │ • Cloud SDK │ │ • Live View │ │ • Saves │ │
│ │ • AI Module │ │ • Assets │ │ • Analytics │ │
│ └─────────────────┘ └──────────────┘ └──────────────────┘ │
│ ↕ ↕ ↕ │
│ C++/GDScript TypeScript/React Node.js/Go │
│ │
└────────────────────────────────────────────────────────────────┘
```
---
## 1. AeThex Engine (Core)
Based on Godot Engine 4.3-stable with AeThex-specific modules.
### Architecture Layers
```
┌────────────────────────────────────────────┐
│ Game Code (GDScript/C#) │ ← Developer's game
├────────────────────────────────────────────┤
│ AeThex Cloud SDK (GDScript) │ ← Cloud API
├────────────────────────────────────────────┤
│ AeThex Modules (C++) │ ← Cloud/AI/Studio
│ • aethex_cloud • aethex_ai │
│ • aethex_studio • aethex_analytics │
├────────────────────────────────────────────┤
│ Godot Core Engine (C++) │ ← Game engine
│ • Rendering • Physics • Audio │
│ • Networking • Scripting • Scene System │
├────────────────────────────────────────────┤
│ Platform Layer (OS-specific) │ ← Windows/Linux/Mac
└────────────────────────────────────────────┘
```
### Module Structure
#### Core Godot Modules (Unchanged)
Located in `engine/modules/`:
- `gltf/` - GLTF import/export
- `noise/` - Procedural noise
- `regex/` - Regular expressions
- `text_server_*` - Text rendering
- `websocket/` - WebSocket support
- `multiplayer/` - Base networking
#### AeThex Custom Modules
Located in `engine/modules/`:
**`aethex_cloud/`** - Cloud services integration
```
aethex_cloud/
├── register_types.{h,cpp} # Module registration
├── aethex_cloud.{h,cpp} # Main singleton
├── auth/ # Authentication
│ ├── auth_manager.{h,cpp}
│ └── session_manager.{h,cpp}
├── saves/ # Cloud saves
│ ├── cloud_save_manager.{h,cpp}
│ └── conflict_resolver.{h,cpp}
├── multiplayer/ # Multiplayer backend
│ ├── multiplayer_manager.{h,cpp}
│ ├── matchmaking.{h,cpp}
│ └── room_manager.{h,cpp}
├── analytics/ # Event tracking
│ ├── analytics_manager.{h,cpp}
│ └── crash_reporter.{h,cpp}
└── api/ # HTTP/WebSocket clients
├── http_client.{h,cpp}
└── websocket_client.{h,cpp}
```
**`aethex_ai/`** - AI assistant integration
```
aethex_ai/
├── register_types.{h,cpp}
├── aethex_ai.{h,cpp} # AI singleton
├── assistant/ # In-game assistant
│ ├── ai_assistant.{h,cpp}
│ └── code_generator.{h,cpp}
└── llm/ # LLM integration
├── openai_client.{h,cpp}
└── anthropic_client.{h,cpp}
```
**`aethex_studio/`** - Studio IDE bridge
```
aethex_studio/
├── register_types.{h,cpp}
├── studio_bridge.{h,cpp} # IDE communication
├── live_reload.{h,cpp} # Hot reload
├── remote_debug.{h,cpp} # Debug bridge
└── asset_sync.{h,cpp} # Asset hot-swapping
```
### Build System
**SCons-based build:**
```bash
# Build targets
scons platform=linuxbsd target=editor # Linux editor
scons platform=windows target=template_release # Windows export template
scons platform=web target=template_release # Web export
```
**Key build files:**
- `engine/SConstruct` - Main build script
- `engine/platform/*/detect.py` - Platform detection
- `engine/modules/*/config.py` - Module configuration
### Singleton Architecture
AeThex exposes functionality through global singletons:
```gdscript
# Automatically available in all scripts
AeThexCloud.connect_to_cloud()
AeThexAuth.login_email("user@example.com", "pass")
AeThexSaves.save_game("slot1", data)
AeThexMultiplayer.create_room("room-123")
AeThexAnalytics.track_event("level_complete", {})
AeThexAI.ask_assistant("How do I add jumping?")
AeThexStudio.reload_scene()
```
**Singleton registration:**
```cpp
// In register_types.cpp
Engine::get_singleton()->add_singleton(
Engine::Singleton("AeThexCloud", AeThexCloud::get_singleton())
);
```
---
## 2. Studio IDE
Web-based collaborative IDE for AeThex development.
### Technology Stack
```
Frontend: Backend:
• React 18 • Node.js + Express
• TypeScript • WebSocket (ws)
• Monaco Editor • PostgreSQL
• TailwindCSS • JWT auth
• Vite build • File system API
```
### Architecture
```
┌────────────────────────────────────────────────────────┐
│ Browser (Frontend) │
│ ┌──────────────┐ ┌──────────────┐ ┌─────────────┐ │
│ │ Code Editor │ │ Scene Tree │ │ Game View │ │
│ │ (Monaco) │ │ │ │ (iframe) │ │
│ └──────────────┘ └──────────────┘ └─────────────┘ │
│ ↓ ↓ ↓ │
│ ┌────────────────────────────────────────────────┐ │
│ │ WebSocket Manager │ │
│ └────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────┘
↓ WebSocket
┌────────────────────────────────────────────────────────┐
│ Studio Backend (Node.js) │
│ ┌────────────┐ ┌─────────────┐ ┌────────────────┐ │
│ │ File API │ │ Live Reload │ │ Collaboration │ │
│ └────────────┘ └─────────────┘ └────────────────┘ │
│ ┌────────────────────────────────────────────────┐ │
│ │ Project Manager (PostgreSQL) │ │
│ └────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────┘
↓ Engine Bridge
┌────────────────────────────────────────────────────────┐
│ AeThex Engine (Local Process) │
│ ┌────────────────────────────────────────────────┐ │
│ │ Studio Bridge Module (C++) │ │
│ └────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────┘
```
### Key Features
**1. Live Reload**
- File watcher monitors changes
- WebSocket pushes updates to engine
- Engine reloads modified resources
- No manual restart needed
**2. Collaboration**
- Real-time code editing (CRDT-based)
- Shared cursor positions
- Voice/text chat
- Project permissions
**3. Asset Management**
- Drag-and-drop uploads
- Asset browser
- Texture preview
- Model viewer
**4. Debug Tools**
- Breakpoints
- Variable inspection
- Console output
- Performance profiler
### File Structure
```
tools/studio/
├── frontend/ # React app
│ ├── src/
│ │ ├── components/ # UI components
│ │ │ ├── Editor/ # Code editor
│ │ │ ├── SceneTree/ # Scene hierarchy
│ │ │ └── GameView/ # Game preview
│ │ ├── hooks/ # React hooks
│ │ ├── services/ # API clients
│ │ │ ├── websocket.ts # WS connection
│ │ │ └── engine.ts # Engine bridge
│ │ └── App.tsx # Main app
│ └── package.json
├── backend/ # Node.js server
│ ├── src/
│ │ ├── routes/ # API routes
│ │ ├── services/ # Business logic
│ │ ├── websocket/ # WS handlers
│ │ └── index.js # Entry point
│ └── package.json
└── README.md
```
---
## 3. Cloud Services
Scalable backend services for multiplayer, saves, and analytics.
### Microservices Architecture
```
┌──────────────────┐
│ API Gateway │
│ (Load Balancer) │
└────────┬─────────┘
│
┌────────────────────┼────────────────────┐
↓ ↓ ↓
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Auth │ │ Saves │ │ Multiplayer │
│ Service │ │ Service │ │ Service │
│ │ │ │ │ │
│ Node.js │ │ Go/Rust │ │ C++/Rust │
│ Express │ │ S3 storage │ │ WebSocket │
└──────────────┘ └──────────────┘ └──────────────┘
↓ ↓ ↓
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ PostgreSQL │ │ Object │ │ Redis │
│ (Users) │ │ Storage │ │ (Sessions) │
└──────────────┘ └──────────────┘ └──────────────┘
```
### Service Responsibilities
**Auth Service (Node.js + PostgreSQL)**
- User registration/login
- JWT token generation
- OAuth providers (Google, GitHub, Discord)
- Session management
- Password reset
**Saves Service (Go + S3)**
- Cloud save storage
- Versioning
- Conflict resolution
- Compression
- Synchronization
**Multiplayer Service (C++/Rust + WebSocket)**
- Real-time relay
- Matchmaking
- Room management
- State synchronization
- Voice chat (WebRTC)
**Analytics Service (Python + ClickHouse)**
- Event ingestion
- Real-time analytics
- Crash reporting
- Query API
### Database Schema
**PostgreSQL (Relational):**
- `users` - User accounts
- `sessions` - Active sessions
- `oauth_connections` - OAuth links
- `projects` - Studio projects
- `collaborators` - Project permissions
**S3 (Object Storage):**
- `saves/{user_id}/{game_id}/{slot}.json.gz`
- `assets/{game_id}/{asset_id}.{ext}`
- `dlc/{game_id}/{version}/...`
**Redis (Cache):**
- `session:{token}` - Session cache
- `user:{id}` - User profile cache
- `leaderboard:{game_id}:{type}` - Leaderboard cache
- `room:{id}` - Active room state
**ClickHouse (Analytics):**
- `events` - Analytics events
- `sessions` - User sessions
- `crashes` - Crash reports
---
## Communication Protocols
### 1. Engine ↔ Cloud Services
**Protocol:** HTTPS + WebSocket
**Authentication:**
```
Authorization: Bearer <jwt-token>
```
**REST API (HTTPS):**
```
POST /api/v1/auth/login
GET /api/v1/saves/:user_id
POST /api/v1/analytics/event
GET /api/v1/leaderboard/:game_id
```
**WebSocket (Real-time):**
```javascript
// Client → Server
{
"type": "join_room",
"room_id": "ABCD-1234",
"player_data": {...}
}
// Server → Client
{
"type": "player_joined",
"player_id": "67890",
"player_data": {...}
}
```
### 2. Engine ↔ Studio IDE
**Protocol:** WebSocket (local network)
**Connection:** `ws://localhost:9003/bridge`
**Messages:**
```javascript
// Studio → Engine
{
"type": "reload_scene",
"scene_path": "res://main.tscn"
}
// Engine → Studio
{
"type": "scene_loaded",
"success": true,
"scene_path": "res://main.tscn"
}
// File change notification
{
"type": "file_changed",
"path": "res://player.gd",
"content": "extends CharacterBody2D\n..."
}
```
### 3. Studio Frontend ↔ Backend
**Protocol:** WebSocket + REST
**WebSocket for:**
- Live file updates
- Collaboration (cursors, edits)
- Real-time logs
- Game preview sync
**REST for:**
- File operations (create/delete)
- Project management
- User authentication
- Asset uploads
---
## Data Flow Examples
### Example 1: Player Login
```
┌─────────┐ ┌─────────┐ ┌──────────┐
│ Game │ │ Engine │ │ Cloud │
└────┬────┘ └────┬────┘ └─────┬────┘
│ │ │
│ login_email() │ │
├──────────────────>│ │
│ │ POST /api/v1/auth/ │
│ │ login │
│ ├───────────────────>│
│ │ │
│ │ { token, user } │
│ │<───────────────────┤
│ { success: true } │ │
│<──────────────────┤ │
│ │ │
```
### Example 2: Multiplayer Game
```
Player 1 Player 2 Cloud Server
│ │ │
│ create_room() │ │
├──────────────────────────────> │
│ │ Room: ABCD-1234 │
│<────────────────────────────────┤
│ │ │
│ │ join_room() │
│ ├────────────────>│
│ │ │
│ player_joined │ │
│<────────────────────────────────┤
│ │ │
│ send_state() │ │
├──────────────────────────────> │
│ │ player_state │
│ │<────────────────┤
│ │ │
```
### Example 3: Cloud Save Sync
```
Game Engine Cloud
│ │ │
│ save_game() │ │
├─────────────>│ │
│ │ Compress │
│ │ Checksum │
│ │ Encrypt │
│ │ │
│ │ POST /saves │
│ ├─────────────>│
│ │ │ [Store S3]
│ │ │ [Update DB]
│ │ │
│ │ {success} │
│ │<─────────────┤
│ {success} │ │
│<─────────────┤ │
│ │ │
```
---
## Deployment Architecture
### Development Environment
```
localhost:
├─ Engine (compiled binary)
├─ Studio Frontend :3000
├─ Studio Backend :9002
├─ Studio Bridge :9003
└─ Local PostgreSQL :5432
```
### Production Environment
```
┌─────────────────────────────────────────────┐
│ Cloudflare CDN │
│ • Static assets • DDoS protection │
└───────────────────┬─────────────────────────┘
↓
┌─────────────────────────────────────────────┐
│ Load Balancer (AWS ALB) │
└───────────────────┬─────────────────────────┘
↓
┌───────────────┬───────────────┬─────────────┐
│ Service 1 │ Service 2 │ Service N │
│ (k8s pod) │ (k8s pod) │ (k8s pod) │
└───────────────┴───────────────┴─────────────┘
↓ ↓ ↓
┌──────────────────────────────────────────────┐
│ Databases: PostgreSQL, Redis, ClickHouse │
└──────────────────────────────────────────────┘
```
**Infrastructure:**
- **Kubernetes** - Container orchestration
- **PostgreSQL** - Primary database
- **Redis** - Caching & sessions
- **S3/MinIO** - Object storage
- **ClickHouse** - Analytics
- **Cloudflare** - CDN & DDoS protection
---
## Security Architecture
### Authentication Flow
```
1. User enters credentials
2. Client hashes password (bcrypt)
3. Send to auth service over TLS
4. Server verifies hash
5. Generate JWT (15 min expiry)
6. Generate refresh token (30 days)
7. Return both tokens
8. Client stores securely
9. Use JWT for API calls
10. Refresh when expired
```
### Data Encryption
**In Transit:**
- TLS 1.3 for all HTTPS
- WSS (WebSocket Secure)
- Certificate pinning (optional)
**At Rest:**
- Database encryption (PostgreSQL)
- S3 server-side encryption
- Password hashing (bcrypt/argon2)
- Sensitive config in secrets manager
**End-to-End:**
- Voice chat (WebRTC DTLS/SRTP)
- Optional game data encryption
### Access Control
**JWT Claims:**
```json
{
"user_id": "12345",
"email": "user@example.com",
"roles": ["player", "developer"],
"permissions": ["read:saves", "write:saves"],
"iat": 1234567890,
"exp": 1234568790
}
```
**Permission Levels:**
- **Guest:** Limited features, no saves
- **Player:** Full game access, cloud saves
- **Developer:** Project management, analytics
- **Admin:** Full system access
---
## Performance Considerations
### Engine Performance
**Optimization strategies:**
- Multithreaded rendering
- Culling & LOD
- Physics optimization
- Script compilation (GDScript → bytecode)
- Resource streaming
**Target metrics:**
- 60 FPS on mid-range hardware
- <100ms input latency
- <500MB RAM for simple games
- <5s startup time
### Cloud Performance
**Target metrics:**
- <100ms API response time (p95)
- <50ms WebSocket latency
- 99.9% uptime SLA
- Support 100k concurrent users
**Scaling strategy:**
- Horizontal scaling (add more pods)
- Database read replicas
- CDN for static assets
- Redis caching
- Connection pooling
---
## Monitoring & Observability
### Engine Telemetry
**Collected metrics:**
- FPS & frame time
- Memory usage
- Draw calls
- Physics objects
- Network bandwidth
**Crash reporting:**
- Stack traces
- Engine version
- OS/hardware info
- Last user actions
### Cloud Monitoring
**Tools:**
- **Prometheus** - Metrics collection
- **Grafana** - Dashboards
- **Loki** - Log aggregation
- **Jaeger** - Distributed tracing
- **Sentry** - Error tracking
**Dashboards:**
- System health
- API performance
- User analytics
- Cost tracking
---
## Development Workflow
### Local Development
```bash
# 1. Build engine
cd engine
scons platform=linuxbsd target=editor -j8
# 2. Start Studio backend
cd tools/studio/backend
npm install && npm run dev
# 3. Start Studio frontend
cd tools/studio/frontend
npm install && npm run dev
# 4. Start cloud services (Docker Compose)
cd services
docker-compose up -d
# 5. Run engine
cd engine/bin
./aethex.linuxbsd.editor.x86_64
```
### CI/CD Pipeline
```yaml
GitHub Actions:
on: push
jobs:
test:
- Lint code
- Run unit tests
- Integration tests
build:
- Build engine (Linux/Windows/Mac)
- Build Studio
- Build Docker images
deploy:
- Push to container registry
- Deploy to Kubernetes
- Run smoke tests
```
---
## Future Architecture Plans
### Planned Features
**Engine:**
- Native AI inference (on-device LLMs)
- Advanced networking (client-side prediction)
- VR/AR support enhancements
- Mobile optimization
**Studio:**
- Visual scripting
- Asset store integration
- Team features (git, reviews)
- Cloud builds
**Cloud:**
- Global CDN expansion
- Edge computing (regional servers)
- Advanced analytics (ML insights)
- Blockchain integration (optional)
### Scalability Roadmap
**Phase 1:** 10k concurrent users
**Phase 2:** 100k concurrent users
**Phase 3:** 1M+ concurrent users
**Required:**
- Multi-region deployment
- Database sharding
- Microservices scaling
- Cost optimization
---
## Technical Decisions Log
### Why Godot as Base?
**Pros:**
- MIT licensed (fully open)
- Modern architecture
- Active community
- GDScript ease of use
- Good performance
**Cons:**
- Less mature than Unity/Unreal
- Smaller asset ecosystem
- Needs cloud integration
**Decision:** Build on Godot, add cloud features
### Why Microservices?
**Pros:**
- Independent scaling
- Language flexibility
- Fault isolation
- Easier maintenance
**Cons:**
- More complex deployment
- Network overhead
- Distributed debugging
**Decision:** Use microservices, manage complexity with K8s
### Why Web-based Studio?
**Pros:**
- No install required
- Cross-platform
- Real-time collaboration
- Easier updates
**Cons:**
- Requires internet
- Performance overhead
- File system limitations
**Decision:** Web-based with local engine bridge
---
## Additional Resources
- [API Reference](API_REFERENCE.md) - Complete API docs
- [Cloud Services Architecture](CLOUD_SERVICES_ARCHITECTURE.md) - Detailed cloud architecture
- [Studio Integration](STUDIO_INTEGRATION.md) - Studio IDE guide
- [Building Guide](../BUILDING_WINDOWS.md) - How to build from source
- [Contributing](../../engine/CONTRIBUTING.md) - Contribution guidelines
---
**Next:** Explore [Getting Started](GETTING_STARTED.md) to build your first AeThex game!
Reference in a new issue View git blame Copy permalink