AeThex-Labs/AeThex-Engine-Core
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2
AeThex-Engine-Core/docs/CLOUD_SERVICES_ARCHITECTURE.md

830 lines
19 KiB
Markdown
Raw Blame History

# AeThex Engine - Cloud Services Architecture
## Vision
Cloud-first game engine infrastructure providing authentication, multiplayer backend,
cloud saves, analytics, and asset delivery - making online games trivial to build.
---
## Core Services
### 1. Authentication & User Management
- Email/password, OAuth (Google, GitHub, Discord)
- User profiles and preferences
- Team/organization management
- Permission systems
### 2. Cloud Saves & Sync
- Automatic save game cloud sync
- Cross-platform save compatibility
- Conflict resolution
- Version history
### 3. Multiplayer Backend
- Matchmaking service
- Real-time relay servers
- P2P connection facilitation
- Voice/text chat infrastructure
### 4. Analytics & Telemetry
- Player behavior tracking
- Performance metrics
- Crash reporting
- A/B testing framework
### 5. Asset Delivery Network (CDN)
- DLC/update distribution
- Asset streaming
- Version management
- Bandwidth optimization
### 6. Leaderboards & Achievements
- Global/friend leaderboards
- Achievement system
- Player statistics
- Social features
---
## System Architecture
### High-Level Overview
```
┌─────────────────────────────────────────────────────┐
│ AeThex Game (Client) │
│ ┌──────────────────────────────────────────────┐ │
│ │ AeThex Cloud SDK (GDScript/C++) │ │
│ │ • Auth • Saves • Multiplayer • Analytics │ │
│ └──────────────────────────────────────────────┘ │
│ ↓ HTTPS/WebSocket ↓ │
└─────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────┐
│ API Gateway (Load Balancer) │
│ • Rate Limiting • Authentication │
│ • Request Routing • SSL Termination │
└─────────────────────────────────────────────────────┘
↓
┌──────────────┬──────────────┬──────────────┬─────────┐
│ Auth │ Saves │ Multiplayer │Analytics│
│ Service │ Service │ Service │ Service │
│ │ │ │ │
│ Node.js │ Go/Rust │ C++/Rust │ Python │
│ PostgreSQL │ S3/Object │ WebSocket │ ClickH.│
└──────────────┴──────────────┴──────────────┴─────────┘
```
---
## Module Structure (Engine Side)
```
engine/modules/aethex_cloud/
├── SCsub
├── config.py
├── register_types.h/cpp
├── aethex_cloud.h/cpp # Main singleton
├── auth/
│ ├── auth_manager.h/cpp # Authentication
│ ├── user_profile.h/cpp # User data
│ └── session_manager.h/cpp # Session handling
├── saves/
│ ├── cloud_save_manager.h/cpp # Save sync
│ ├── save_conflict_resolver.h/cpp
│ └── save_metadata.h/cpp
├── multiplayer/
│ ├── matchmaking.h/cpp # Matchmaking
│ ├── relay_client.h/cpp # Relay connection
│ ├── voice_chat.h/cpp # Voice integration
│ └── room_manager.h/cpp # Room/lobby system
├── analytics/
│ ├── analytics_manager.h/cpp # Event tracking
│ ├── crash_reporter.h/cpp # Crash reports
│ └── performance_tracker.h/cpp # Performance
├── cdn/
│ ├── asset_downloader.h/cpp # Asset loading
│ ├── dlc_manager.h/cpp # DLC handling
│ └── update_checker.h/cpp # Updates
├── social/
│ ├── leaderboard.h/cpp # Leaderboards
│ ├── achievements.h/cpp # Achievements
│ └── friends_list.h/cpp # Social features
├── api/
│ ├── http_client.h/cpp # HTTP requests
│ ├── websocket_client.h/cpp # WebSocket
│ └── api_request.h/cpp # Request builder
└── ui/
├── auth_dialog.h/cpp # Login UI
├── cloud_panel.h/cpp # Cloud dashboard
└── analytics_viewer.h/cpp # Analytics view
```
---
## Backend Services
### Tech Stack Recommendation
#### Authentication Service
**Technology:** Node.js + Express + PostgreSQL
**Responsibilities:**
- User registration/login
- JWT token generation
- OAuth integration
- Password reset
- Email verification
**API Endpoints:**
```
POST /api/v1/auth/register
POST /api/v1/auth/login
POST /api/v1/auth/refresh
POST /api/v1/auth/logout
GET /api/v1/auth/profile
PUT /api/v1/auth/profile
POST /api/v1/auth/oauth/google
```
#### Cloud Save Service
**Technology:** Go/Rust + S3/MinIO
**Responsibilities:**
- Store save game data
- Version management
- Conflict resolution
- Compression
**API Endpoints:**
```
GET /api/v1/saves/:user_id
POST /api/v1/saves/:user_id
GET /api/v1/saves/:user_id/:save_id
DELETE /api/v1/saves/:user_id/:save_id
GET /api/v1/saves/:user_id/history
```
#### Multiplayer Service
**Technology:** C++/Rust + WebSocket/UDP
**Responsibilities:**
- Real-time relay
- Matchmaking
- Room management
- Voice chat (WebRTC)
**WebSocket Events:**
```
// Client → Server
join_room
leave_room
send_message
player_state_update
// Server → Client
room_joined
room_left
player_joined
player_left
state_update
```
#### Analytics Service
**Technology:** Python/Go + ClickHouse/TimescaleDB
**Responsibilities:**
- Event ingestion
- Real-time analytics
- Crash report storage
- Query API
---
## Client SDK (GDScript API)
### Authentication
```gdscript
# Singleton: AeThexCloud
# Login
var result = await AeThexCloud.auth.login_email("user@example.com", "password")
if result.success:
print("Logged in as: ", result.user.username)
# OAuth Login
AeThexCloud.auth.login_oauth("google")
# Get current user
var user = AeThexCloud.auth.get_current_user()
print(user.email, user.username, user.avatar_url)
# Logout
AeThexCloud.auth.logout()
```
### Cloud Saves
```gdscript
# Save game to cloud
var save_data = {
"level": 5,
"health": 100,
"inventory": ["sword", "shield"],
"position": Vector3(10, 0, 20)
}
var result = await AeThexCloud.saves.save("slot1", save_data)
if result.success:
print("Game saved to cloud")
# Load from cloud
var loaded = await AeThexCloud.saves.load("slot1")
if loaded.success:
var data = loaded.data
player.level = data.level
player.health = data.health
# List all saves
var saves_list = await AeThexCloud.saves.list()
for save in saves_list:
print(save.name, save.timestamp, save.size)
# Auto-sync (background)
AeThexCloud.saves.enable_auto_sync(true)
```
### Multiplayer
```gdscript
# Join matchmaking
AeThexCloud.multiplayer.matchmaking.join_queue({
"mode": "deathmatch",
"region": "us-east",
"skill_range": [1000, 1500]
})
# Handle match found
AeThexCloud.multiplayer.matchmaking.match_found.connect(func(match_info):
print("Match found! Joining room: ", match_info.room_id)
AeThexCloud.multiplayer.join_room(match_info.room_id)
)
# Send player state
AeThexCloud.multiplayer.send_state({
"position": player.position,
"rotation": player.rotation,
"health": player.health
})
# Receive other players' state
AeThexCloud.multiplayer.player_state_received.connect(func(player_id, state):
var other_player = get_node("Player_" + str(player_id))
other_player.position = state.position
other_player.rotation = state.rotation
)
# Voice chat
AeThexCloud.multiplayer.voice.enable(true)
AeThexCloud.multiplayer.voice.mute_player(player_id, true)
```
### Analytics
```gdscript
# Track events
AeThexCloud.analytics.track_event("level_completed", {
"level": 5,
"time": 120.5,
"deaths": 3
})
# Track custom metrics
AeThexCloud.analytics.track_metric("fps", Engine.get_frames_per_second())
# Track screen view
AeThexCloud.analytics.track_screen("main_menu")
# User properties
AeThexCloud.analytics.set_user_property("vip_status", true)
# Crash reporting (automatic)
AeThexCloud.analytics.enable_crash_reporting(true)
```
### Leaderboards
```gdscript
# Submit score
AeThexCloud.social.leaderboards.submit_score("high_scores", 1000)
# Get leaderboard
var leaderboard = await AeThexCloud.social.leaderboards.get("high_scores", {
"range": "global", # or "friends"
"limit": 10
})
for entry in leaderboard:
print(entry.rank, entry.username, entry.score)
```
### Achievements
```gdscript
# Unlock achievement
AeThexCloud.social.achievements.unlock("first_kill")
# Track progress
AeThexCloud.social.achievements.set_progress("kill_100_enemies", 45)
# Get all achievements
var achievements = await AeThexCloud.social.achievements.get_all()
for ach in achievements:
print(ach.name, ach.unlocked, ach.progress)
```
---
## Security & Privacy
### Authentication
- JWT tokens with short expiry (15 min)
- Refresh tokens (30 days)
- Secure password hashing (bcrypt/argon2)
- Rate limiting on auth endpoints
- CAPTCHA for registration
### Data Encryption
- TLS 1.3 for all communications
- At-rest encryption for save data
- End-to-end encryption for voice chat
- Encrypted credentials storage
### Privacy
- GDPR compliant
- COPPA compliant (for kids' games)
- Opt-in analytics only
- Data deletion on request
- Privacy policy integration
---
## Pricing Model
### Free Tier
- 5,000 monthly active users
- 1 GB cloud saves per user
- 100 GB CDN bandwidth
- Basic analytics
- Community support
### Pro Tier ($99/month)
- 50,000 MAU
- 5 GB cloud saves per user
- 1 TB CDN bandwidth
- Advanced analytics
- Priority support
- Custom branding
### Enterprise (Custom)
- Unlimited MAU
- Custom storage
- Dedicated servers
- 24/7 support
- SLA guarantees
- White-label options
---
## Implementation Phases
### Phase 1: Foundation (Month 1-2)
- [ ] Basic auth service
- [ ] Simple cloud saves
- [ ] API client in engine
- [ ] UI dialogs
### Phase 2: Multiplayer (Month 3-4)
- [ ] Matchmaking service
- [ ] Relay servers
- [ ] Room management
- [ ] Basic voice chat
### Phase 3: Analytics (Month 5-6)
- [ ] Event tracking
- [ ] Crash reporting
- [ ] Dashboard UI
- [ ] Query system
### Phase 4: Social (Month 7-8)
- [ ] Leaderboards
- [ ] Achievements
- [ ] Friends system
- [ ] Social sharing
### Phase 5: CDN & DLC (Month 9-12)
- [ ] Asset delivery
- [ ] DLC management
- [ ] Update system
- [ ] Version control
---
## Deployment
### Infrastructure (Kubernetes)
```yaml
# Simplified deployment structure
apiVersion: apps/v1
kind: Deployment
metadata:
name: aethex-auth-service
spec:
replicas: 3
selector:
matchLabels:
app: aethex-auth
template:
metadata:
labels:
app: aethex-auth
spec:
containers:
- name: auth
image: aethex/auth-service:latest
ports:
- containerPort: 8080
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: db-credentials
key: url
```
### Monitoring
- Prometheus + Grafana
- Error tracking (Sentry)
- Log aggregation (ELK/Loki)
- Uptime monitoring
- Cost tracking
---
## Migration Path
### For Developers
1. Install AeThex Cloud module
2. Sign up for cloud account
3. Get API keys
4. Add cloud SDK to project
5. Implement auth/saves/multiplayer
6. Test thoroughly
7. Deploy games with cloud features
### For Players
1. Create AeThex account (optional)
2. Cloud saves work automatically
3. Multiplayer just works
4. Cross-platform progress
---
## Competitive Advantages
**vs. Custom Backend:**
- Zero infrastructure management
- Built-in best practices
- Automatic scaling
- Cost-effective
**vs. PlayFab/GameSparks:**
- Engine-integrated (not third-party)
- Open-source friendly
- More affordable
- Better DX for AeThex users
**vs. Epic Online Services:**
- Language-agnostic
- No platform lock-in
- More flexible pricing
- Indie-friendly
---
**Next Steps:** Build auth service MVP + basic cloud saves
---
## API Protocol Specifications
### REST API Standards
**Base URL:** `https://api.aethex.io/v1`
**Request Format:**
```json
{
"jsonrpc": "2.0",
"id": "unique-request-id",
"method": "auth.login",
"params": {
"email": "user@example.com",
"password": "hashed_password"
}
}
```
**Response Format:**
```json
{
"jsonrpc": "2.0",
"id": "unique-request-id",
"result": {
"success": true,
"data": {
"user_id": "12345",
"token": "jwt-token-here",
"refresh_token": "refresh-token-here"
}
}
}
```
**Error Format:**
```json
{
"jsonrpc": "2.0",
"id": "unique-request-id",
"error": {
"code": -32600,
"message": "Invalid credentials",
"data": {
"field": "password",
"reason": "Password incorrect"
}
}
}
```
### WebSocket Protocol
**Connection URL:** `wss://ws.aethex.io/v1/multiplayer`
**Authentication:**
```json
{
"type": "auth",
"token": "jwt-token-here"
}
```
**Room Join:**
```json
{
"type": "join_room",
"room_id": "ABCD-1234",
"player_data": {
"username": "Player1",
"character": "warrior",
"level": 10
}
}
```
**State Update (Client → Server):**
```json
{
"type": "state_update",
"timestamp": 1234567890,
"data": {
"position": {"x": 100, "y": 200, "z": 50},
"rotation": {"x": 0, "y": 45, "z": 0},
"health": 80,
"custom": {}
}
}
```
**State Broadcast (Server → Clients):**
```json
{
"type": "player_state",
"player_id": "67890",
"timestamp": 1234567890,
"data": {
"position": {"x": 150, "y": 220, "z": 55},
"rotation": {"x": 0, "y": 90, "z": 0},
"health": 75
}
}
```
**RPC Call:**
```json
{
"type": "rpc",
"method": "take_damage",
"target": "player_id_67890",
"args": [25, "fire"]
}
```
---
## Data Formats
### Cloud Save Format
**Metadata:**
```json
{
"save_id": "uuid-here",
"user_id": "12345",
"game_id": "my-awesome-game",
"slot_name": "slot1",
"version": "1.2.3",
"timestamp": "2024-01-01T12:00:00Z",
"size_bytes": 1024,
"checksum": "sha256-hash",
"compressed": true,
"engine_version": "4.3.0"
}
```
**Save Data (JSON):**
```json
{
"player": {
"name": "Hero",
"level": 15,
"xp": 4500,
"position": {"x": 100.5, "y": 0, "z": 200.3},
"rotation": {"x": 0, "y": 180, "z": 0}
},
"inventory": [
{"id": "sword_legendary", "quantity": 1, "durability": 85},
{"id": "potion_health", "quantity": 5}
],
"quests": {
"completed": ["quest_1", "quest_2"],
"active": ["quest_3"],
"progress": {"quest_3": 2}
},
"settings": {
"volume": 0.8,
"difficulty": "hard"
}
}
```
### Analytics Event Format
```json
{
"event_id": "uuid",
"user_id": "12345",
"game_id": "my-game",
"event_type": "level_completed",
"timestamp": "2024-01-01T12:00:00Z",
"session_id": "session-uuid",
"properties": {
"level": 5,
"time_seconds": 120.5,
"deaths": 3,
"score": 1000,
"difficulty": "hard"
},
"device": {
"os": "Windows",
"os_version": "10.0.19041",
"engine_version": "4.3.0",
"game_version": "1.2.3"
}
}
```
---
## Database Schema
**services/database/schema.sql:**
```sql
-- Users table
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
email VARCHAR(255) UNIQUE NOT NULL,
username VARCHAR(50) UNIQUE NOT NULL,
password VARCHAR(255) NOT NULL,
avatar_url VARCHAR(500),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
last_login TIMESTAMP,
is_active BOOLEAN DEFAULT true,
is_verified BOOLEAN DEFAULT false
);
CREATE INDEX idx_users_email ON users(email);
CREATE INDEX idx_users_username ON users(username);
-- OAuth providers
CREATE TABLE oauth_connections (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
provider VARCHAR(50) NOT NULL,
provider_user_id VARCHAR(255) NOT NULL,
access_token TEXT,
refresh_token TEXT,
expires_at TIMESTAMP,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE(provider, provider_user_id)
);
-- Sessions
CREATE TABLE sessions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
token TEXT NOT NULL,
refresh_token TEXT NOT NULL,
expires_at TIMESTAMP NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
ip_address INET,
user_agent TEXT
);
-- Cloud saves
CREATE TABLE cloud_saves (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
game_id VARCHAR(100) NOT NULL,
slot_name VARCHAR(50) NOT NULL,
data JSONB NOT NULL,
metadata JSONB,
version INTEGER DEFAULT 1,
checksum VARCHAR(64),
size_bytes INTEGER,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
UNIQUE(user_id, game_id, slot_name, version)
);
CREATE INDEX idx_saves_user_game ON cloud_saves(user_id, game_id);
-- Analytics events
CREATE TABLE analytics_events (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
game_id VARCHAR(100) NOT NULL,
event_type VARCHAR(100) NOT NULL,
properties JSONB,
session_id UUID,
timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
device_info JSONB
);
CREATE INDEX idx_analytics_game ON analytics_events(game_id);
CREATE INDEX idx_analytics_event ON analytics_events(event_type);
```
---
## Performance & Monitoring
### Optimization Strategies
**Caching (Redis):**
- User sessions (15 min TTL)
- User profiles (5 min TTL)
- Leaderboards (1 min TTL)
- Save metadata (10 min TTL)
**Database:**
- Connection pooling
- Prepared statements
- Batch inserts for analytics
- Cursor-based pagination
**WebSocket:**
- Message batching (60 FPS → 20 FPS)
- Delta compression
- Binary protocol option
- Automatic reconnection
### Metrics to Track
**System:**
- Request latency (p50, p95, p99)
- Error rates by endpoint
- Active connections
- Cache hit rate
**Business:**
- Daily/Monthly Active Users
- New registrations
- Average session length
- Cloud save frequency
---
## Implementation Guide
See [API Reference](API_REFERENCE.md) for GDScript usage examples.
**Development Phases:**
1. **Phase 1 (MVP):** Auth service + Cloud saves
2. **Phase 2:** Multiplayer relay + Matchmaking
3. **Phase 3:** Analytics + Crash reporting
4. **Phase 4:** Leaderboards + Achievements
5. **Phase 5:** CDN + DLC system
Reference in a new issue View git blame Copy permalink