Repository navigation
Content Download
MakaMek uses a content downloading and caching system to handle unit data files that are not included in the application binary. This system ensures efficient loading while providing persistent storage across sessions.
- MainMenuViewModel - Entry point for content loading
- UnitCachingService - Orchestrates unit loading and caching
- GitHubResourceStreamProvider - Downloads content from GitHub
- IFileCachingService - Platform-specific caching implementations
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ MainMenuViewModel│───▶│UnitCachingService│───▶│GitHubResource │
│ │ │ │ │StreamProvider │
└─────────────────┘ └──────────────────┘ └─────────────────┘
│ │
▼ ▼
┌──────────────────┐ ┌─────────────────┐
│ Caches unit │ │IFileCaching │
│ data/images │ │Service │
└──────────────────┘ └─────────────────┘
Unit data is distributed as MMUX packages (MakaMek Unit eXchange format):
unit.mmux
├── unit.json # Unit specifications (JSON)
└── unit.png # Unit sprite image (PNG)
unit.json contains:
- Model name and specifications
- Component definitions
- Movement and combat stats
- Weight class and type information
// IndexedDB-based persistent caching
// Same SHA256 hash logic for consistency
// Data stored as Uint8Array in IndexedDB
[JSImport("getFromCache", "cacheStorage")]
[return: JSMarshalAs<JSType.Promise<JSType.Object>>()]
private static partial Task<JSObject> GetFromCacheAsObjectJs(string cacheKey);Both implementations use identical SHA256 hashing:
private static string GetHashedCacheKey(string originalKey)
{
var hashBytes = SHA256.HashData(Encoding.UTF8.GetBytes(originalKey));
return Convert.ToHexString(hashBytes).ToLowerInvariant();
}This ensures cache compatibility across platforms.
- MainMenuViewModel loads on startup
- PreloadUnits() runs in background
- UnitCachingService.EnsureInitialized() triggers
- LoadUnitsFromStreamProviders() processes each provider
- GitHubResourceStreamProvider downloads unit files
- Files cached for future sessions
Loading Content... → Loading Items... → {X} Items Loaded
- Network failures: Continue with cached content
- Corrupted files: Skip and continue with other units
- Provider failures: Log errors but continue with other providers
- Location: User application data directory
-
Format: Binary files with
.cacheextension - Persistence: Until manually cleared
- Size: Limited by available disk space
-
Location: IndexedDB (
MakaMekCachedatabase) -
Format:
Uint8Arrayobjects infileCachestore - Persistence: Until browser data cleared
- Size: Limited by browser storage quotas (~60% of disk space)
- Open DevTools (F12)
- Go to Application tab
- Navigate to Storage → IndexedDB
- Select MakaMekCache database
- Inspect fileCache object store
// Check database contents
const request = indexedDB.open('MakaMekCache');
request.onsuccess = (event) => {
const db = event.target.result;
const transaction = db.transaction(['fileCache'], 'readonly');
const store = transaction.objectStore('fileCache');
const getAllRequest = store.getAllKeys();
getAllRequest.onsuccess = () => {
console.log('Cached files:', getAllRequest.result);
};
};// Process units in parallel batches
var batches = unitIdList.Chunk(MaxDegreeOfParallelism);
foreach (var batch in batches)
{
var batchTasks = batch.Select(unitId => ProcessUnitAsync(provider, unitId));
await Task.WhenAll(batchTasks); // Parallel execution
}- Clear cache before testing
- Monitor network tab for initial downloads
- Check cache storage after loading
- Refresh page - should load from cache (no network requests)
| Issue | Symptom | Solution |
|---|---|---|
| Cache miss | Files re-downloaded | Check cache key consistency |
| Empty cache | No IndexedDB entries | Verify save operations |
| Wrong format | Uint8Array issues | Check marshalling code |
| Network errors | Download failures | Check GitHub API access |
Unit and terrain assets under data/ are also published to a Cloudflare R2 bucket by a dedicated release workflow (.github/workflows/deploy-data-release.yml). Git remains the source of truth; the bucket is a flat mirror of the data/ folder at the last released tag.
The bucket is always flat — there are no versioned subfolders and no per-release copies. Re-releases simply sync the single copy in place. Versioning happens at the bucket level: a breaking change (one the app's manifest/format can't read) is published to a new bucket, and the app's base URL is switched to point at it. Within any given bucket the layout stays flat.
-
Bucket provisioning: the R2 bucket is created upfront via Pulumi (
src/MakaMek.Infra/MakaMek.Infra.Data, run by.github/workflows/infra-data.yml, Cloudflare provider). See hub-deployment.md for the same ad-hoc workflow pattern. A new bucket (and corresponding base URL) is provisioned whenever a breaking change ships; non-breaking releases reuse the existing flat bucket. -
Trigger: push of a
v*tag, or a manualworkflow_dispatchrun. -
Manifest generation:
.github/scripts/generate-data-manifest.mjsscansdata/recursively and writesmanifest.jsonto the workspace root. -
Upload:
aws s3 sync(S3-compatible endpoint, regionauto) mirrors the wholedata/folder into the bucket with--delete, so removed files disappear from the bucket.manifest.jsonis excluded from the sync (it lives outsidedata/, so--deletewould otherwise treat the root copy as remote-only and remove it) and is then published to the bucket root after the sync completes.
Configuration uses repository secrets (R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_R2_BUCKET) and the public base URL variable vars.DATA_R2_BASE_URL.
{
"version": 1,
"generatedAtUtc": "2026-08-27T00:00:00.000Z",
"fileCount": 200,
"files": [
{
"path": "units/mechs/Atlas.mmux",
"name": "Atlas.mmux",
"hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"url": "https://<data-r2-domain>/units/mechs/Atlas.mmux"
}
]
}Top level:
-
version: Manifest schema version (currently
1), for future evolution - generatedAtUtc: ISO-8601 timestamp of generation
-
fileCount: Number of entries in
files - files: Array of file entries
Per-file:
-
path: Path relative to
data/(forward slashes, includes subfolders) -
name: File name (last segment of
path) - hash: SHA-256 of the raw file bytes (used as an opaque cache-version marker)
-
url: Public download URL (
DATA_R2_BASE_URL+/+path)
This top-level manifest is unrelated to the per-package
manifest.jsoninside each.mmux/.mmtxarchive — those schemas are unchanged.
The existing GitHub-based runtime path (GitHubResourceStreamProvider) stays active until the application-side switch to R2 is implemented (tracked separately). This pipeline realizes the "CDN Integration" future enhancement: once switched, the app consumes manifest.json from the bucket and downloads files via their url field.
- Delta Updates: Only download changed files
- Delta Updates: Only download changed files
- Compression: Reduce storage size
- Preloading: Background content preparation