OutWit.Controller.Grid
1.2.1
dotnet add package OutWit.Controller.Grid --version 1.2.1
NuGet\Install-Package OutWit.Controller.Grid -Version 1.2.1
<PackageReference Include="OutWit.Controller.Grid" Version="1.2.1" />
<PackageVersion Include="OutWit.Controller.Grid" Version="1.2.1" />
<PackageReference Include="OutWit.Controller.Grid" />
paket add OutWit.Controller.Grid --version 1.2.1
#r "nuget: OutWit.Controller.Grid, 1.2.1"
#:package OutWit.Controller.Grid@1.2.1
#addin nuget:?package=OutWit.Controller.Grid&version=1.2.1
#tool nuget:?package=OutWit.Controller.Grid&version=1.2.1
OutWit.Controller.Grid
An OmnibusCloud controller that provides distributed computing capabilities across multiple nodes.
Overview
This controller enables workload distribution across multiple compute nodes in the OmnibusCloud network. It handles task partitioning, node selection based on capabilities and benchmarks, and result aggregation.
Dependencies
OutWit.Controller.Variables(version 1.0.0 or higher) — forInt/IntCollection/ProcessingOptionstypes.
Activities
Grid.ForEach
Distributes iteration over a collection across available compute nodes.
Grid.ForEach(item in collection) => Transformer(item);
Grid.ForEach(item in collection, options) => Transformer(item);
Parameters:
item- Iteration variable namecollection- Collection to iterate overoptions- OptionalProcessingOptionsfor controlling distribution
Returns: Collection of transformed results
Grid.Delegate
Runs a single activity on the one most-suitable (fastest compatible) node and
returns its result — Grid.ForEach without the iteration. Use it for an inherently
sequential / heavy step that cannot be fanned out (e.g. a physics-simulation bake)
and must run once on a capable worker before the rest of the job distributes.
result = Grid.Delegate => Activity(args);
result = Grid.Delegate(options) => Activity(args);
Grid.Delegate => Activity(args); ~ procedural: no return value ~
Parameters:
options- OptionalProcessingOptions(node selection)
Returns: the delegated activity's single result (or nothing, if it is not a function)
Like Grid.ForEach, node compatibility is keyed off the delegated activity's type, and
only the variables that activity actually references are forwarded to the node. The single
task is allocated to the fastest compatible node via the same WitGridTaskAllocator.
Grid.Replicas
Builds the options of a Grid.ForEach call whose selected items run more than once, as
optional redundancy for reliability. It runs on the host and needs nothing on the nodes.
ProcessingOptions:checked = Grid.Replicas(2, 0.05); ~ 5 % of the items run twice ~
ProcessingOptions:all = Grid.Replicas(2); ~ every item runs twice ~
results = Grid.ForEach(item in items, checked) => Transformer(item);
Parameters:
copies- How many times a selected item runs, from 1 to 3share- Optional share of the items that are selected, from 0 to 1 (rounded up to a whole item); all of them when omitted
What it guarantees:
- The executions of one item run on different machines within the call, also after a node fails and its work moves. One computer registered under two accounts counts once. Different machines are not necessarily different owners.
- The platform picks the items at random and shuffles the tasks, so a node cannot tell whether the item it holds also runs elsewhere. Selected and unselected items must be in one call for that to hold: two calls reveal which is which.
- With too few machines an item gets fewer copies. A job fails only when an item has no result at all.
What it does not do:
- It does not compare results.
resultsis a flat list with one entry per execution, in no particular order, so the transformer has to return something that identifies its item. The controller compares; the platform does not vote. - It does not speed anything up: the copies are extra work.
Only the Balanced strategy supports copies; Queued reports an error. Copies are set in a script, because these options are not sent through SDK job parameters. A call with copies uses this cloud's own nodes only.
How It Works
- Node Discovery: Finds compatible nodes based on activity requirements
- Task Building: Creates tasks for each item in the collection
- Task Allocation: Distributes tasks across nodes using benchmark data
- Parallel Execution: Runs tasks on all nodes simultaneously
- Result Aggregation: Collects and combines results
Usage Examples
Basic Distributed Processing
Job:DistributedExample()
{
IntCollection:numbers = IntRange(1, 1000);
IntCollection:results;
results = Grid.ForEach(n in numbers) => ExpensiveCalculation(n);
}
With Processing Options
Job:ConfiguredDistribution()
{
IntCollection:data = IntRange(1, 10000);
IntCollection:results;
ProcessingOptions:opts = ProcessingOptions();
results = Grid.ForEach(item in data, opts) => ProcessItem(item);
}
Processing Matrix Rows
Job:MatrixProcessing(MatrixSparse:matrix)
{
IntCollection:rowIndices = IntRange(0, Matrix.RowCount(matrix));
VectorSparseCollection:processedRows;
processedRows = Grid.ForEach(idx in rowIndices) => Matrix.GetRow(matrix, idx);
}
Processing Options
Control how work is distributed:
| Property | Description |
|---|---|
Strategy |
Balanced (pre-allocate) or Queued (pull-based) |
MaxClients |
Maximum number of nodes to use |
CanRunInParallelOnClient |
Allow parallel execution within each node |
Strategy: Balanced
Work is pre-calculated and distributed based on node benchmarks. Best for:
- Predictable workloads
- Stable node availability
- Minimizing total completion time
Strategy: Queued
Nodes pull tasks from a central queue. Best for:
- Unpredictable task durations
- Dynamic node availability
- Self-balancing workloads
Project Structure
OutWit.Controller.Grid/
Activities/ - Grid.ForEach + Grid.Delegate DTOs
Adapters/ - WitActivityAdapterGridForEach (fan-out) + WitActivityAdapterGridDelegate (single node)
Builders/ - WitGridTaskBuilder — per-task variable scoping + work estimate (BuildTasks fan-out / BuildTask single)
Interfaces/ - Internal marker interface shared with adapters
Utils/ - Exception-message helpers
Properties/ - Localized error strings (Resources.resx)
build/ - Consumer-side MSBuild .targets shipped inside the nupkg
WitControllerGridModule.cs - Plugin entry point (DI registrations)
OutWit.Controller.Grid.Model/
WitGridTask.cs - Per-task (activity + variables + estimated work)
WitGridTaskGroup.cs - A node's queue of tasks + rate / ETA accounting
WitGridTaskAllocator.cs - LPT-greedy schedule + "minimize node count" heuristic, keeps copies of an item apart
WitGridReplicaPlanner.cs - Which items run more than once, and the shuffled order of the executions
Neither the Grid.ForEach nor the Grid.Delegate activity is MemoryPack-serialised — Grid is host-only (only IWitControllerHost, not IWitControllerNode), so these activities stay on the host and only their inner transformer crosses the wire per task. See the comment on WitActivityGridForEach for the design rationale.
Architecture
Grid.ForEach
|
v
[Task Builder] --> Creates WitGridTask for each item
|
v
[Node Manager] --> Finds compatible nodes via IWitNodesManager
|
v
[Task Allocator] --> Distributes tasks based on benchmarks
|
v
[Parallel Execution] --> Sends tasks to nodes
|
v
[Result Aggregation] --> Combines results into output collection
Requirements for Transformer Activities
Activities used with Grid.ForEach should:
- Have
[Requirement*]attributes if they need specific capabilities - Implement benchmarking for optimal task allocation
- Be serializable for network transfer
SDK Limitations
When using the dev-time OmnibusCloud SDK:
- Only local execution is supported (MAX_NODES = 1)
- Grid.ForEach will execute sequentially on the local machine
- Use the full OmnibusCloud runtime for actual distributed computing
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net10.0 is compatible. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed. |
-
net10.0
- OutWit.Controller.Grid.Model (>= 1.1.1)
- OutWit.Engine.Assets.MSBuild (>= 1.0.1)
- OutWit.Engine.Data (>= 1.1.5)
- OutWit.Engine.Interfaces (>= 1.1.6)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.