The number of simultaneous synchronous requests a server is capable of supporting is determined by the number of threads in the server's thread pool. If all of the threads are busy dispatching long-running operations, then no threads are available to process new requests and therefore clients may experience an unacceptable lack of responsiveness.
Asynchronous Method Dispatch (AMD), the server-side equivalent of AMI, addresses this scalability issue. Using AMD, a server can receive a request but then suspend its processing in order to release the dispatch thread as soon as possible. When processing resumes and the results are available, the server can provide its results to the Ice run time for delivery to the client.
AMD is transparent to the client, that is, there is no way for a client to distinguish a request that, in the server, is processed synchronously from a request that is processed asynchronously.
In practical terms, an AMD operation typically queues the request data for later processing by an application thread (or thread pool). In this way, the server minimizes the use of dispatch threads and becomes capable of efficiently supporting thousands of simultaneous clients.
On this page:
Enabling AMD with Metadata in C#
To enable asynchronous dispatch, you must add an ["amd"] metadata directive to your Slice definitions. The directive applies at the interface and the operation level. If you specify ["amd"] at the interface level, all operations in that interface use asynchronous dispatch; if you specify ["amd"] for an individual operation, only that operation uses asynchronous dispatch. In either case, the metadata directive replaces synchronous dispatch, that is, a particular operation implementation must use synchronous or asynchronous dispatch and cannot use both.
Consider the following Slice definitions:
["amd"] interface I
{
bool isValid();
float computeRate();
}
interface J
{
["amd"] void startProcess();
int endProcess();
}
In this example, both operations of interface I use asynchronous dispatch, whereas, for interface J, startProcess uses asynchronous dispatch and endProcess uses synchronous dispatch.
Specifying metadata at the operation level (rather than at the interface level) minimizes complexity: although the asynchronous model is more flexible, it is also more complicated to use. It is therefore in your best interest to limit the use of the asynchronous model to those operations that need it, while using the simpler synchronous model for the rest.
AMD Mapping in C#
The asynchronous mapping for an operation differs in several ways from its synchronous mapping:
- The dispatch method name has the suffix
Async - For an operation that returns
voidand has no out parameters, the return type of the dispatch method isSystem.Threading.Tasks.Task - For an operation that returns at least one value, the dispatch method returns
System.Threading.Tasks.Task<T>, whereTrepresents the return type as described below - The dispatch method does not declare any out parameters
Let's start with some simple examples to demonstrate the asynchronous mapping:
["amd"]
interface Example
{
void opVoid(int n);
string opString();
void opStringOut(out string s);
}
The Slice-to-C# compiler generates the following base class:
public abstract class ExampleDisp_ : Ice.ObjectImpl, Example
{
public abstract System.Threading.Tasks.Task opVoidAsync(int n, Ice.Current current = null);
public abstract System.Threading.Tasks.Task<string> opStringAsync(Ice.Current current = null);
public abstract System.Threading.Tasks.Task<string> opStringOutAsync(Ice.Current current = null);
...
}
Pay particular attention to the mappings for opString and opStringOut. For operations like these that return a single value of type T (whether it's a non-void return value or an out parameter), the method returns Task<T>.
Finally, for an operation that returns multiple values, the Slice compiler generates an additional structure to hold the results. This "result type" is shared between the asynchronous proxy mapping and the asynchronous dispatch mapping. Refer to the AMI discussion for details of the mapping for result types.
Let's add another operation to our example to demonstrate the mapping for multiple return values:
["amd"]
interface Example
{
void opVoid(int n);
string opString();
void opStringOut(out string s);
string opAll(bool flag, out int count);
}
The mapping for opAll is shown below:
public struct Example_OpAllResult
{
public Example_OpAllResult(string returnValue, int count)
{
this.returnValue = returnValue;
this.count = count;
}
public string returnValue;
public int count;
}
public abstract class ExampleDisp_ : Ice.ObjectImpl, Example
{
...
public abstract System.Threading.Tasks.Task<Example_OpAllResult> opAllAsync(bool flag, Ice.Current current = null);
...
}
As you can see, the Slice compiler generated the Example_OpAllResult structure to encapsulate the results of an asynchronous invocation of opAll.
AMD Tasks in C#
A servant's implementation of an asynchronous dispatch method is responsible for providing a Task object that must eventually complete successfully or raise an exception.
Failing to complete a task will cause the client's invocation to hang because no response will ever be sent.
Using .NET's task-based asynchronous pattern provides servants with a lot of flexibility in their implementations. Here's a very simple example:
using System.Threading.Tasks;
public class ExampleI : ExampleDisp_
{
public override Task<string> opStringAsync(Ice.Current current)
{
return Task.FromResult<string>("hello world!");
}
}
The Task.FromResult method produces an already-completed task with the given result value. Clearly this implementation isn't taking advantage of asynchronous programming at all; it's just a more complex version of a synchronous implementation. Here's a slightly more interesting version:
using System.Threading.Tasks;
public class ExampleI : ExampleDisp_
{
public override Task<string> opStringAsync(Ice.Current current)
{
return Task<string>.Run(() =>
{
longRunningOperation();
return "hello world!";
});
}
}
We've called Task.Run to create a new task and supplied a lambda as our task implementation. The dispatch method returns the task to the Ice run time. Upon completion of the task, Ice will return the response to the client. We can simplify this example using the async and await keywords as follows:
using System.Threading.Tasks;
public class ExampleI : ExampleDisp_
{
public override async Task<string> opStringAsync(Ice.Current current)
{
await longRunningOperation();
return "hello world!";
}
}
These examples are just scratching the surface of what's possible for asynchronous dispatch implementations.
AMD Thread Safety in C#
As with the synchronous mapping, you can add the marshaled-result metadata to operations that return mutable types in order to avoid potential thread-safety issues. The return type of your operation will change to be Task<OpMarshaledResult>.
Chaining AMI and AMD Invocations in C#
Since the asynchronous proxy API and the asynchronous dispatch API both use tasks, chaining nested invocations together without blocking Ice's thread pool threads becomes very straightforward. Continuing our example from the previous section, suppose our servant also holds a proxy to another object of the same type and derives its response from that of the other object:
using System.Threading.Tasks;
public class ExampleI : ExampleDisp_
{
public override Task<string> opStringAsync(Ice.Current current)
{
return other.opStringAsync().ContinueWith((task) =>
{
return "hello " + task.Result;
});
}
private ExamplePrx other;
}
We've initiated an asynchronous proxy invocation on the other object and used a lambda continuation to form our result. Here's a much simpler version that uses the async and await keywords:
using System.Threading.Tasks;
public class ExampleI : ExampleDisp_
{
public override async Task<string> opStringAsync(Ice.Current current)
{
return "hello " + await other.opStringAsync();
}
private ExamplePrx other;
}
The Ice dispatch thread is not blocked by the nested call to opStringAsync. Rather, the use of await causes the dispatch thread to be released back to Ice, and the remainder of the code will be executed as a continuation when Ice receives the reply.
AMD Exceptions in C#
There are two processing contexts in which the logical implementation of an AMD operation may need to report an exception: the dispatch thread (the thread that receives the invocation and calls the servant's dispatch method), and the response thread (the thread that completes the task).
These are not necessarily two different threads: it is legal for the dispatch method to return an already-completed task.
Although we recommend that the task be used to return all exceptions to the client, it is legal for the dispatch method to raise an exception directly.
AMD Example in C#
For a more realistic example of using AMD in Ice, let's define the Slice interface for a simple computational engine:
module Demo
{
sequence<float> Row;
sequence<Row> Grid;
exception RangeError {}
interface Model
{
["amd"] Grid interpolate(Grid data, float factor)
throws RangeError;
}
}
Given a two-dimensional grid of floating point values and a factor, the interpolate operation returns a new grid of the same size with the values interpolated in some interesting (but unspecified) way.
Our servant class derives from Demo.ModelDisp_ and supplies a definition for the interpolateAsync method that adds a job to a work queue. The queue uses a Job object to hold the arguments and perform the interpolation (not shown). The job also creates a task in which the interpolation is performed.
using System.Threading;
using System.Threading.Tasks;
using System.Collections.Generic;
public class ModelI : Demo.ModelDisp_
{
public override Task<float[][]> interpolateAsync(float[][] data, float factor, Ice.Current current)
{
return _workQueue.add(data, factor);
}
private WorkQueue _workQueue = ...;
}
class WorkQueue {
public Task<float[][]> add(float[][] data, float factor)
{
lock(this)
{
Job j = new Job(data, factor);
_jobs.AddLast(j);
Monitor.Pulse(this);
return j.task();
}
}
private void Dispatch() // Runs in a separate thread
{
while(true)
{
lock(this)
{
Monitor.Wait(this);
Job job = _jobs.First.Value;
_jobs.RemoveFirst();
job.task().Start();
jot.task().Wait();
}
}
}
private LinkedList<Job> _jobs = new LinkedList<Job>();
class Job
{
public Job(float[][] data, float factor)
{
_data = data;
_factor = factor;
_task = new Task<float[][]>(() => return execute());
}
public Task<float[][]> task()
{
return _task;
}
private float[][] execute()
{
if(!interpolateGrid())
{
throw new Demo.RangeError();
}
else
{
return _data;
}
}
private boolean interpolateGrid()
{
// ...
}
private float[][] _grid;
private float _factor;
private Task<float[][]> _task;
}
}
If interpolateGrid returns false, the task raises an exception to indicate that a range error has occurred. If interpolation was successful, the task returns the modified grid as its result.