Document Number: N2276=07-0136
Anthony Williams <anthony@justsoftwaresolutions.co.uk>
Just Software Solutions Ltd
2007-05-07
At Oxford, the combined EWG and LWG voted to proceed with work on Thread Pools and Futures for C++0x, even though this work had previously been destined for TR2. This paper is provided to further discussions in this area. It draws heavily on N2094 and N2185.
The building blocks of the system are three class templates: future, packaged_task, and
promise.
future<R>A future<R> encapsulates a potential value of type R, or an exception. When the
future is ready, you can retrieve the encapsulated value or exception either by using the explicit
get() member function, or by making use of the implicit conversion to R. If the future is
not ready, then the access functions will block until the future becomes ready. If a
ready future encapsulates a value, then the access functions will return that value. If a ready
future encapsulates an exception, then the access functions will throw a copy of the encapsulated
exception.
packaged_task<R>This is a wrapper for a callable object with a return type of R. A packaged_task is itself a
callable type — invoking it's function call operator will invoke the encapsulated callable object. This still doesn't get
you the value returned by the encapsulated object, though — to do that you need a
future<R> which you can obtain using the get_future() member function. Once the
function call operator has been invoked, all futures associated with a particular packaged_task will
become ready, and will store either the value returned by the invocation of the encapsulated object, or the exception
thrown by said invocation. packaged_task is intended to be used in cases where the user needs to build a thread
pool or similar structure because the standard facilities do not have the desired characteristics.
promise<R>This is an alternative means of creating futures without a callable object — the value of type
R to be returned by the associated futures is specified directly by invoking the
set_value() member function. Alternatively, the exception to be returned can be specified by invoking the
set_exception() member function. This allows the result to be set from a callback invoked from code that has no
knowledge of the promise or its associated futures.
As well as the low level building blocks, this proposal also includes high level support in the form of the
thread_pool class, and the launch_in_thread and launch_in_pool function templates.
launch_in_poollaunch_in_pool submits a task to a system-supplied thread pool to be run. It takes a single parameter: the
function or callable object to run, and returns a future which encapsulate the result of the function call. The
intent is that the library implementor can provide a thread pool that provides what they believe to be optimal performance
characteristics for their customers, taking advantage of system-specific information. It is intended that the vast majority of
applications that need thread pool functionality will use this function to allow the system to define an appropriate thread
pool.
launch_in_threadlaunch_in_thread is similar to launch_in_pool, but it creates a separate thread dedicated to the
task being run, which terminates when the task completes. It takes a single parameter: the function or callable object to run,
and returns a future which encapsulate the result of the function call. This is an enhancement over
std::thread for tasks that need a dedicated thread, but where the caller wishes to capture the returned value or
any unhandled exception.
thread_poolA thread_pool is just that: a pool of threads. These threads will run for the lifetime of the
thread_pool object. The user can submit a task to the thread_pool, which will then be added to a queue
of tasks. When one of the threads in the pool becomes idle, it will take a task from the queue and execute it. Once the task is
complete, the thread will try and obtain a new task, and so on. A task can be submitted to the thread_pool by
passing a callable object to the submit() member function. In return, the caller is given a future to
encapsulate the result of the task.
<threadpool> synopsis
namespace std
{
template <typename R> class future;
template <typename R> class future<R&&>;
template <> class future<void>;
template <typename R> class packaged_task;
template <typename F>
packaged_task<typename result_of<F()>::type> package_task(F const& f);
template <typename F>
packaged_task<typename result_of<F()>::type> package_task(F&& f);
template <typename R> class promise;
template <typename R> class promise<R&>;
template <> class promise<void>;
template <typename F>
future<typename result_of<F()>::type> launch_in_pool(F const& f);
template <typename F>
future<typename result_of<F()>::type> launch_in_pool(F&& f);
template <typename F>
future<typename result_of<F()>::type> launch_in_thread(F const& f);
template <typename F>
future<typename result_of<F()>::type> launch_in_thread(F&& f);
class thread_pool;
class future_canceled;
class future_result_moved;
class promise_result_already_set;
}
future
namespace std
{
template<typename R>
class future
{
public:
// copying
future(const future& other);
future& operator=(future const& other);
// functions to retrieve the stored value
operator R() const;
R get() const;
R move();
bool result_moved() const;
// functions to check ready state, and wait for ready
bool ready() const;
bool has_exception() const;
bool has_value() const;
void wait() const;
bool timed_wait(target_time const& wait_until) const;
// cancel the task generating the result
void cancel();
};
}
futures
future(const future& other);
Effects: The only public constructor exposed by future is the copy
constructor. The copy constructor will yield a second future which is waiting for the same result as
*this. Both the *this and the copy will become ready when the result is available, and both
will return the same stored value, or throw the same exception when the result is accessed.
future& operator=(future const& other);
Effects: Change *this to wait for the same result as other. *this and
other will both become ready when the result is available, and both
will return the same stored value, or throw the same exception when the result is accessed.
Returns: *this
operator R() const;
Returns: get()
R get() const;
Effects: Blocks until *this is ready, and retrieves the stored result. This function
is a cancellation point if ready() would return false on entry. If result_moved would
return true on entry, throws an exception of type future_result_moved.
Returns: A copy of the stored value, if any.
Throws: An exception of type future_result_moved as described above, or if there is no stored
value, throws a copy of the stored exception. If the copy constructor of the stored value throws an exception, this is
propagated to the caller.
R move();
Effects: Blocks until *this is ready, and retrieves the stored result. This function
is a cancellation point if ready() would return false on entry. If result_moved would
return true on entry, throws an exception of type future_result_moved.
Returns: A copy of the stored value x, if any, as-if by return std::move(x).
Throws: An exception of type future_result_moved as described above, or if there is no stored
value, throws a copy of the stored exception.
bool result_moved() const;
Returns: true if move has already been called on any of the futures
associated with the same state as *this, false otherwise.
bool ready() const;
Returns: true if *this has a stored result (whether a value or an exception),
false otherwise.
bool has_exception() const;
Returns: true if *this is ready with a stored exception,
false otherwise.
bool has_value() const;
Returns: true if *this is ready with a stored value,
false otherwise.
void wait() const;
Effects: Blocks until *this is ready. This function is a cancellation point if
ready() would return false on entry.
bool timed_wait(target_time const& wait_until) const;
Effects: Blocks until *this is ready or the specified time is reached. This
function is a cancellation point if ready() would return false on entry.
Returns: true if *this is ready, false if the operation
timed out.
void cancel();
Effects: None, if *this is ready on entry to this function. Otherwise attempts to
cancel the task generating the result asynchronously. If the cancellation attempt is successful, then *this
shall become ready, with a stored exception of type future_canceled. This function shall return
immediately, and not wait for *this to become ready.
future<void>
namespace std
{
template<>
class future<void>
{
public:
// copying
future(const future& other);
future& operator=(future const& other);
// functions to retrieve the stored value
void get() const;
bool result_moved() const;
// functions to check ready state, and wait for ready
bool ready() const;
bool has_exception() const;
bool has_value() const;
void wait() const;
bool timed_wait(target_time const& wait_until) const;
// cancel the task generating the result
void cancel();
};
}
This specialization is identical to the primary template, except that there is no implicit conversion operator or
move function, and get, has_value and result_moved behave as described
below.
void get() const;
Effects: Blocks until *this is ready. This function is a cancellation point if
ready() would return false on entry.
Returns: Nothing.
Throws: A copy of the stored exception, if any.
bool has_value() const;
Returns: true if *this is ready with no stored exception,
false otherwise.
future<R&&>
namespace std
{
template<typename R>
class future<R&&>
{
public:
// copying
future(const future& other);
future& operator=(future const& other);
// functions to retrieve the stored value
operator R();
R get();
R move();
bool result_moved() const;
// functions to check ready state, and wait for ready
bool ready() const;
bool has_exception() const;
bool has_value() const;
void wait() const;
bool timed_wait(target_time const& wait_until) const;
// cancel the task generating the result
void cancel();
};
}
This specialization is identical to the primary template, except that get behaves as described below.
operator R();
R get();
Returns: move().
packaged_task
namespace std
{
template<typename R>
class packaged_task
{
private:
// packaged_task is not copyable
packaged_task(packaged_task&); // for exposition only
packaged_task& operator=(packaged_task&); // for exposition only
public:
// construction and destruction
template<typename F>
explicit packaged_task(F const& f);
template<typename F>
explicit packaged_task(F&& f);
packaged_task(packaged_task&& other);
~packaged_task();
// assignment
packaged_task& operator=(packaged_task&& other);
void swap(packaged_task& other);
// result retrieval
future<R> get_future();
// execution
void operator()();
// cancellation
void cancel();
};
}
packaged_task
template<typename F>
packaged_task(F const& f);
template<typename F>
packaged_task(F&& f);
Effects: Construct a new packaged_task that encapsulates a copy of the callable object f.
packaged_task(packaged_task&& other);
Effects: Construct a new packaged_task that encapsulates the callable object previously
encapsulated in other. All futures associated with other become associated with the
newly constructed packaged_task. other no longer has an associated callable object or any associated
futures.
~packaged_task();
Effects: Destroy the packaged_task and its encapsulated callable object, if any. If the
encapsulated callable object has not yet been invoked, all futures associated with *this become
ready, with a stored exception of type future_canceled.
packaged_task& operator=(packaged_task&& other);
Effects: As if:
packaged_task temp(other);
temp.swap(*this);
Returns: *this
void swap(packaged_task& other);
Effects: *this encapsulates the callable object previously encapsulated by other;
other encapsulates the callable object previously encapsulated by *this. Any futures
previously associated with other become associated with *this; any futures previously
associated with *this become associated with other.
packaged_task
future<R> get_future();
Returns: A new future associated with *this. If the encapsulated callable object
has already been invoked, then the future is ready, with the stored result the same as-if the
future had been constructed prior to the invocation.
packaged_task
void operator()();
Effects: Nothing, if *this has no encapsulated callable object, or the encapsulated callable
object has already been invoked. Otherwise, invokes the encapsulated callable object. If the invocation of the encapsulated
callable object returns normally, then all futures associated with *this become ready with a
stored value that is a copy of the value returned. If the invocation of the encapsulated callable object throws an exception,
then all futures associated with *this become ready with a stored exception that is a copy of
the exception thrown. If the invocation of the encapsulated callable object is cancelled, then the futures
associated with *this will become ready, with a stored exception of type future_canceled.
packaged_task
void cancel();
Effects: Nothing, if *this has no encapsulated callable object, or the invocation of the
encapsulated callable object has already completed. If the invocation of the encapsulated callable object has started on another
thread, but not completed, then the task is cancelled as-if by calling cancel() on the std::thread
object for the thread in which the invocation of the encapsulated callable object is running. Otherwise, destroys the
encapsulated callable object without invoking it. All futures associated with *this become
ready, with a stored exception of type future_canceled.
package_task
template<typename F>
packaged_task<typename result_of<F()>::type> package_task(F const& f);
template<typename F>
packaged_task<typename result_of<F()>::type> package_task(F&& f);
Effects: Constructs a new packaged_task encapsulating a copy of the supplied callable object
f. The template parameter R for the packaged_task is the return type of
f().
Returns: The newly constructed packaged_task.
promise
namespace std
{
template<typename R>
class promise
{
private:
// promise is not copyable
promise(promise&); // for exposition only
promise& operator=(promise&); // for exposition only
public:
// Construction and Destruction
promise();
promise(promise&& other);
~promise();
// Assignment
promise& operator=(promise&& other);
void swap(promise& other);
// Result retrieval
future<R> get_future();
// Updating the state
void cancel();
void set_value(R const& r);
void set_value(R&& r);
void set_exception(exception_ptr e);
};
}
promise();
Effects: Create a new promise with no stored value, no stored exception, and no associated
futures.
promise(promise&& other);
Effects: Transfer the state currently associated with other to *this. All
futures previously associated with other become associated with *this.
~promise();
Effects: If *this currently has neither a stored value, nor a stored exception, all
futures associated with *this become ready, with a stored exception of type
future_canceled.
promise& operator=(promise&& x);
Effects: As if:
promise temp(other);
temp.swap(*this);
Returns: *this
void swap(promise& other);
Effects: *this encapsulates the stored value or exception (if any) previously encapsulated by
other; other encapsulates the stored value or exception (if any) previously encapsulated by
*this. Any futures previously associated with other become associated with
*this; any futures previously associated with *this become associated with
other.
promise
future<R> get_future();
Returns: A new future associated with *this. If *this already has a
stored value or exception, then the future is ready, with the stored value or exception,
respectively. Otherwise, the future is not ready.
promise
void cancel();
Effects: Nothing, if *this already has a stored value or exception. Otherwise, all
futures associated with *this become ready, with a stored exception of type
future_canceled.
void set_value(R const& r);
void set_value(R&& r);
Effects: If *this already has a stored value or exception, throws an exception of type
promise_result_already_set. Otherwise, all futures associated with *this become
ready, with a stored value that is a copy of r. If the copy constructor of r throws an
exception, the associated futures become ready with a stored exception that is a copy of that exception,
as if set by set_exception(current_exception()).
void set_exception(exception_ptr e);
Effects: If *this already has a stored value or exception, throws an exception of type
promise_result_already_set. Otherwise, all futures associated with *this become
ready, with a stored exception that is a copy of the exception referred to by e.
promise<void>
namespace std
{
template<>
class promise<void>
{
private:
// promise is not copyable
promise(promise&); // for exposition only
promise& operator=(promise&); // for exposition only
public:
// Construction and Destruction
promise();
promise(promise&& other);
~promise();
// Assignment
promise& operator=(promise&& other);
void swap(promise& other);
// Result retrieval
future<void> get_future();
// Updating the state
void cancel();
void set_value();
void set_exception(exception_ptr e);
};
}
This specialization behaves identically to the primary template, except that set_value behaves as described
below.
void set_value();
Effects: If *this already has a stored exception, or set_value has already been
called, throws an exception of type promise_result_already_set. Otherwise, all futures associated with
*this become ready, with no stored exception.
promise<R&>
namespace std
{
template<typename R>
class promise<R&>
{
private:
// promise is not copyable
promise(promise&); // for exposition only
promise& operator=(promise&); // for exposition only
public:
// Construction and Destruction
promise();
promise(promise&& other);
~promise();
// Assignment
promise& operator=(promise&& other);
void swap(promise& other);
// Result retrieval
future<R&> get_future();
// Updating the state
void cancel();
void set_value(R& r);
void set_exception(exception_ptr e);
};
}
This specialization behaves identically to the primary template, except that set_value behaves as described
below.
void set_value();
Effects: If *this already has a stored value or exception, throws an exception of type
promise_result_already_set. Otherwise, all futures associated with *this become
ready, with a stored value that is a reference to r.
launch_in_pool
template<typename F>
future<typename result_of<F()>::type> launch_in_pool(F const& f);
template<typename F>
future<typename result_of<F()>::type> launch_in_pool(F&& f);
Effects: Submits a copy of f to the system-supplied thread pool for execution. Constructs a
new future associated with the result of invoking the submitted copy of f. When the system thread pool
invokes the stored copy of f, the future will become ready, with a copy of the value returned
or the execution thrown by the invocation. The template parameter R for the future is the return type
of f(). A task running in the system-supplied thread pool can submit a task with launch_in_pool and
wait on the future returned without causing deadlock.
Returns: The newly constructed future.
launch_in_thread
template<typename F>
future<typename result_of<F()>::type> launch_in_thread(F const& f);
template<typename F>
future<typename result_of<F()>::type> launch_in_thread(F&& f);
Effects: Start a new thread which will execute a copy of f. Constructs a new
future associated with the result of invoking the copy of f. When the newly created thread has invoked
the stored copy of f, the future will become ready, with a copy of the value returned or the
execution thrown by the invocation. The template parameter R for the future is the return type of
f().
Returns: The newly constructed future.
thread_pool
namespace std
{
class thread_pool
{
private:
// thread_pool is not copyable
thread_pool(thread_pool&); // for exposition only
thread_pool& operator=(thread_pool&); // for exposition only
public:
// Construction and Destruction
explicit thread_pool(unsigned thread_count);
~thread_pool();
// Submitting tasks for execution
template<typename F>
future<typename result_of<F()>::type> submit_task(F const& f);
template<typename F>
future<typename result_of<F()>::type> submit_task(F&& f);
};
}
thread_pool
thread_pool(unsigned thread_count);
Effects: Create a new thread_pool with thread_count threads, and an empty task
queue.
~thread_pool();
Effects: Cancel all threads in the thread pool. Any tasks on the queue that have not yet been scheduled for
execution on a thread in the pool are cancelled: all futures associated with such tasks become ready, with
a stored exception of type future_canceled. Blocks until all threads in the pool have finished execution.
thread_pool for execution
template<typename F>
future<typename result_of<F()>::type> submit_task(F const& f);
template<typename F>
future<typename result_of<F()>::type> submit_task(F&& f);
Effects: Adds a copy of f to the task queue associated with *this for execution.
Constructs a new future associated with the result of invoking the submitted copy of f. When the
stored copy of f is invoked by one of the threads in the pool, the future will become ready,
with a copy of the value returned or the execution thrown by the invocation. The template parameter R for the
future is the return type of f().
Returns: The newly constructed future.
Thanks to Peter Dimov and Howard Hinnant for their earlier papers, and feedback given on the thoughts presented here.
Thanks to Clark Nelson for allowing me to submit this revised draft after the deadline.