[Question] Default Executor

[zeliard]

https://github.com/awslabs/aws-sdk-cpp/blob/master/aws-cpp-sdk-core/source/utils/threading/Executor.cpp#L25

As you know, default executor is inefficient. (creating a thread - executing a function - terminating a thread )

Is there any other plan to change or substituable implementation (internally used or so)?

[JonathanHenson]

Thank you for your suggestion!

The efficiency of the default executor is largely dependent on the platform and use case. When we first started this project, I went on the assumption that the operating system implementation is the best default. In modern linux kernels and the modern windows kernels thread creation has been optimized by orders of magnitude to say.... 5 or more years ago, and it is often advisable to let the OS manage this task. In systems programming tasks, my advice is to use thread pools only for throttling but not for efficiency. I remember watching a talk from Bjarn Strosroup and Herb Sutter on this topic from C++ con last year which made this recommendation, but for the life of me I can't find a transcript of it anywhere at the moment.

In general, by choosing C++ 11 and later we have taken the general guiding principle that you should use the defaults unless you have a compelling, and verifiable with profiler metrics, reason to do otherwise.

For embedded systems or gaming, this is a completely different scenario. Hence, we want sane defaults that can be overridden for each use case. Boost, for example has an excellent pooled thread executor implementation (at least it did when I was looking at it a year or so ago).

One more thing I just thought about is that one of our core goals with this api is to not introduce any invasive dependencies. However, if you are using something like boost, you could just write a thin wrapper to marshall into boost and use that.

Furthermore, I am aware that just about every argument in C++ ends up being some sort of religious zealot argument based on theory and preference. To counter this trend we tend to be quite data driven in our development process. If you could provide some metrics on different implementations on modern operating systems, that would be helpful in our next iteration in providing better value to our customers.

I would welcome a pull request with a new pooled executor--along with your profiler metrics-- that someone could easily install, which we could offer to our customers. Remember, that it cannot have any additional dependencies beyond the core c++ libraries or the operating system apis in order for us to include it into this offering.

[zeliard]

Thank you for kind answer.

One more question,
Why did you divide the interface into two types: **Callable and **Async?
Rather, how about using std::async instead of thread creation and detaching? (let the OS manage thread-pooling..)

In gaming area, the performance of dispatching tasks is important. So you'd better give in detail in a ReadMe page like you mentioned: "Hence, we want sane defaults that can be overridden for each use case."

FYI:
In the past, we also used a custom executor like this: https://github.com/zeliard/Dispatcher

[JonathanHenson]

The callable makes a std::async call. This seems most useful to me for batching with futures and things of that nature. However overriding the behavior of std::async is non-trivial and it doesn't really have the elegance of style that using closures has.

Async supports a more functional interface and it is trivial to implement a custom executor. They are two very different use-cases IMHO hence, two separate interfaces.

You can see both use cases in code in the dynamodb integration tests.

[zeliard]

I understood the intention about interface and use-case issues.

One more thing:
How about below YetAnotherDefaultExecutor as an another option?
This satisfies the conditions above. (no dependency, no thread creation every time, easy to change, ...)

https://github.com/zeliard/aws-sdk-cpp/blob/master/aws-cpp-sdk-core/include/aws/core/utils/threading/YetAnotherDefaultExecutor.h
https://github.com/zeliard/aws-sdk-cpp/blob/master/aws-cpp-sdk-core/source/utils/threading/YetAnotherDefaultExecutor.cpp
https://github.com/zeliard/aws-sdk-cpp/blob/master/aws-cpp-sdk-core/include/aws/core/utils/memory/ObjectPool.h

[bretambrose]

I can only speak for myself here, but I lack the expertise to be able to confidently evaluate the correctness of a lockless queue implementation; I would far prefer to use an industrial-strength, well-vetted implementation like what you can find in boost or tbb.

This seems like a premature optimization: the cost of locking/unlocking a mutex is insignificant compared to the round trip time of an http request-response. A much-simpler mutex-based solution might be more appropriate.

Ultimately, we think these kinds of details and decisions are probably best left to the application or middleware programmer using the SDK.