Answering email about error handling in concurrent code

Someone emailed me today asking:

I’m writing because I’m somewhat conscious of what I would consider a rather large hole in the parallel programming literature.

… What if one or more of your tasks throws an exception? Should the thread that runs the task swallow it? Should the caught exceptions get stashed somewhere so that the "parent" thread can deal with them once the tasks are complete? (This is somewhat tricky currently in a language such as C++(98) where one cannot store an exception caught with the "catch(…)" construct). Perhaps all tasks should have a no-throw guarantee? Perhaps some kind of asynchronous error handlers might be installed, somewhat like POSIX signals? The options are many, but choosing a strategy is hard for those of us with little parallel programming experience.

I thought I’d share my response here:

That’s an excellent question. Someone asked that very question in Stockholm last month at my Effective Concurrency course, and my answer started out somewhat dismissive: "Well, it’s about the same as you do in sequential code, and all the same guarantees apply; nothrow/nofail is only for a few key functions used for commit/rollback operations, and you’d usually target the basic guarantee unless adding the strong guarantee comes along naturally for near-free. So it’s pretty much the same as always. Although, well, of course futures may transport exceptions across threads, but that’s still the same because they manifest on .get(). And of course for parallel loops you may get multiple concurrent exceptions from multiple concurrent loop bodies that get aggregated into a single exception; and then there’s the question of whether you start new loop bodies that haven’t started yet (usually no) but do you interrupt loop bodies that are in progress (probably not), and… oh, hmm, yeah, I guess it would be good to write an article about that."

So the above is now adding to my notes of things to write about. :-) Maybe some of that stream-of-consciousness may be helpful until I can get to writing it up in more detail.

I pointed him to Doug Lea’s Concurrent Programming in Java pages 161-176, "Dealing with Failure", adding that I haven’t read it in detail but the subtopics look right. Also Joe Duffy’s Concurrent Programming on Windows, pages 721-733.

If you know of a good standalone treatise focused on error handling in concurrent code, please mention it in the comments.

Effective Concurrency: Break Up and Interleave Work to Keep Threads Responsive

This month’s Effective Concurrency column, “Break Up and Interleave Work to Keep Threads Responsive”, is now live on DDJ’s website.

Sorry for the long title; suggestions welcome. I always try to word the title to make it (a) short, (b) active, and (c) advice, but sometimes I’ll settle for two of those, or just one, until a better suggestion comes along.

From the article:

What happens when this thread must remain responsive to new incoming messages that have to be handled quickly, even when we’re in the middle of servicing an earlier lower-priority message that may take a long time to process?

If all the messages must be handled on this same thread, then we have a problem. Fortunately, we also have two good solutions, both of which follow the same basic strategy: Somehow break apart the large piece of work to allow the thread to perform other work in between, interleaved between the chunks of the large item. Let’s consider the two major ways to implement that interleaving, and their respective tradeoffs in the areas of fairness and performance.

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

Use Thread Pools Correctly: Keep Tasks Short and Nonblocking (Apr 2009)

Eliminate False Sharing (May 2009)

Break Up and Interleave Work to Keep Threads Responsive (Jun 2009)

Truth In Spam

This afternoon I was just finishing up my next Effective Concurrency article (it’ll be up in a few days), when some spam email arrived. Just as my fingers’ auto-delete macro was about to fire, I noticed something odd about the name of the attachment and did a double-take:

Cool! There must be some kind of new truth-in-advertising laws for spammers.

Yes, I know that as programmers we could argue about naming all day long. We could point out that maybe “virusLoader.gif” or “exploit_exploit_muhaha.gif” would be a little better, and argue about the relative merits of camel case and underscores. But there’s no need; I think “runnable.gif” is short, clear, and definitely good enough. (Evidently someone else thought so too, and just shipped it.)

Dress Re-Hearsal?

An amusing hearse, seen on a neighborhood street:

Here’s a close-up of the license plate:

 

 

Made my morning.

VS2010 Beta 1 Now Available

For those of you who are interested in using or trying Microsoft development tools, I’m happy to report that Visual Studio 2010 Beta 1 is now available.

If you’re interested in:

• concurrency and parallel computing, check out the new concurrency runtime (ConcRT) that implements efficient work stealing for scalable code, the Asynchronous Agents Library and the Parallel Patterns Library (PPL) for C++, and the Task Parallel Library (TPL) and Parallel LINQ (PLINQ) for .NET. And some nifty concurrency debugging and profiling tools, too.
• C++0x features, check out the lambda functions (my fave), move-semantic rvalue references, the new auto, its good friend decltype, and the ever-helpful static_assert.
• functional programming in VS, enjoy the F# programming language.

Remember this is just a beta and not intended for production use, but there’s a lot of cool stuff to play around with and it should run fine side-by-side with VS2008. Feedback via forums or filing a bug/suggestion is always appreciated. Enjoy!

Effective Concurrency: Eliminate False Sharing

This month’s Effective Concurrency column, “Eliminate False Sharing”, is now live on DDJ’s website.

People keep writing asking me about my previous mentions of false sharing, even debating whether it’s really a problem. So this month I decided to treat it in depth, including:

• A compelling and realistic example where just changing a couple of lines to remove false sharing takes an algorithm from zero scaling to perfect scaling – even when many threads are merely doing reads. Hopefully after this nobody will argue that false sharing isn’t a problem. :-)
• How your performance monitoring and analysis tools do and/or don’t help you uncover the problem, and how to use them effectively to identify the culprit. Short answer: CPU activity monitors aren’t very helpful, but cycles-per-instruction (CPI) and cache miss rate measurements attributed to specific lines of source code are your friend.
• The two ways to correct the code: Reduce the frequency of writes to the too-popular cache line, or add padding to move other data off the line.
• Reusable code in C++ and C#, and a note about Java, that you can use to use padding (and alignment if available) to put frequently-updated objects on their own cache lines.

From the article:

In two previous articles I pointed out the performance issue of false sharing (aka cache line ping-ponging), where threads use different objects but those objects happen to be close enough in memory that they fall on the same cache line, and the cache system treats them as a single lump that is effectively protected by a hardware write lock that only one core can hold at a time. … It’s easy to see why the problem arises when multiple cores are writing to different parts of the same cache line… In practice, however, it can be even more common to encounter a reader thread using what it thinks is read-only data still getting throttled by a writer thread updating a different but nearby memory location…

A number of readers have asked for more information and examples on where false sharing arises and how to deal with it. … This month, let’s consider a concrete example that shows an algorithm in extremis due to false sharing distress, how to use tools to analyze the problem, and the two coding techniques we can use to eliminate false sharing trouble. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

“Use Thread Pools Correctly: Keep Tasks Short and Nonblocking” (Apr 2009)

“Eliminate False Sharing” (May 2009)

You Know When Your UI Needs Help When…

Seen at a gas station:

You know your UI has usability issues when people tape multiple signs on your gas pump to help people get through the intricate and error-prone process of purchasing fuel.

Why does the upper note exist? The trouble is that there’s a Debit button but not a Credit button, and so since Credit is the default, users need to remember to override the default before swiping the card. People will sometimes naturally forget the special step because there was originally no reminder that this needed to be done, and because most other pumps don’t work that way. One solution would be to print the information prominently near the card reader, thus standardizing the note and making it more visible. A better and simpler solution would be to do what most pumps do: Avoid the opportunity for forgetting to specify the right thing by having both buttons, Debit and Credit, and simply prompting the user to press one or the other after they swipe their card.

But the trouble behind the lower note is even more blatant and, frankly, inexcusable: Is there any good reason not to conditionally print “ENTER PIN” or “ENTER ZIP,” instead of just always printing “ENTER DATA” which is not only unclear but also one of the classic geeky words to avoid in a consumer-oriented UI?

Then again, I’m amazed that in this day and age I still see output like “1 item(s) purchased.” Apparently it’s still more important to write the programmer-friendly printf( “%d item(s)”, count ) than the user-friendly printf( “%d item%s”, count, (count==1 ? “” : “s”) ).

Effective Concurrency: Use Thread Pools Correctly – Keep Tasks Short and Nonblocking

This month’s Effective Concurrency column, “Use Thread Pools Correctly: Keep Tasks Short and Nonblocking”, is now live on DDJ’s website.

From the article:

… But the thread pool is a leaky abstraction. That is, the pool hides a lot of details from us, but to use it effectively we do need to be aware of some things a pool does under the covers so that we can avoid inadvertently hitting performance and correctness pitfalls. Here’s the summary up front:

1. Tasks should be small, but not too small, otherwise performance overheads will dominate.

2. Tasks should avoid blocking (waiting idly for other events, including inbound messages or contested locks), otherwise the pool won’t consistently utilize the hardware well — and, in the extreme worst case, the pool could even deadlock.

Let’s see why. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

“Use Thread Pools Correctly: Keep Tasks Short and Nonblocking” (Apr 2009)

A Wryly Repurposed Quotation

In my travels, I recently came across this empty store with an almost-empty box beside the front door. As seen in Monterey, CA:

Evidently some character had also noticed the empty store with its empty box, and decided to do a little walk-by wry economic commentary via repurposed quotation. Zooming on the once-empty box:

 

New Dates for Effective Concurrency Seminar in Europe: May 27-29, Stockholm, Sweden

Now that I’m over the icky flu that forced me to postpone the seminar two weeks ago, I’m happy to say that we have new dates: Effective Concurrency (Europe) will be held on May 27-29, 2009, in Stockholm, Sweden. I’ll cover the following topics:

• Fundamentals: Define basic concurrency goals and requirements • Understand applications’ scalability needs • Key concurrency patterns
• Isolation — Keep work separate: Running tasks in isolation and communicate via async messages • Integrating multiple messaging systems, including GUIs and sockets • Building responsive applications using background workers • Threads vs. thread pools
• Scalability — Re-enable the Free Lunch: When and how to use more cores • Exploiting parallelism in algorithms • Exploiting parallelism in data structures • Breaking the scalability barrier
• Consistency — Don’t Corrupt Shared State: The many pitfalls of locks–deadlock, convoys, etc. • Locking best practices • Reducing the need for locking shared data • Safe lock-free coding patterns • Avoiding the pitfalls of general lock-free coding • Races and race-related effects
• High Performance Concurrency: Machine architecture and concurrency • Costs of fundamental operations, including locks, context switches, and system calls • Memory and cache effects • Data structures that support and undermine concurrency • Enabling linear and superlinear scaling
• Migrating Existing Code Bases to Use Concurrency
• Near-Future Tools and Features

I hope to see some of you there!