Someone emailed me today asking:

I’m writing because I’m somewhat conscious of what I would consider a rather large hole in the parallel programming literature.

… What if one or more of your tasks throws an exception? Should the thread that runs the task swallow it? Should the caught exceptions get stashed somewhere so that the "parent" thread can deal with them once the tasks are complete? (This is somewhat tricky currently in a language such as C++(98) where one cannot store an exception caught with the "catch(…)" construct). Perhaps all tasks should have a no-throw guarantee? Perhaps some kind of asynchronous error handlers might be installed, somewhat like POSIX signals? The options are many, but choosing a strategy is hard for those of us with little parallel programming experience.

I thought I’d share my response here:

That’s an excellent question. Someone asked that very question in Stockholm last month at my Effective Concurrency course, and my answer started out somewhat dismissive: "Well, it’s about the same as you do in sequential code, and all the same guarantees apply; nothrow/nofail is only for a few key functions used for commit/rollback operations, and you’d usually target the basic guarantee unless adding the strong guarantee comes along naturally for near-free. So it’s pretty much the same as always. Although, well, of course futures may transport exceptions across threads, but that’s still the same because they manifest on .get(). And of course for parallel loops you may get multiple concurrent exceptions from multiple concurrent loop bodies that get aggregated into a single exception; and then there’s the question of whether you start new loop bodies that haven’t started yet (usually no) but do you interrupt loop bodies that are in progress (probably not), and… oh, hmm, yeah, I guess it would be good to write an article about that."

So the above is now adding to my notes of things to write about. :-) Maybe some of that stream-of-consciousness may be helpful until I can get to writing it up in more detail.

I pointed him to Doug Lea’s Concurrent Programming in Java pages 161-176, "Dealing with Failure", adding that I haven’t read it in detail but the subtopics look right. Also Joe Duffy’s Concurrent Programming on Windows, pages 721-733.

If you know of a good standalone treatise focused on error handling in concurrent code, please mention it in the comments.

This month’s Effective Concurrency column, “Break Up and Interleave Work to Keep Threads Responsive”, is now live on DDJ’s website.

Sorry for the long title; suggestions welcome. I always try to word the title to make it (a) short, (b) active, and (c) advice, but sometimes I’ll settle for two of those, or just one, until a better suggestion comes along.

From the article:

What happens when this thread must remain responsive to new incoming messages that have to be handled quickly, even when we’re in the middle of servicing an earlier lower-priority message that may take a long time to process?

If all the messages must be handled on this same thread, then we have a problem. Fortunately, we also have two good solutions, both of which follow the same basic strategy: Somehow break apart the large piece of work to allow the thread to perform other work in between, interleaved between the chunks of the large item. Let’s consider the two major ways to implement that interleaving, and their respective tradeoffs in the areas of fairness and performance.

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

Use Thread Pools Correctly: Keep Tasks Short and Nonblocking (Apr 2009)

Eliminate False Sharing (May 2009)

Break Up and Interleave Work to Keep Threads Responsive (Jun 2009)

This afternoon I was just finishing up my next Effective Concurrency article (it’ll be up in a few days), when some spam email arrived. Just as my fingers’ auto-delete macro was about to fire, I noticed something odd about the name of the attachment and did a double-take:

Cool! There must be some kind of new truth-in-advertising laws for spammers.

Yes, I know that as programmers we could argue about naming all day long. We could point out that maybe “virusLoader.gif” or “exploit_exploit_muhaha.gif” would be a little better, and argue about the relative merits of camel case and underscores. But there’s no need; I think “runnable.gif” is short, clear, and definitely good enough. (Evidently someone else thought so too, and just shipped it.)

An amusing hearse, seen on a neighborhood street:

Here’s a close-up of the license plate:

Made my morning.

For those of you who are interested in using or trying Microsoft development tools, I’m happy to report that Visual Studio 2010 Beta 1 is now available.

If you’re interested in:

• concurrency and parallel computing, check out the new concurrency runtime (ConcRT) that implements efficient work stealing for scalable code, the Asynchronous Agents Library and the Parallel Patterns Library (PPL) for C++, and the Task Parallel Library (TPL) and Parallel LINQ (PLINQ) for .NET. And some nifty concurrency debugging and profiling tools, too.
• C++0x features, check out the lambda functions (my fave), move-semantic rvalue references, the new auto, its good friend decltype, and the ever-helpful static_assert.
• functional programming in VS, enjoy the F# programming language.

Remember this is just a beta and not intended for production use, but there’s a lot of cool stuff to play around with and it should run fine side-by-side with VS2008. Feedback via forums or filing a bug/suggestion is always appreciated. Enjoy!

This month’s Effective Concurrency column, “Eliminate False Sharing”, is now live on DDJ’s website.

People keep writing asking me about my previous mentions of false sharing, even debating whether it’s really a problem. So this month I decided to treat it in depth, including:

• A compelling and realistic example where just changing a couple of lines to remove false sharing takes an algorithm from zero scaling to perfect scaling – even when many threads are merely doing reads. Hopefully after this nobody will argue that false sharing isn’t a problem. :-)
• How your performance monitoring and analysis tools do and/or don’t help you uncover the problem, and how to use them effectively to identify the culprit. Short answer: CPU activity monitors aren’t very helpful, but cycles-per-instruction (CPI) and cache miss rate measurements attributed to specific lines of source code are your friend.
• The two ways to correct the code: Reduce the frequency of writes to the too-popular cache line, or add padding to move other data off the line.
• Reusable code in C++ and C#, and a note about Java, that you can use to use padding (and alignment if available) to put frequently-updated objects on their own cache lines.

From the article:

In two previous articles I pointed out the performance issue of false sharing (aka cache line ping-ponging), where threads use different objects but those objects happen to be close enough in memory that they fall on the same cache line, and the cache system treats them as a single lump that is effectively protected by a hardware write lock that only one core can hold at a time. … It’s easy to see why the problem arises when multiple cores are writing to different parts of the same cache line… In practice, however, it can be even more common to encounter a reader thread using what it thinks is read-only data still getting throttled by a writer thread updating a different but nearby memory location…

A number of readers have asked for more information and examples on where false sharing arises and how to deal with it. … This month, let’s consider a concrete example that shows an algorithm in extremis due to false sharing distress, how to use tools to analyze the problem, and the two coding techniques we can use to eliminate false sharing trouble. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

“Use Thread Pools Correctly: Keep Tasks Short and Nonblocking” (Apr 2009)

“Eliminate False Sharing” (May 2009)

Seen at a gas station:

You know your UI has usability issues when people tape multiple signs on your gas pump to help people get through the intricate and error-prone process of purchasing fuel.

Why does the upper note exist? The trouble is that there’s a Debit button but not a Credit button, and so since Credit is the default, users need to remember to override the default before swiping the card. People will sometimes naturally forget the special step because there was originally no reminder that this needed to be done, and because most other pumps don’t work that way. One solution would be to print the information prominently near the card reader, thus standardizing the note and making it more visible. A better and simpler solution would be to do what most pumps do: Avoid the opportunity for forgetting to specify the right thing by having both buttons, Debit and Credit, and simply prompting the user to press one or the other after they swipe their card.

But the trouble behind the lower note is even more blatant and, frankly, inexcusable: Is there any good reason not to conditionally print “ENTER PIN” or “ENTER ZIP,” instead of just always printing “ENTER DATA” which is not only unclear but also one of the classic geeky words to avoid in a consumer-oriented UI?

Then again, I’m amazed that in this day and age I still see output like “1 item(s) purchased.” Apparently it’s still more important to write the programmer-friendly printf( “%d item(s)”, count ) than the user-friendly printf( “%d item%s”, count, (count==1 ? “” : “s”) ).

This month’s Effective Concurrency column, “Use Thread Pools Correctly: Keep Tasks Short and Nonblocking”, is now live on DDJ’s website.

From the article:

… But the thread pool is a leaky abstraction. That is, the pool hides a lot of details from us, but to use it effectively we do need to be aware of some things a pool does under the covers so that we can avoid inadvertently hitting performance and correctness pitfalls. Here’s the summary up front:

1. Tasks should be small, but not too small, otherwise performance overheads will dominate.

2. Tasks should avoid blocking (waiting idly for other events, including inbound messages or contested locks), otherwise the pool won’t consistently utilize the hardware well — and, in the extreme worst case, the pool could even deadlock.

Let’s see why. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

“Use Thread Pools Correctly: Keep Tasks Short and Nonblocking” (Apr 2009)

In my travels, I recently came across this empty store with an almost-empty box beside the front door. As seen in Monterey, CA:

Evidently some character had also noticed the empty store with its empty box, and decided to do a little walk-by wry economic commentary via repurposed quotation. Zooming on the once-empty box:

Now that I’m over the icky flu that forced me to postpone the seminar two weeks ago, I’m happy to say that we have new dates: Effective Concurrency (Europe) will be held on May 27-29, 2009, in Stockholm, Sweden. I’ll cover the following topics:

• Fundamentals: Define basic concurrency goals and requirements • Understand applications’ scalability needs • Key concurrency patterns
• Isolation — Keep work separate: Running tasks in isolation and communicate via async messages • Integrating multiple messaging systems, including GUIs and sockets • Building responsive applications using background workers • Threads vs. thread pools
• Scalability — Re-enable the Free Lunch: When and how to use more cores • Exploiting parallelism in algorithms • Exploiting parallelism in data structures • Breaking the scalability barrier
• Consistency — Don’t Corrupt Shared State: The many pitfalls of locks–deadlock, convoys, etc. • Locking best practices • Reducing the need for locking shared data • Safe lock-free coding patterns • Avoiding the pitfalls of general lock-free coding • Races and race-related effects
• High Performance Concurrency: Machine architecture and concurrency • Costs of fundamental operations, including locks, context switches, and system calls • Memory and cache effects • Data structures that support and undermine concurrency • Enabling linear and superlinear scaling
• Migrating Existing Code Bases to Use Concurrency
• Near-Future Tools and Features

I hope to see some of you there!

The latest casualties in the technical education world are the Software Development conferences – SD West, SD Best Practices, and Architecture & Design World – which are being discontinued effective immediately, making the SD West that was just held earlier this month the last of its kind. The conferences were run by the same company that ran some important but now-defunct magazines including C/C++ Users Journal and of course Software Development Magazine, as well as Dr. Dobb’s which still exists but recently went web-only.

SD, especially SD West, has been a mainstay of the programming conferences world since 1985. I’m going to miss these shows. They were a wonderful gathering place for talks and tutorials by A-list speakers covering many technologies, from C++ to Java to AJAX and more. More personally, Bjarne Stroustrup and I immensely enjoyed giving our two-day Stroustrup & Sutter seminar there several times in recent years, and skipped this year’s event in anticipation of doing a fresh one again next spring. Alas, it’s not to be (at least not with SD).

SD, you will be missed.

This month’s Effective Concurrency column, “Use Threads Correctly = Isolation + Asynchronous Messages”, is now live on DDJ’s website.

From the article:

Explicit threads are undisciplined. They need some structure to keep them in line. In this column, we’re going to see what that structure is, as we motivate and illustrate best practices for using threads — techniques that will make our concurrent code easier to write correctly and to reason about with confidence.

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns and the DDJ print magazine issue in which they first appeared:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

Right now I should be at 40,000 feet somewhere over Baffin Island on my way to Stockholm for Effective Concurrency Europe, but instead I’m in bed with a fever that I’ve had since Wednesday night and still unable to talk. The organizer and I have been staying in touch with flu updates every few hours to see if lots of “TLC and OTC” (tender loving care and over-the-counter drugs) could get me well enough to fly and push through teaching for three days; that usually works, but this time it’s a really nasty one, and we have to postpone.

I’m sorry to have to disappoint this week and for any inconvenience this will surely cause some of you, and I hope that you’ll understand. The organizer has already emailed everyone who has registered to notify you of the cancellation of this week’s dates (this post partly serves as a backup to that email), and we’ll keep you posted as we reschedule. The response has exceeded expectations by a good margin, and I’m grateful for your strong interest and planning to take time away from your schedules to come and talk concurrency with me. I’m still very much looking forward to seeing you there in the near future. (And minus potentially spreading wicked germs.)

Best wishes to you and yours,

Herb

Like many areas in the United States, Seattle has recently been hit with layoffs and downsizing in our industry. So it’s quite timely that Steve McConnell’s company Construx, in the Seattle area, is offering free training for laid-off software workers:

After listening to doom and gloom economic reports for the past few months, we decided we would try to do something to brighten our little corner of the world. Here’s our official press release about it:

Construx Software has designated 25% of its public seminar seats free of charge to software workers who have been laid off. Construx seminars help software professionals improve their technical and managerial skills. Seminar attendees will be more effective when they reenter the workforce. Construx hopes this program will help laid-off software workers reenter the workforce more quickly.

Please read the full article for more details.

I’m very pleased to report that this program will include my next U.S. Effective Concurrency seminar which will be held in Bellevue, WA on July 27-30, 2009. I’ll blog about that seminar again as we get closer, but it’s on the Construx calendar now.

I hope that this will be helpful to some people, and I look forward to seeing some of you in Bellevue.

Aside: Please Be Professional

One problem with offering things for free is that people don’t always value them. I was surprised to learn that a surprising number of people who’ve asked for and received this free admission at Construx have been no-shows.

The following should go without saying, but here it is: As a courtesy to Construx and to other attendees, please don’t sign up unless you intend to come, and if something comes up that prevents you from attending please notify Construx. Otherwise, you may be taking the place of someone who would have liked to attend in your place (seating is limited), and Construx is out the cost of food (they provide breakfast and lunch) and printed color binders and such that go to waste.

Other Free Materials

Most of you know that virtually all the articles and columns I’ve ever written have always been available for free on my own website or via magazine websites (modulo some link rot, sigh). These include all the Effective Concurrency articles, and my C++-focused “Guru of the Week” and magazine articles.

The only things I’ve written that aren’t legally freely available are the final texts of my books, including translations, to which the publishing company owns the copyright; but the books are based on the same freely-available English articles, and Addison-Wesley/Pearson has never had any issue with those staying available, so the basic material is all there.

But there’s news on the “final book contents” front, too, at least in English: Once my current book(s) on Effective Concurrency is done, Jim Hyslop and I plan to compile a book of the C++ Conversations articles we coauthored in C++ Report and C/C++ Users Journal (thanks again for your patience, Jim!) and I also plain to write a second edition of Exceptional C++ – and both books will be updated for C++0x, which is now feature-complete and undergoing its first round of international review. It’s quite interesting just how often using C++0x language and library features (from lambdas to shared_ptrs and concurrency) really help solve the old problems in even more elegant and robust ways… and I’m sure we’ll throw in a couple of new problems and solutions too. My goal is to post these books’ draft Items, perhaps in the form of blog entries, as we write them. But I’m also working with the publisher to see if perhaps we could even post the very final Items in that format with their permission. I’ll post more news about that as there’s something to say.

This month’s Effective Concurrency column, “Sharing Is the Root of All Contention”, is now live on DDJ’s website.

This article aims to address the root cause behind some frequently made assertions: Statements like “locks kill scalability” and “CAS kills scalability” are mostly true but focus on symptoms rather than causes; and others such as “reader/writer mutexes are great for scalability” at minimum need some heavy qualification.

From the article:

Quick: What fundamental principle do all of the following have in common?

• Two children drawing with crayons.
• Three customers at Starbucks adding cream to their coffee.
• Four processes running on a CPU core.
• Five threads updating an object protected by a mutex.
• Six threads running on different processors updating the same lock-free queue.
• Seven modules using the same reference-counted shared object.

Answer: In each case, multiple users share a resource that requires coordination because not everyone can use it simultaneously — and such sharing is the root cause of all resulting contention that will degrade everyone’s performance. (Note: The only kind of shared resource that doesn’t create contention is one that inherently requires no synchronization because all users can use it simultaneously without restriction, which means it doesn’t fall into the description above. For example, an immutable object can have its unchanging bits be read by any number of threads or processors at any time without synchronization.)

In this article, I’ll deliberately focus most of the examples on one important case, namely mutable (writable) shared objects in memory, which are an inherent bottleneck to scalability on multicore systems. But please don’t lose sight of the key point, namely that "sharing causes contention" is a general principle that is not limited to shared memory or even to computing.

The Inherent Costs of Sharing

Here’s the issue in one sentence: Sharing fundamentally requires waiting and demands answers to expensive questions. …

Aside: This will be in the first electronic-only Dr. Dobb’s Report, and I’m going to have to get used to Dobb’s being electronic-only. One consequence of this change that I just discovered today is that an article can go online hours after I correct the proofs. That may seem like it shouldn’t be surprising in the days of blogs, but a magazine isn’t (or at least wasn’t) like a blog. When I wrote for magazines like C++ Report in the late 1990s, the time between submitting the final column manuscript and seeing it in print was about two to three months, depending on the magazine. Since the mid-2000s with Dobb’s, it’s been only one month from proofs to print — and Dobb’s could have posted the online versions even sooner, but held them back to be semi-synced with the print magazine. Now that the print constraint has disappeared, apparently I can (and, today, did) start work on a Friday morning to find the Dobb’s proofs I corrected the night before already on the web. Nifty. (And thanks again, Deirdre!)

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns and the DDJ print magazine issue in which they first appeared:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

I just saw a CNN headline that read: “Young workers scrimp to live on $15/wk.” Before reading further, what do you think: Is that stunning and shocking? Or shockingly typical?

The story turned out to be a piece about white-collar workers in China trying to live frugally, spending only 100 Yuan on travel and food during the workweek to conserve funds. Of course, the workers’ actual total expenses and income are higher, because that $15/week figure doesn’t include weekend expenses and other major costs like rent. Even so, the story is considered newsworthy here, and is probably a shock to a number of readers in the western world.

But the headline wouldn’t surprise readers who are familiar with the approximate distribution of income/GDP/wealth in the world.

To illustrate, here are two personal data points from 2006, when my wife and I traveled to Kenya and Zambia to visit friends:

• Income: In Kenya, we were told that being a staff worker at a safari lodge is considered a good job. What does it pay? About $2 per day, for long hours and six-day weeks. This isn’t unusual; in about 30 countries, including Kenya, more than half of the population earns under $2 per day. An estimated two billion people – 30% of the world’s population – live on an income of less than $3 per day. And $3 per day is about what the attention-grabbing CNN headline implies, though the actual story behind that headline is much less bad.
• Cost of living: But what happens when we consider, not just dollar-for-dollar comparisons, but purchasing power? Isn’t it less expensive to live in less-developed countries? Yes, it usually is, especially for shelter and services – there’s been some talk lately on the U.S. news about retiring in Mexico as a way for older people to save money in this economy – but the difference for the same quality goods is often less than one might think. In Lusaka, the capital of Zambia, we found there were only three grocery stores [*] having similar goods to what we would expect to find in a U.S.-style Safeway, although of course the Zambian stores were much smaller than U.S. stores (more the size of a medium Trader Joe’s) and offered far less selection diversity (something like a factor of 20 fewer varieties or brands). When we visited one of the stores, I picked up a few Western-style items, totaled the price and converted to U.S. dollars in my head, and found that those comparable products in Lusaka cost nearly the same as we would have paid for the same items in Seattle. Our local friend replied: “Right, nobody who lives here would ever think of buying a can of soda pop.” Certainly not when a can of Coke costs a day’s wage for many people, and doesn’t confer any significant nutritional benefit.

The gulf between the western- and world-median standards of living is, simply put, vast – and growing. The standard of living that’s normal for most of the planet’s population is well nigh unimaginable to many of us in the western world, and even for those of us who’ve been there, it’s one thing to see it and quite another to really understand what such a life would be like. I don’t claim to.

[*] They might well be the only such stores in the country, not just the capital.

A number of people have asked whether I will be teaching my Effective Concurrency seminar in Europe. The answer is yes:

Effective Concurrency (Europe) will be held on March 16-18, 2009, in Stockholm, Sweden. This is my only public European seminar in 2009. I’ll cover the following topics:

• Fundamentals: Define basic concurrency goals and requirements • Understand applications’ scalability needs • Key concurrency patterns
• Isolation — Keep work separate: Running tasks in isolation and communicate via async messages • Integrating multiple messaging systems, including GUIs and sockets • Building responsive applications using background workers • Threads vs. thread pools
• Scalability — Re-enable the Free Lunch: When and how to use more cores • Exploiting parallelism in algorithms • Exploiting parallelism in data structures • Breaking the scalability barrier
• Consistency — Don’t Corrupt Shared State: The many pitfalls of locks–deadlock, convoys, etc. • Locking best practices • Reducing the need for locking shared data • Safe lock-free coding patterns • Avoiding the pitfalls of general lock-free coding • Races and race-related effects
• High Performance Concurrency: Machine architecture and concurrency • Costs of fundamental operations, including locks, context switches, and system calls • Memory and cache effects • Data structures that support and undermine concurrency • Enabling linear and superlinear scaling
• Migrating Existing Code Bases to Use Concurrency
• Near-Future Tools and Features

I hope to see some of you there!

While walking our dogs recently, we came across several of these signs — ironically, in front of our neighborhood school.

This month’s Effective Concurrency column, “volatile vs. volatile”, is now live on DDJ’s website and also appears in the print magazine. (As a historical note, it’s DDJ’s final print issue, as I mentioned previously.)

This article aims to answer the frequently asked question: “What does volatile mean?” The short answer: “It depends, do you mean Java/.NET volatile or C/C++ volatile?” From the article:

What does the volatile keyword mean? How should you use it? Confusingly, there are two common answers, because depending on the language you use volatile supports one or the other of two different programming techniques: lock-free programming, and dealing with ‘unusual’ memory.

Adding to the confusion, these two different uses have overlapping requirements and impose overlapping restrictions, which makes them appear more similar than they are. Let’s define and understand them clearly, and see how to spell them correctly in C, C++, Java and C# — and not always as volatile. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns and the DDJ print magazine issue in which they first appeared:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

A few days ago I posted a challenge to name the researcher/team and approximate year each of the following 16 important technologies was first demonstrated. In brief, they were:

• The personal computer for dedicated individual use all day long.
• The mouse.
• Internetworks.
• Network service discovery.
• Live collaboration and desktop/app sharing.
• Hierarchical structure within a file system and within a document.
• Cut/copy/paste, with drag-and-drop.
• Paper metaphor for word processing.
• Advanced pattern search and macro search.
• Keyword search and multiple weighted keyword search.
• Catalog-based information retrieval.
• Flexible interactive formatting and line drawing.
• Hyperlinks within a document and across documents.
• Tagging graphics, and parts of graphics, as hyperlinks.
• Shared workgroup document collaboration with annotations etc.
• Live shared workgroup collaboration with live audio/video teleconference in a window.

A single answer to all of the above: Doug Engelbart and his ARC team, in what is now known as “The Mother of All Demos”, on Monday, December 9, 1968.

Last month, we marked the 40th anniversary of the famous Engelbart Demo, a truly unique “Eureka!” moment in the history of computing. 40 years go, Engelbart and his visionary team foresaw — and prototyped and demonstrated — many essential details of what we take for granted as our commonplace computing environment today, including all of the above-listed technologies, most of them demonstrated for the first time in that talk.

This talk would be noteworthy and historic just for being the first time a “mouse” was shown and called by that name. Yet the mouse was just one of over a dozen important innovations to be compellingly presented with working prototype implementations.

Note: Yes, some of the individual technologies have earlier theoretical roots. I deliberately phrased the question to focus on implementations because it’s great to imagine a new idea, but it isn’t engineering until we prove it can work by actually building it. For example, consider hypertext: Vannevar Bush’s Memex, vintage 1945, was a theorectical “proto-hypertext” system but it unfortunately remained theoretical, understandably so given the nascent state of computers at the time. Project Xanadu, started in 1960, pursued similar ideas but wasn’t demonstrated until 1972. The Engelbart Demo was the first time that hypertext was publicly shown in a working form, together with a slew of other important working innovations that combined to deliver an unprecedented tour de force. What made it compelling wasn’t just the individual ideas, but the working demonstrations to show that the ideas worked and how they could combine and interact in wonderful ways.

Recommended viewing

You can watch the 100-minute talk here (Stanford University) in sections with commentary, and here (Google Video) all in one go.

We enjoy such an abundance of computing riches that it’s easy to take wonderful technological ideas for granted. Yet so many of the pieces of our modern computing experience that we consider routine today were at one time unimaginable. After all, back in the early days of computing, we were still discovering what these newfangled room-filling gadgets might eventually become capable of — who could have known then what using computers would be like today?

Of course, we have these technologies today because some visionaries did know, did imagine them… and, best of all, built and demonstrated them.

Hence today’s challenge:

Quiz: For each of the following 16 technologies that have become commonplace in our modern computing experience, give the researcher/team and approximate year that a working prototype was first demonstrated. How many can you answer without a web search?

• The personal computer for dedicated individual use, that one person can have at their disposal all day long. (Hint: Before the Altair in 1975 and Apple I in 1976.)
• Mouse input with a graphical pointer. (Hint: Before the Xerox Alto at Xerox PARC in 1973.)
• Internetworks across campuses and cities. (Hint: Before Ethernet at Xerox PARC (again) in 1973.)
• Discovery of ‘who’s got what service’ in an internetwork.
• Using internetworks for live collaboration, not just file sharing. (Hint: Before RDP and others.)
• Hierarchical structure within a file system and within a document. (Hint: Before Unix.)
• Cut/copy/paste, with drag-and-drop.
• Paper metaphor for word processing, starting with a blank piece of paper and the applying  formatting and navigating levels in the structure of text.
• Advanced pattern search and macro search within documents. (Hint: Before MIT’s Emacs.)
• Keyword search and multiple weighted keyword search. (Hint: Long before Google (alternate link).)
• Information retrieval through indirect construction of a catalog.
• Flexible interactive formatting and line drawing.
• Hyperlinks within a document and across documents, and “jumping on a link” to navigate. (Hint: Before Tim Berners-Lee invented the World Wide Web in 1989-1990.) (Hint’: Yes, before HyperCard too.)
• Tagging graphics, and parts of graphics, as hyperlinks. (Hint: Before Flickr.)
• Workgroup collaboration on a document, including collaborative annotations, allowing members of a group to use and modify a document. (Hint: Before Lotus Notes and Ward Cunningham’s Wikis.)
• The next step up from that: Live collaboration on a document with screen sharing on the two writers’ computers so they can see what the other is doing — with live audio/video teleconference in a window at the same time. (Hint: Not Skype or LiveMeeting.)

[edited 2009.01.15 to add link to DDJ's announcement]

2008 was quite a year, full of landmark events that were certainly historic, if not always welcome.

If I had to pick one technology-related highlight from the past year, it would be this: A notable inflection point in the ongoing shift from traditional media to the web. Given that that tide is still in progress, why single out 2008? I think we’ll look back at 2008, especially the fourth quarter, as a turning point when the web became an A-list media outlet and first started to beat up, and even replace, major legacy competitors in newspapers, technical magazines, movies, and TV. An inflection point, if you will, where the web clearly stopped being the pencil-necked upstart, and visibly emerged as the new gorilla flexing its advertising-revenue-pumped-up muscles and kicking sand on the others.

In 2008, and particularly in the last month, the web began to outright replace some existing newspapers and technology and programming magazines.

• December 2008: The first two major city newspapers go web-only or web-mostly. The Detroit News and The Detroit Free Press announced truncated print editions and reduced print delivery. Newspapers in several other cities seem likely to follow fairly soon.
• As of January 2009: PC Magazine is going “digital only.” After the January issue, the print magazine will disappear. Instead, the content will appear exclusively on the web.
• As of January 2009: Dr. Dobb’s Journal is permanently suspending print publication and going web-only. Some of the content will be available as a new “Dr. Dobb’s Report” section of InformationWeek. My Effective Concurrency column will continue, and I’ll continue to blog when new columns go live on the web, so if you’re reading the column via this blog nothing will significantly change for you.

As of next month, the only major technical programmer’s trade magazines still available in print, that I know of, are platform- and technology-specific ones like asp.netPRO and MSDN Magazine — and even those increasingly feature online-only content. For example, from MSDN Mag’s January 2009 editor’s note:

As we continue to grow our coverage to keep pace with the rapidly expanding set of technologies, we will often offer content exclusively online at msdn.microsoft.com/magazine. So please check in frequently!

Gotta love RSS (and Atom etc.): Every text feed is like a magazine or newspaper column, every blogger a columnist. Every audio/video podcast feed is like a radio or TV series, or a radio station or TV channel. And our feed reader is the new magazine/newspaper, as we subscribe to columnists to make our personal custom newsmagazine. But RSS readers, along with RSS-consuming clients like iTunes, are more — they’re our personal selection, not only of the columns we want to read (on whatever topics we want, including the funnies section), but also of the media we want to hear and watch. It’s increasingly our way to choose the text, audio, and video we want all together. Who knew that a large chunk of the coming media convergence would come in the shape of RSS readers?

And other media are feeling the pressure from Dr. Web, the new gorilla:

• This week (late December 2008), even cable operator Time Warner pushed web delivery for TV. Time Warner has had various contract disputes with Viacom and some local stations. But as part of the dispute:
  Time Warner will respond to Viacom’s advertisement, [Time Warner spokesman] Mr. Dudley said, by highlighting the availability of television content on the Internet.

  “We will be telling our customers exactly where they can go to see these programs online,” Mr. Dudley said. “We’ll also be telling them how they can hook up their PCs to a television set.”

• During October-December 2008, Netflix’s Watch Instantly has started to turn into a juggernaut. It’s interesting enough that you can watch 12,000+ movies and TV shows streamed over the net to your PC at high quality. As of October 2008, you can get them on your Mac. As of November 2008, on your Xbox. As of December 2008, on your TiVo. As noted in one recent article:
  “It’s a good strategic move,” said Andy Hargreaves, an analyst with Pacific Crest Securities. “Netflix sees the world will go digital, no matter what they do. They realize there is more to be lost by waiting than doing it early.”

And I won’t even get started on SaaS: hosted rich GUI apps served up on the web, for example the December 2008 (again) announcement about Office Online.

Yes, don’t forget 2008, especially December 2008: The month the first major newspapers moved mostly to the web and abandoned print at least partly; the month that PC Magazine and Dr. Dobb’s suspended print publication and went web-only; the month Netflix Watch Instantly arrived in the living room on TiVos after hitting Xboxes a fortnight before; the month cable provider Time Warner threatened to tell people to watch TV on the net; and the month even Microsoft announced their intent to deliver significant Office web applications.

I for one welcome Dr. Web, our new Gorilla and media overlord!

Besides, what choice do I have?

Lest people get the wrong idea, I enjoy reading Jeff Atwood’s blog and agree with much of what he writes so entertainingly and provocatively. So far I’ve only responded when I strongly felt differently about something, which has been a grand total of twice now.

So let me also offer an example of something I wholeheartedly agree with. Yesterday, Jeff cited what is also my own favorite Programming Pearls figure:

Despite the enduring wonder of the yearly parade of newer, better hardware, we’d also do well to remember my all time favorite graph from Programming Pearls:

Everything is fast for small n.

Spot on. If you’re a professional programmer and haven’t read Programming Pearls yet, “run don’t walk” to your bookstore of choice.

Incidentally, just to tie this in to parallel computing as well, Jeff’s article also cites a nice graph of optimizations that improved NDepend:

Patrick Smacchia’s lessons learned from a real-world focus on performance is a great case study in optimization.

Patrick was able to improve nDepend analysis performance fourfold, and cut memory consumption in half. As predicted, most of this improvement was algorithmic in nature, but at least half of the overall improvement came from a variety of different optimization techniques.

As I’ve said many times, measure twice, optimize once: Know when and where to optimize. Profilers are your friend. As Patrick writes:

When it comes to enhancing performance there is only one way to do things right: measure and focus your work on the part of the code that really takes the bulk of time, not the one that you think takes the bulk of time.

And of course, in our ever-more-multicore world, the contribution of parallelization gain will continue to grow and dominate the optimization of CPU-bound code. But as Patrick also notes, realizing that gain is not always trivial:

While we get the 15% gain from between 1 and 2 processors, the gain is almost zero between 2 and 4 processors. We identified some potential IO contentions and memory issues that will require more attention in the future. This leads to another lesson: Don’t expect that scaling on many processors will be something easy, even if you don’t share states and don’t use synchronization.

Yesterday, the ever-popular Jeff Atwood (of Coding Horror fame) wrote an article [*] on how not to write Web 2.0 UIs. Unfortunately, it’s exactly backwards: What he identifies as a problem is in fact not only desirable, but necessary.

• [*] Aside: Jeff, I know you love pictures, but is that particular gratuitous one really necessary? Yes, I know it’s CGI, but it made me really hesitate about linking to your post and has nothing to do with your technical point.

Jeff observes correctly that when you write an application to run on a platform like Windows and/or OS X, your application should follow the local look-and-feel. Fine so far. But he then repeats a claim that I believe is incorrect, at least today, and based on a fallacy — and adds another fallacy:

[Quoting Bill Higgins]

• … a Windows application should look and feel like a Windows application, a Mac application should look and feel like a Mac application, and a web application should look and feel like a web application.

Bill extends this to web applications: a web app that apes the conventions of a desktop application is attempting to cross the uncanny valley of user interface design. This is a bad idea for all the same reasons; the tiny flaws and imperfections of the simulation will be grossly magnified for users.

…

• When you build a “desktop in the web browser”-style application, you’re violating users’ unwritten expectations of how a web application should look and behave.

There are actually two fallacies here.

Fallacy #1: “Look and feel like a web application”

The first fallacy is here:

a web application should look and feel like a web application.

…

violating users’ unwritten expectations of how a web application should look and behave.

These assertions beg the question: What does a web application “look and feel like,” and what do users expect? Also, are you talking about Web 1.0, where there is an answer to these questions, or Web 2.0, where there isn’t?

For Web 1.0 applications, the answer is fairly easy: They look like hyperlinked documents built on technologies like HTML and CSS. That’s what people expect, and get.

But the examples Bill uses aren’t Web 1.0 applications, they’re Web 2.0 applications. For Web 2.0 applications, there are no widely accepted UI standards, and applications are all over the map. Indeed, the whole point of Ajax-y/Web2.0-y applications is to get beyond the current 1.0-standard functionality.

Not only are there are no widely-accepted UI standards, there aren’t even many widely-accepted UI technologies. Consider how many dissimilarities there are among just Flash, Silverlight, and JavaFX as these technologies compete for developer share. Then consider that even within any one of these technologies people actually build wildly diverse interfaces.

Here’s the main example these bloggers used:

Consider the Zimbra web-based email that Bill refers to.

It’s pretty obvious that their inspiration was Microsoft Outlook, a desktop application.

But what’s wrong with Zimbra?

Here’s a Better Question #1: How could you do better?

And for bonus points, Still Better Question #2: What about OWA? Consider that Microsoft already provides essentially the same thing, with the same approach, in the form of Outlook Web Access, which looks remarkably like the usual Outlook [caveat: this is an example of why I write above and below that 'most' Web 2.0 apps don't try to emulate a particular OS look-and-feel; this one does]. For example (a couple of sample shots taken from this brief video overview):

The rich UI isn’t a bug, it really is a feature — a killer feature we’re going to be seeing more of, not less of, because this is what delivering software-as-a-service (SaaS) is all about. Although I use the desktop Outlook most of the time, I like OWA and think it’s the best web-based email and calendaring I’ve seen especially when I’m away from my machine (and I’ve tried several others, though granted you do need to be using Exchange). I suspect its UI conventions are probably pretty accessible even to non-Windows users, though that’s probably debatable and your mileage may vary.

Fallacy #2: “A web app that apes the conventions of a desktop application is [wrong]“

The second fallacy is Jeff’s comment (boldface original):

a web app that apes the conventions of a desktop application is attempting to cross the uncanny valley of user interface design.

I think this is flawed for three reasons.

First, the “uncanny valley” part makes the assumption that people will find it jarring that the app tries to look like a desktop application. (This concern is related to, but different from, the concern of fallacy #1.) But most such apps aren’t doing that at all, because they know their users will access them from PCs, Macs, and lots of different environments, and they have to look reasonable regardless of the user’s native environment. They’re usually not trying to duplicate a given platform’s native look and feel.

Second, what they are doing is borrowing from UI components and conventions that already work well in desktop GUI environments, and are common across many of those environments. When you have no standards for Web 2.0 look and feel, then doing the best you can by borrowing from ideas we already know work pretty well isn’t just okay, it’s necessary. What else can you do?

Finally, the worst part is this: The whole point of SaaS is to deliver desktop-like rich-GUI applications on the web. So what is being labeled ‘wrong’ above is the whole point of what we’re doing as an industry.

“SaaS/Web 2.0 on Web 1.0″: The new “GUI on DOS”

Most SaaS/Web 2.0 applications today look and feel pretty much the way GUI applications looked and felt like on DOS, before technologies like Windows and OS/2 PM existed. Around the late 1980s, people wrote lots of GUI applications that ran on DOS, but we didn’t have a widely-used common GUI infrastructure that handled basic windows and menus and events, much less standards like CUA that tried to say how to use such a common infrastructure if we had it. So they each did their own thing, borrowing where possible from what seemed to work well for GUIs on other platforms.

Twenty years ago, everyone writing GUIs on DOS designed the UIs as best they could, borrowing where possible from what they saw worked on platforms like the Macintosh and Xerox Alto and Star — but the results were all over the map, and would stay that way until a standard environment, followed by standard guidelines, came into being.

Today, everyone writing rich Web 2.0 applications is doing their own thing, borrowing as best they can from Macs and Windows and others — but the results are all over the map, and will continue to be until there actually is such a thing as a UI standard for rich-GUI web applications. You can see that in the differences between Zimbra and Outlook Web Access. In the meantime, it’s not just okay to borrow from what we’ve learned on the desktop; it’s necessary.

And the question isn’t whether metaphors users already understand on the desktop will migrate to the web, but which ones and how soon, because it’s the whole point of SaaS. The industry will soon be going well beyond Google Apps; with offerings like Office Online already announced for the short term, which puts still more rich-client GUI apps like word processors and spreadsheets in the browser (with functionality somewhere between Google Apps and the desktop version of Office).

Zimbra and Outlook Web Access aren’t examples of poor web app design, but exactly the opposite: They’re just the beginning of the next wave of rich web apps.

This month’s Effective Concurrency column is special — it turned into a feature-length article. (I don’t know whether it’ll officially be called a “feature” or a “column” in the print issue.) “Measuring Parallel Performance: Optimizing a Concurrent Queue” just went live on DDJ’s site, and will also appear in the print magazine.

From the article:

How would you write a fast, internally synchronized queue, one that callers can use without any explicit external locking or other synchronization? Let us count the ways…or four of them, at least, and compare their performance. We’ll start with a baseline program and then successively apply three optimization techniques, each time stopping to measure each change’s relative performance for queue items of different sizes to see how much each trick really bought us.

The goal of the article is to see how to measure and understand our code’s parallel performance and measure the actual effect of specific optimizations. Disclaimer: The goal of this article is not to write the fastest possible queue in the world (though it’s pretty good). I’ve already had plenty of email on recent queue-related columns from people who sent me their “faster” implementations; writing lock-free queues seems to be a popular indoor sport. Interestingly, for well of half of the ones I received, a 30-second glance at the code was enough to determine that the code had to be incorrect. Why? Because if it doesn’t do any synchronization on the shared variables — if there aren’t any locks, atomics, fences or other synchronization in the code — then it has races, which will manifest in practice even on forgiving platforms like x86/x64, and there’s no need to look further. (For more details, see the September 2008 column, Lock-Free Code: A False Sense of Security. Even some code submissions I received in response to that very article were broken for the same reasons shown in that article.)

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns and the DDJ print magazine issue in which they first appeared:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

Oops, I just noticed that I forgot to blog about one recent Effective Concurrency column: “Writing Lock-Free Code: A Corrected Queue” which also appeared in the October 2008 print issue of Dr. Dobb’s Journal.

From the article:

As we saw last month [1], lock-free coding is hard even for experts. There, I dissected a published lock-free queue implementation [2] and examined why the code was quite broken. This month, let’s see how to do it right.

Here is the complete-as-of-this-writing set of links to the published Effective Concurrency columns. As always, the months reflect the magazine print issue dates; they usually hit the web a bit sooner:

August 2007: The Pillars of Concurrency

September 2007: How Much Scalability Do You Have or Need?

October 2007: Use Critical Sections (Preferably Locks) to Eliminate Races

November 2007: Apply Critical Sections Consistently

December 2007: Avoid Calling Unknown Code While Inside a Critical Section

January 2007: Use Lock Hierarchies to Avoid Deadlock

February 2008: Break Amdahl’s Law!

March 2008: Going Superlinear

April 2008: Super Linearity and the Bigger Machine

May 2008: Interrupt Politely

June 2008: Maximize Locality, Minimize Contention

July 2008: Choose Concurrency-Friendly Data Structures

August 2008: The Many Faces of Deadlock

September 2008: Lock-Free Code: A False Sense of Security

October 2008: Writing Lock-Free Code: A Corrected Queue

November 2008: Writing a Generalized Concurrent Queue

December 2008: Understanding Parallel Performance

Wow, DDJ just posted the previous one a few days ago, and already the next Effective Concurrency column is available: “Understanding Parallel Performance” just went live, and will also appear in the print magazine.

From the article:

Let’s say that we’ve slickly written our code to apply divide-and-conquer algorithms and concurrent data structures and parallel traversals and all our other cool tricks that make our code wonderfully scalable in theory. Question: How do we know how well we’ve actually succeeded? Do we really know, or did we just try a couple of tests on a quad-core that looked reasonable and call it good? What key factors must we measure to understand our code’s performance, and answer not only whether our code scales, but quantify how well under different circumstances and workloads? What costs of concurrency do we have to take into account?

This month, I’ll summarize some key issues we need to keep in mind to accurately analyze the real performance of our parallel code. I’ll list some basic considerations, and then some common costs. Next month, I have a treat in store: We’ll take some real code and apply these techniques to analyze its performance in detail as we successively apply a number of optimizations and measure how much each one actually buys us, under what conditions and in what directions, and why.

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns (based on the magazine print issue dates):

August 2007: The Pillars of Concurrency

September 2007: How Much Scalability Do You Have or Need?

October 2007: Use Critical Sections (Preferably Locks) to Eliminate Races

November 2007: Apply Critical Sections Consistently

December 2007: Avoid Calling Unknown Code While Inside a Critical Section

January 2007: Use Lock Hierarchies to Avoid Deadlock

February 2008: Break Amdahl’s Law!

March 2008: Going Superlinear

April 2008: Super Linearity and the Bigger Machine

May 2008: Interrupt Politely

June 2008: Maximize Locality, Minimize Contention

July 2008: Choose Concurrency-Friendly Data Structures

August 2008: The Many Faces of Deadlock

September 2008: Lock-Free Code: A False Sense of Security

October 2008: Writing Lock-Free Code: A Corrected Queue

November 2008: Writing a Generalized Concurrent Queue

December 2008: Understanding Parallel Performance

The next Effective Concurrency column, “Writing a Generalized Concurrent Queue”, just went live on DDJ’s site, and also appears in the print magazine.

From the article:

Last month [1], I showed code for a lock-free queue that supported the limited case of exactly two threads—one producer, and one consumer. That’s useful, but maybe not as exciting now that our first rush of lock-free coding glee has worn off. This month, let’s tackle the general problem of supporting multiple producers and multiple consumers with as much concurrency as possible. The code in this article uses four main design techniques: …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns (based on the magazine print issue dates):

August 2007: The Pillars of Concurrency

September 2007: How Much Scalability Do You Have or Need?

October 2007: Use Critical Sections (Preferably Locks) to Eliminate Races

November 2007: Apply Critical Sections Consistently

December 2007: Avoid Calling Unknown Code While Inside a Critical Section

January 2007: Use Lock Hierarchies to Avoid Deadlock

February 2008: Break Amdahl’s Law!

March 2008: Going Superlinear

April 2008: Super Linearity and the Bigger Machine

May 2008: Interrupt Politely

June 2008: Maximize Locality, Minimize Contention

July 2008: Choose Concurrency-Friendly Data Structures

August 2008: The Many Faces of Deadlock

September 2008: Lock-Free Code: A False Sense of Security

October 2008: Writing Lock-Free Code: A Corrected Queue

November 2008: Writing a Generalized Concurrent Queue

The ISO C++ committee met in San Francisco, CA, on September 15-20. You can find the minutes here, including the votes to approve papers.

The most important thing the committee accomplished was this:

Complete C++0x draft published for international ballot

The biggest goal entering this meeting was to make C++0x feature-complete and stay on track to publish a complete public draft of C++0x for international review and comment — in ISO-speak, an official Committee Draft or CD. As I predicted in the summer, the committee achieved that at this meeting. Now the world will know the shape of C++0x in good detail. Here’s where to find it: The September C++0x working draft document is essentially the same as the September 2008 CD.

This is “it”, feature-complete C++0x, including the major feature of “concepts” which had its own extensive set of papers for language and library extensions — I’ll stop there, but there are still more concepts papers at the mailing page and some more still to come during the CD phase. (If you get the impression that concepts is a big feature, well, it is indeed easily the biggest addition we made in C++0x.)

What’s next? As I’ve mentioned before, we’re planning to have two rounds of international comment review. The first of two opportunities for national bodies to give their comments is now underway; the second round will probably be this time next year. The only changes expected to be made between that CD and the final International Standard are bug fixes and clarifications. It’s helpful to think of a CD as a feature-complete beta, and we’re on track to ship one more beta before the full release.

And a new convener

On a personal note, I’m very happy to see this accomplished at the last meeting during my convenership. I’ve now served as secretary and then convener (chair) of the ISO C++ committee for over 10 years, and my second three-year term as convener ended one week after the San Francisco meeting. A decade is enough; I decided not to volunteer for another term as chair.

As of a few weeks ago, P. J. Plauger is the new convener of ISO/IEC JTC1/SC22/WG21 (C++). Many of you will know P.J. (or Bill, as he’s known within the committee) from his long service to the C and C++ communities, including that he has been a past convener of the ISO C standards committee, past editor of the C/C++ Users Journal, the principal author of the Dinkumware implementation of the C++ standard library, and recipient of the 2004 Dr Dobb’s Journal Excellence in Programming Award, among various other qualifications and honors. He has been a regular participant at ISO C++ meetings for about as long as they’ve been held, and his long experience with both the technology and the ISO standards world will serve WG21 well.

I’m very happy to have been able to chair the committee during the development of C++0x. Now as we move from “develop mode” into “ship mode” it will be great to have his experienced hand guiding the committee through the final ISO process. Thanks for volunteering, Bill!

This spring at SD West in Santa Clara, Bjarne and I did a fresh-and-updated S&S event with lots of new material.

We don’t usually repeat the same material, but this time there’s been such demand that we agreed to do a repeat… four weeks from today, in Boston. More information and talk descriptions follow.

CONTENT ADVISORY

Again, usually our S&S events feature mostly new material, but this one is almost identical to the material we did in spring 2008 in Santa Clara. Bjarne is substituting one talk, and will present “C++ in Safety-Critical Systems” instead of his talk on C++’s design and evolution; and we’ll both be updating to the material to reflect the current state of the draft ISO C++0x standard. But otherwise it’ll be identical.

So:

• If you missed our event this spring, here’s your second chance! It was our highest-rated S&S ever, and in the post-conference survey we asked the question “Would you recommend this course to a colleague?” and 100% said yes.
• If you already attended this spring and came to all our sessions, you’ve seen nearly all this material already, but feel free to encourage a friend or colleague to attend who you think would benefit from the material.

The Talks

Wednesday, October 29, 2008

C++0x Overview (Bjarne Stroustrup)

Safe Locking: Best Practices to Eliminate Race Conditions (Herb Sutter)

How to Design Good Interfaces (Bjarne Stroustrup)

Lock-Free Programming in C++—or How to Juggle Razor Blades (Herb Sutter)

Grill the Experts: Ask Us Anything! (Bjarne Stroustrup & Herb Sutter)

Thursday, October 30, 2008

["Best of Stroustrup & Sutter"] Update of talk voted “Most Informative” at S&S 2007: Concepts and Generic Programming in C++0x (Bjarne Stroustrup)

What Not to Code: Avoiding Bad Design Choices and Worse Implementations (Herb Sutter)

C++ in Safety-Critical Systems (Bjarne Stroustrup)

How to Migrate C++ Code to the Manycore “Free Lunch” (Herb Sutter)

Discussion on Questions Raised During the Seminar (Herb Sutter & Bjarne Stroustrup)

Registration

This two-day seminar is getting billing on two different conferences that are running at the same time in the Hynes Convention Center: SD Best Practices and the Embedded Systems Conference. S&S is technically part of both conferences, which means you can attend S&S via either one… either

• register via SDBP, or
• register via ESC

and both ways will include options that include our two-day seminar.

I look forward to seeing many of you in Boston! Best wishes,

Herb

Even genuinely newsworthy topics can get distorted when commentators exaggerate or use data selectively. Here are two recent examples I noticed.

“This is the worst financial crisis since the Great Depression.” It’s true that it’s bad and even historic, and this sound bite correctly doesn’t actually claim it’s as bad as the Depression. I hope it doesn’t turn out to be in the same league as that; then, people were lining up at soup kitchens. For now, however, Apple is still on track to sell 10 million iPhone 3Gs this year, which says something.

“Yesterday [Monday, September 29, 2008] saw the worst single-day plunge in Dow Jones history.” “It’s a new Black Monday.” Well, these got my attention, because I remember Black Monday on October 19, 1987 very well. I was working in IT at a major bank, doing software application support for traders and related departments. When I went up to the trading floor that day, I immediately knew something was badly wrong because of the eerie sound as the elevator doors opened — a sound you never hear during trading hours, and certainly not from a room full of traders standing at their desks: silence.

Yesterday’s loss of 777 points was stunning as the largest single-day point loss in Dow history. But as a percentage loss that’s not even in the top 10 Bad Dow Days, all but two of which occurred before 1935. Those two since the Depression occurred on October 19 and 26, 1987, when the Dow lost 22.6% and then another 8% of its total value in single sessions, respectively #2 and #9 on All-Time Bad Dow Days list. For perspective, as of this writing the Dow is down 19.8% so far this entire year, and it surely hasn’t been a good year. Today’s crisis is already historic and could well get worse yet, of course, but some of us do remember some pretty bad ones in the past.

As good old Sam Clemens said (approximately), there are lies, darned lies, and statistics. Even when the statistics are true, always cross-check them for perspective. Even the best news and the worst news can be overstated, and viewing the same data from multiple angles helps ensure we understand it properly.

[Edited 9/30 to add: At 22.6%, Black Monday in 1987 was actually the worst Dow Day ever if you don't count "reopening days" after unusual market closures when the markets catch up with events that happened while they were closed. So when was the all-time worst Dow Day ever? Perhaps surprisingly, the answer is not in 1929, though several of the all-time top 10 were in that year. Rather, it was December 12, 1914, when the Dow dropped 24.4% after the markets reopened after being closed entirely for over four months due to the outbreak of World War I. (The markets closed on July 30, two days after Austria-Hungary declared war and a day before Germany did.) That helps put in perspective just how bad October 1987 was, and of course that today's crisis is also pretty bad even if it hasn't beaten those prior records, yet.]

A few days ago at UIUC, Ralph Johnson gave a very nice talk on “Parallel Programming Patterns.” It’s now online, and here’s the abstract:

Parallel programming is hard. One proposed solution is to provide a standard set of patterns. Learning the patterns would help people to become expert parallel programmers. The patterns would provide a vocabulary that would let programmers think about their programs at a higher level than the programming language. The patterns could steer programmers away from common errors and towards good design principles.

There have been a number of papers about parallel patterns, and one book Patterns for Parallel Programming. None of them have become popular. I think the problem is that parallel programming is diverse and requires more design expertise than traditional software design. Thus, parallel programming experts use more patterns than parallel programming expert. I’ll critique the existing patterns and explain what I think should be done to make a set of patterns that can be as effective for parallel programming as patterns have been for object-oriented design.

If Johnson’s name sounds familiar, it should: He’s one of the “Gang of Four” authors of the seminal book Design Patterns.

Recommended viewing.

The first offering of the three-day Effective Concurrency course in May went very well. We’re doing it again later this month — this will be the last offering this year.

Here’s the brief information (more details below):

3-Day Seminar: Effective Concurrency

September 22-4, 2008
Bellevue, WA, USA
Developed and taught by Herb Sutter

This course covers the fundamental tools that software developers need to write effective concurrent software for both single-core and multi-core/many-core machines. To use concurrency effectively, we must identify and solve four key challenges:

• Leverage the ability to perform and manage work asynchronously
• Build applications that naturally run faster on new hardware having more and more cores
• Manage shared objects in memory effectively to avoid races and deadlocks
• Engineer specifically for high performance

This seminar will equip attendees to reason correctly about concurrency requirements and tradeoffs, to migrate existing code bases to be concurrency-enabled, and to achieve key success factors for a concurrent programming project. Most code examples in the course can be directly translated to popular platforms and concurrency libraries, including Linux, Windows, Java, .NET, pthreads, and the forthcoming ISO C++0x standard.

Here’s a summary of what we’ll cover during the three days:

Fundamentals

• Define basic concurrency goals and requirements
• Understand applications’ scalability needs
• Key concurrency patterns

Isolation: Keep Work Separate

• Running tasks in isolation and communicate via async messages
• Integrating multiple messaging systems, including GUIs and sockets
• Building responsive applications using background workers
• Threads vs. thread pools

Scalability: Re-enable the Free Lunch

• When and how to use more cores 
• Exploiting parallelism in algorithms 
• Exploiting parallelism in data structures 
• Breaking the scalability barrier

Consistency: Don’t Corrupt Shared State

• The many pitfalls of locks–deadlock, convoys, etc.
• Locking best practices
• Reducing the need for locking shared data
• Safe lock-free coding patterns
• Avoiding the pitfalls of general lock-free coding
• Races and race-related effects

Migrating Existing Code Bases to Use Concurrency

Near-Future Tools and Features

High Performance Concurrency

• Machine architecture and concurrency
• Costs of fundamental operations, including locks, context switches, and system calls
• Memory and cache effects
• Data structures that support and undermine concurrency
• Enabling linear and superlinear scaling

I hope to get to meet some of you in the Seattle area!

The adage, quoted again this week by Bruce Schneier:

The plural of “anecdote” is not “data.”

But lest we enshrine raw data as holy in itself, another perspective:

And the plural of “datum” is not “proof.”

Last month, I was privileged to be part of a panel in a webinar on Embedded Multicore Development moderated by Richard Nass, Editor-in-Chief of Embedded Systems Design, Embedded.com, and the Embedded Systems Conferences. It’s online and available on demand.

I say “privileged” especially because of the stature of the other panelists. These distinguished gentlemen were:

• Edward A. Lee, Professor and Chair of the EE&CS department at U.C. Berkeley, and known among other things for “The Problem with Threads”.
• James Reinders, the chief concurrency evangelist at Intel and author of the Threading Building Blocks book.
• James Truchard, the co-founder, President and CEO of National Instruments.

It was a fun discussion, and I hope informative.

[Lock-free code is] hard even for experts. It’s easy to write lock-free code that appears to work, but it’s very difficult to write lock-free code that is correct and performs well. Even good magazines and refereed journals have published a substantial amount of lock-free code that was actually broken in subtle ways and needed correction.

To illustrate, let’s dissect some peer-reviewed lock-free code that was published here in DDJ just two months ago …

Today I received an email that asked:

I have recently come across your excellent articles on concurrency and the changes in software writing paradigm. They make a lot of sense, but I am having trouble translating them to my world of Telecom oriented web services, where practically everything is run through a DBMS. It seems to me we get everything “free”, simply by using an inherently concurrent multi-everything beast such as that :-) .

Could you please share your thoughts on the issue in one of your coming blog entries? It seems to me nowadays most complex systems would take advantage of a DBMS, certainly any application that is internet based, telecom oriented, or enterprise level. Be it in C++, Java, or PHP and its ilk, using a DBMS – often as a sort of message queue – is one of the best practices that ensures parallelism.

Sure. At right is a slide I give in talks that summarizes the answer to this question, and I’ve addressed this and other similar issues in an ACM Queue article.

The problem with taking advantage of multicore/manycore hardware isn’t (as much) on the server, it’s on the client. When experienced people say things like, “but the concurrency problem is already solved, we’ve been building scalable software for years,” that’s server or niche-client application people talking. That kind of laid-back sound bite sure isn’t coming from mainstream client application developers.

Typical Server Workloads

On the server:

• Workloads typically already have lots of inherent concurrency (000s or 000,000s of incoming requests for web/dbms/etc. operations), and it’s easy to launch independent requests concurrently.
• Shared data is typically inside highly structured relational databases where we have decades of experience with automatic concurrency control. The DBMS itself knows how to optimistically run transactions in parallel and control conflicts by escalating row locks to page locks, index locks to table locks, and so on.
• The programming model is typically transactions, and all the programmer has to know about is to write “begin transaction; /* … read/write whatever stuff you feel the need to, then … */ end transaction;” which is about as sweet as it gets.

We already know how to build somewhat scalable server apps. Sure, it’s still rocket science and takes expert knowledge to do well. But we generally know the rocket science, have experts who can implement it with repeatable success, and have regularly scheduled missions to the “scalable servers” space station. With some care, we can say that the “concurrency problem is already solved” here.

Typical Client Workloads

The world is very different for typical mainstream client applications (i.e., I’m not talking about Photoshop and a handful of others), where:

• Workloads don’t have lots of inherent concurrency. The user clicks one button, and we have to figure out how to divide the work and recombine it in order to get the answer faster on many cores.
• Shared data is typically in unstructured pointer-chasing graphs of objects in shared memory that require explicit concurrency control. Note that “unstructured” doesn’t mean there’s no structure — of course there’s some — but it’s a gloriously diverse pile of objects and containers, and more like an organically growing shantytown with unplanned twisty little alleys and passages, than the nice rectangular downtown city street plan of a nice rectangular database table.
• The programming model to protect shared data is to use error-prone explicit locks. You have to remember which locks protect which data, and not to acquire them in inconsistent and deadlock-prone orders.

We’re still discovering and productizing the rocket science here. You could say that the tools like OpenMP that we do have now are still at the V-2 stage — they have limited applicability, are somewhat fussy, and don’t always land where you aim them.

But we’re working on it. Up-and-coming tools like Threading Building Blocks are like the Mercury and Venera missions, setting out to reach successively higher goals and repeatability… and we’re starting to see what are perhaps Apollo- and ISS-class missions in the form of PLINQ, the Task Parallel Library, and one for native C++ we’ll be announcing in October at PDC. In part, these tools are trying to see how much we can make client workloads look more like server workloads, notably in providing a transaction-oriented programming model. For example, transactional memory is an area of active research that would let us write “begin transaction; /* … read/write whatever memory variables you feel the need to, then … */ end transaction;”, and if successful it could eventually replace many or even most existing uses of locks.

We have rightly celebrated some successful ‘manned’ flights with client products like Photoshop (parallel rendering) and Excel (parallel recalc) that scale to a number of cores. We’re on the road to, but still working toward, establishing the infrastructure and technology base to enable regularly scheduled commercial flights/shipments of scalable client applications that “light up” on multicore/manycore machines.

… That’s the classic deadlock example from college. Of course, two isn’t a magic number. An improved definition of deadlock is: “When N threads enter a locking cycle where each tries to take a lock the next already holds.”

“But wait,” someone might say. “I once had a deadlock just like the code you just showed, but it didn’t involve locks at all—it involved messages.” …

I just received the following question, whose answer is the same in C++, C#, and Java.

Question: In the following code, why isn’t the destructor/disposer ever called to clean up the Widget when the constructor emits an exception? You can entertain this question in your mainstream language of choice:

Interestingly, we can answer this even without knowing the calling code… but there is typical calling code that is similar in all three languages that reinforces the answer.

Construction and Lifetime

The fundamental principles behind the answer is the same in C++, C#, and Java:

1. A constructor conceptually turns a suitably sized chunk of raw memory into an object that obeys its invariants. An object’s lifetime doesn’t begin until its constructor completes successfully. If a constructor ends by throwing an exception, that means it never finished creating the object and setting up its invariants — and at the point the exceptional constructor exits, the object not only doesn’t exist, but never existed.
2. A destructor/disposer conceptually turns an object back into raw memory. Therefore, just like all other nonprivate methods, destructors/disposers assume as a precondition that “this” object is actually a valid object and that its invariants hold. Hence, destructors/disposers only run on successfully constructed objects.

I’ve covered some of these concepts and consequences before in GotW #66, “Constructor Failures,” which appeared in updated and expanded form as Items 17 and 18 of More Exceptional C++.

In particular, if Gadget’s constructor throws, it means that the Gadget object wasn’t created and never existed. So there’s nothing to destroy or dispose: The destructor/disposer not only isn’t needed to run, but it can’t run because it doesn’t have a valid object to run against.

Incidentally, it also means that the Gadget constructor isn’t exception-safe, because it and only it can clean up resources it might leak. In the C++ version, the usual simple way to write the code correctly is to change the w member’s type from Widget* to shared_ptr<Widget> or an equivalent smart pointer type that owns the object. But let’s leave that aside for now to explore the more general issue.

A Look At the Calling Code

Next, let’s see how these semantics are actually enforced, whether by language rules or by convention, on the calling side in each language. Here are the major idiomatic ways in each language to use an Gadget object in an exception-safe way:

Consider the two cases where an exception can occur:

• What if an exception is thrown while using myGadget – that is, during “do something with myGadget”? In all cases, we know that myGadget’s destructor/dispose method is guaranteed to be called.
• But what if an exception is thrown while constructing myGadget? Now in all cases the answer is that the destructor/dispose method is guaranteed not to be called.

Put another way, we can say for all cases: The destructor/dispose is guaranteed to be run if and only if the constructor completed successfully.

Another Look At the Destructor/Dispose Code

Finally, let’s return to each key line of code one more time:

The motivation for the nullness tests in the original example was to clean up partly-constructed objects. That motivation is suspect in principle — it means the constructors aren’t exception-safe because only they can clean up after themselves — and as we’ve seen it’s flawed in practice because the destructors/disposers won’t ever run on the code paths that the original motivation cared about. So we don’t need the nullness tests for that reason, although you might still have nullness tests in destructors/disposers to handle ‘optional’ parts of an object where a valid object might hold a null pointer or reference member during its lifetime.

In this particular example, we can observe that the nullness tests are actually unnecessary, because w will always be non-null if the object was constructed successfully. There is no (legitimate) way that you can call the destructor/disposer (Furthermore, C++ developers will know that the test is unnecessary for a second reason: Delete is a no-op if the pointer passed to it is null, so there’s never a need to check for that special case.)

Conclusion

When it comes to object lifetimes, all OO languages are more alike than different. Object and resource lifetime matters, whether or not you have a managed language, garbage collection (finalizers are not destructors/disposers!), templates, generics, or any other fancy bell or whistle layered on top of the basic humble class.

The same basic issues arise everywhere… the code you write to deal with them is just spelled using different language features and/or idioms in the different languages, that’s all. We can have lively discussions about which language offers the easiest/best/greenest/lowest-carbon-footprint way to express how we want to deal with a particular aspect of object construction, lifetime, and teardown, but those concerns are general and fundamental no matter which favorite language you’re using to write your code this week.

This story has been picked up semi-widely since last night. I’m sure this Steven Prentice they quote is a fine (Gartner) Fellow, but really:

The computer mouse is set to die out in the next five years and will be usurped by touch screens and facial recognition, analysts believe.

Seriously, does anyone who uses computers daily really believe this kind of prediction just because someone at Gartner says so? Dude, sanity check: 1. What functions do you use your mouse for? 2. How many of those functions can be done by pointing at your screen or smiling at the camera: a) at all; and b) with equivalent high precision and low arm fatigue? Of course the mouse, including direct equivalents like the touchpad/trackpad, will be replaced someday. But to notice that people like to turn and shake their Wii controllers and iPhones and then make the leap to conclude that this will replace mice outright in the short term seems pretty thin even for Gartner.

When you read a report from Gartner, Forrester, IDC and their brethren research firms, remember that you’re either getting real-world data (aka research) or a single analyst’s personal predictions (aka crystal-ball gazing). Research firms are good at what they’re good at, namely research:

• They’re “decent” at compiling current industry market data. Grade: A.
• They’re “pretty okay” when they limit themselves to simple short-term extrapolation of that data, such as two-year projections of cost changes of high-speed networking in Canada or cell phone penetration in India. Grade: A-.

But when they try bigger technology movement predictions like “X will replace Y in Z years” they average somewhere around “spotty,” and on their off days they dip down into “I think you forgot to sanity check that sound bite” territory. It’s a pity that some venture capitalists take the research analysts’ word as gospel. Reliability of technology shift predictions: D+.

Two weeks ago, I broke down and bought a Kindle. I like it:

• It’s a good and well-designed reader, and the experience is much better than the other e-book reading I’ve done before on phones and PDAs. I like how you when you bookmark a page, you can see it… the corner of the page gets a little dog-ear. [1]
• It’s got a nice e-paper screen that uses ambient light, not backlight, which makes it readable anywhere just like a printed page — it’s even better, not worse, in direct sunlight.
• It’s light and thin and sturdy. Sure beats carrying three or four books on a trip.
• It has great battery life. I’ve only charged it once so far, when I first received it… since then I’ve had it for 11 days and read a full book and a half, and it still has 75% of its first charge left. (It helps that I turn the wireless off unless I’m actively using it.)
• Fast, free wireless everywhere in the U.S., without computers or WiFi.

But today, it transformed my reading experience.

This morning, I was unsuspectingly reading my feeds in Google Reader as usual, blissfully unaware that the way I read books was about to change. Among other articles, I noticed that Slashdot ran a book review of Inside Steve’s Brain (that’s Jobs, not Wozniak or Ballmer). The review made me want to read the book. That’s when the new reality started, because I was interested in the book now, and had time to start reading it now:

• Normally, I would have ordered it from Amazon and waited for it to arrive. But what usually happens then is that the book arrives a week later, and when it gets here I don’t have time to start it right away or I don’t quite feel like that kind of book just at the moment… and it goes on a shelf, with a 70% probably of being picked up at some point in the future.
• Today, I searched for the book title on my Kindle, clicked “Buy”, and a few minutes later started reading Inside Steve’s Brain while eating lunch. [2]

That convenience isn’t merely instant gratification, it’s transformative. I suspect I’m going to be buying reading even more books now, even though I have a few little nits with the device, such as that the next and previous page buttons are a little too easy to press in some positions.

In other news, the Kindle also supports reading blogs and newspapers and some web surfing, but those are less compelling for me because I tend to do those things in the context of work, which means I’m already sitting at a computer with a bigger color screen and full keyboard. Maybe someday I’ll do it on e-paper. Until then, just living inside a virtual bookstore is plenty for me. Kindle + the Amazon Kindle store = iPod + iTunes for books. [3]

Here’s a useful summary article on Kindle features from a prospective user’s point of view.

Notes

1. The first two books I downloaded? The Design of Everyday Things, which was interestingly apropros to read on a new device like the Kindle with its nice dog-ear feedback and other well-designed features, and Foundation, which I hadn’t read in ages.

2. And it cost less than half what the dead-tree version would (though the latter was hardcover).

3. Caveat: I’m not actually an iPod owner, and I hate how Apple keeps insisting on installing iTunes on my computer just because I have Safari or QuickTime installed (mutter grumble evil product-tying monopolies mutter grumble :-) ). But apparently everyone else loves them, and they have indeed changed their industry.

A commenter asked:

thread_local X tlsX; ??

Herb, I hope you aren’t backtracking on Hungarian Notation now that you work for Microsoft. Say it aint so…

It ain’t so. Besides, Microsoft’s Framework Developer’s Guide prominently intones: “Do not use Hungarian notation.”

Warts like “tls” and “i” are about lifetime and usage, not type. Here “tls” denotes that each thread gets its own copy of the value that is constructed and destroyed once per thread that uses it (lifetime) and doesn’t need to be synchronized using mutexes (usage). As another example of usage warts, I’ll also use “i” for a variable that’s used as an index or for iteration — and given that the “i” convention goes back to before BASIC, we shouldn’t try to pin the Hungarian tail on that donkey.

Having said that, though, many people have variously decried and defended different forms of Hungarian, and you may notice a pattern… they’re mostly:

• decrying Systems Hungarian, the wildly popularized version which focuses on encoding the type, and
• defending Applications Hungarian, the originally described version which focuses on encoding the usage.

Today, “Hungarian” nearly always means Systems Hungarian. The main trouble with Systems Hungarian comes from trying to embed information about a variable’s type into the variable’s name by prepending an encoded wart like the venerable sz, pach, ul, and their ilk. Although potentially helpful in a weakly-typed language like C, that’s known to be brittle and the prefixes tend to turn into lies as variable types morph during maintenance. The warting systems also don’t extend well to user-defined types and templates.

I’ve railed against the limitations and perils of Hungarian with Andrei Alexandrescu in our book C++ Coding Standards, where I made sure it was stated right up front as part of “Item 0: Don’t sweat the small stuff (or, Know what not to standardize)”:

Example 3: Hungarian notation. Notations that incorporate type information in variable names have mixed utility in type-unsafe languages (notably C), are possible but have no benefits (only drawbacks) in object-oriented languages, and are impossible in generic programming. Therefore, no C++ coding standard should require Hungarian notation, though a C++ coding standard might legitimately choose to ban it.

and with Jim Hyslop in our article “Hungarian wartHogs”:

“… the compiler already knows much more than you do about an object’s type. Changing the variable’s name to embed type information adds little value and is in fact brittle. And if there ever was reason to use some Hungarian notation in C-style languages, which is debatable, there certainly remains no value when using type-safe languages.”

There’s an amusing real-world note later in that article. I’ll pick up where the Guru is saying:

“I recall only one time when Hungarian notation was useful on a project. … One of the programmers on the project was named Paul,” the Guru explained. “Several months into the project, while still struggling to grow a ponytail and build his report-writing module, he pointed out that Hungarian notation had helped him find a sense of identity, for he now knew what he was…” She paused.

I blinked. It took me about ten seconds, and then I shut my eyes and grimaced painfully. “Pointer to array of unsigned long,” I groaned.

She smiled, enjoying my pain. “True story,” she said.

It is indeed a true story. I worked with him in 1993. The joke was bad then, too, but at least using Hungarian was more defensible because the project’s code was written in C. I found it handy at the time, but now that I work in more strongly typed languages I find the type-embedding version of Hungarian more actively harmful than actually useful.

You might call me a Hungarian emigrant, now living happily as an expat out in eastern C++ near the Isle of Managed Languages.

The ISO C++ committee met in Sophia Antipolis, France on June 6-14. You can find the minutes here (note that these cover only the whole-group sessions, not the breakout technical sessions where we spend most of the week).

Here’s a summary of what we did, with links to the relevant papers to read for more details, and information about upcoming meetings.

Highlights: Complete C++0x draft coming in September

The biggest goal entering this meeting was to make C++0x feature-complete and stay on track to publish a complete public draft of C++0x this September for international review and comment — in ISO-speak, an official Committee Draft or CD. We are going to achieve that, so the world will know the shape of C++0x in good detail this fall.

We’re also now planning to have two rounds of international comment review instead of just one, to give the world a good look at the standard and two opportunities for national bodies to give their comments, the first round starting after our September 2008 meeting and the second round probably a year later. However, the September 2008 CD is “it”, feature-complete C++0x. The only changes expected to be made between that CD and the final International Standard are bug fixes and clarifications. It’s helpful to think of a CD as a feature-complete beta.

Coming into the June meeting, we already had a nearly-complete C++0x internal working draft — most features that will be part of C++0x had already been “checked in.” Only a few were still waiting to become stable enough to vote in, including initializer lists, range-based for loops, and concepts.

Of these, concepts is the long-pole feature for C++0x, which isn’t surprising given that it’s the biggest new language feature we’re adding to Standard C++. A primary goal of the June meeting, therefore, was to make as much progress on concepts as possible, and to see if it would be possible to vote that feature into the C++0x working draft at this meeting. We almost did that, thanks to a lot of work not only in France but also at smaller meetings throughout the winter and spring: For the first time, we ended a meeting with no known issues or controversial points in the concepts standardese wording, and we expect to “check in” concepts into the working draft at the next meeting in September, which both cleared the way for us to publish a complete draft then and motivated the plan to do two rounds of public review rather than one, just to make sure the standard got enough “bake time” in its complete form.

Next, I’ll summarize some of the major features voted into the draft at the June meeting.

Initializer lists

C++0x initializer lists accomplish two main objectives:

• Uniformity: They provide a uniform initialization syntax you can use consistently everywhere, which is especially helpful when you write templates.
• Convenience: They provide a general-purpose way of using the C initializer syntax for all sorts of types, notably containers.

See N2215 for more on the motivation and original design, and N2672 and N2679 for the final standardese.

Example: Initializing aggregates vs. classes

It’s convenient that we can initialize aggregates like this:

but the syntax is slightly different for classes with constructors:

In C++0x, you can still do all of the above, but initializer lists give us a regular way to initialize all kinds of types:

Having a uniform initialization syntax is particularly helpful when writing template code, so that the template can easily work with a wide variety of types.

Example: Initializing arrays vs. containers

One place where the lack of uniform initialization has been particularly annoying — at least to me, when I write test harnesses to exercise the code I show in articles and talks — is when initializing a container with some default values. Don’t you hate it when you want to create a container initialized to some known values, and if it were an array you can just write:

but if it’s a container like a vector, you have to default-construct the container and then push every entry onto it individually:

or, even more embarrassingly, initialize an array first for convenience and then construct the vector as a copy of the array, using the vector constructor that takes a range as an iterator pair:

Arrays are weaker than containers in nearly every other way, so it’s annoying that they get this unique convenience just because of their having been built into the language since the early days of C.

The lack of convenient initialization has been even more irritating with maps:

In C++0x, we can initialize any container with known values as conveniently as arrays:

As a bonus, a uniform initialization syntax even makes arrays easier to deal with. For example, the array initializer syntax didn’t support arrays that are dynamically allocated or class members arrays:

In C++0x, the array initialization syntax is uniformly available in these cases too:

More concurrency support

Last October, we already voted in a state-of-the-art memory model, atomic operations, and a threading package. That covered the major things we wanted to see in this standard, but a few more were still in progress. Here are the major changes we made this time that relate to concurrency.

Thread-local storage (N2659): To declare a variable which will be instantiated once for each thread, use the thread_local storage class. For example:

Dynamic initialization and destruction with concurrency (N2660) handles two major cases:

• Static and global variables can be concurrently initialized and destroyed if you try to access them on multiple threads before main() begins. If more than one thread could initialize (or use) the variable concurrency, however, it’s up to you to synchronize access.
• Function-local static variables will have their initialization automatically protected; while one thread is initializing the variable, any other threads that enter the function and reach the variable’s declaration will wait for the initialization to complete before they continue on. Therefore you don’t need to guard initialization races or initialization-use races, and if the variable is immutable once constructed then you don’t need to do any synchronization at all to use it safely. If the variable can be written to after construction, you do still need to make sure you synchronize those post-initialization use-use races on the variable.

Thread safety guarantees for the standard library (N2669): The upshot is that the answer to questions like “what do I need to do to use a vector<T> v or a shared_ptr<U> sp thread-safely?” is now explicitly “same as any other object: if you know that an object like v or sp is shared, you must synchronize access to it.” Only a very few objects need to guarantee internal synchronization; the global allocator is one of those, though, and so it gets that stronger guarantee.

The “other features” section below also includes a few smaller concurrency-related items.

More STL algorithms (N2666)

We now have the following new STL algorithms. Some of them fill holes (e.g., “why isn’t there a copy_if or a counted version of algorithm X?”) and others provide handy extensions (e.g., “why isn’t there an all_of or any_of?”).

• all_of( first, last, pred )
  returns true iff all elements in the range satisfy pred
• any_of( first, last, pred )
  returns true iff any element in the range satisfies pred
• copy_if( first, last result, pred)
  the “why isn’t this in the standard?” poster child algorithm
• copy_n( first, n result)
  copy for a known number of n elements
• find_if_not( first, last, pred )
  returns an iterator to the first element that does not satisfy pred
• iota( first, last, value )
  for each element in the range, assigns value and increments value “as if by ++value”
• is_partitioned( first, last, pred )
  returns true iff the range is already partitioned by pred; that is, all elements that satisfy pred appear before those that don’t
• none_of( first, last, pred )
  returns true iff none of the elements in the range satisfies pred
• partition_copy( first, last, out_true, out_false, pred )
  copy the elements that satisfy pred to out_true, the others to out_false
• partition_point( first, last, pred)
  assuming the range is already partitioned by pred (see is_partitioned above), returns an iterator to the first element that doesn’t satisfy pred
• uninitialized_copy_n( first, n, result )
  uninitialized_copy for a known number of n elements

Other approved features

• N2435 Explicit bool for smart pointers
• N2514 Implicit conversion operators for atomics
• N2657 Local and unnamed types as template arguments
• N2658 Constness of lambda functions
• N2667 Reserved namespaces for POSIX
• N2670 Minimal support for garbage collection and reachability-based leak detection 
• N2661 Time and duration support
• N2674 shared_ptr atomic access
• N2678 Error handling specification for Chapter 30 (Threads)
• N2680 Placement insert for standard containers

Next Meeting

Although we just got back from France, the next meeting of the ISO C++ standards committee is already coming up fast:

• September 14-20, 2008: San Francisco, California, USA

The meetings are public, and if you’re in the area please feel free to drop by.

What is a high-performance data structure? To answer that question, we’re used to applying normal considerations like Big-Oh complexity, and memory overhead, locality, and traversal order. All of those apply to both sequential and concurrent software.

But in concurrent code, we need to consider two additional things to help us pick a data structure that is also sufficiently concurrency-friendly:

• In parallel code, your performance needs likely include the ability to allow multiple threads to use the data at the same time. If this is (or may become) a high-contention data structure, does it allow for concurrent readers and/or writers in different parts of the data structure at the same time? If the answer is, “No,” then you may be designing an inherent bottleneck into your system and be just asking for lock convoys as threads wait, only one being able to use the data structure at a time.
• On parallel hardware, you may also care about minimizing the cost of memory synchronization. When one thread updates one part of the data structure, how much memory needs to be moved to make the change visible to another thread? If the answer is, “More than just the part that has ostensibly changed,” then again you’re asking for a potential performance penalty, this time due to cache sloshing as more data has to move from the core that performed the update to the core that is reading the result.

It turns out that both of these answers are directly influenced by whether the data structure allows truly localized updates. …

From the ancient dramatist Seneca the Younger:

“Our plans miscarry because they have no aim. When a man does not know what harbor he is making for, no wind is the right wind.”

And from the Bard, not to be outdone in metaphors of ships and seas:

“There is a tide in the affairs of men,
Which, taken at the flood, leads on to fortune;
Omitted, all the voyage of their life
Is bound in shallows and miseries.
We must take the current when it serves,
Or lose our ventures.”

[6/25: Added YouTube availability and notes.]

A few weeks ago, the BBC was in town to tape a special interview/documentary on Bill Gates. As part of the footage they got, there’s a Bill-in-a-technical-review-meeting shot that includes yours truly at a whiteboard presenting an overview-plus-drilldown on C++0x lambda functions. It was a good review; Bill’s a sharp guy with a broad and deep background and incisive questions.

Here’s the trailer and the program [YouTube]. Some bits of our VC++ team review start around 4:30 of part one and 8:05 of part two, with a few other shots later on. If you watch really closely, you can see a brief flash of a guy in a blue shirt at a whiteboard with code on it in HerbWhiteboardScrawl Sans 60pt, and then Bill gesturing and making comments about simulating lambdas using macros instead of baking them into the language, and variable capture consequences.

The BBC documentary “How a Geek Changed the World” first aired today in the UK on BBC2; I’m guessing it will repeat. I don’t know when it will air in the U.S, but as of this writing it’s on YouTube.

Jeff Atwood just wrote a nice piece on why type inference is convenient, using a C# sample:

I was absolutely thrilled to be able to refactor this code:

StringBuilder sb = new StringBuilder(256);
UTF8Encoding e = new UTF8Encoding();
MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();

Into this:

var sb = new StringBuilder(256);
var e = new UTF8Encoding();
var md5 = new MD5CryptoServiceProvider();

It’s not dynamic typing, per se; C# is still very much a statically typed language. It’s more of a compiler trick, a baby step toward a world of Static Typing Where Possible, and Dynamic Typing When Needed.

It’s worth making a stronger demarcation among:

• type inference, which you can do in any language
• static vs. dynamic typing, which is completely orthogonal but all too often confused with inference
• strong vs. weak typing, which is mostly orthogonal (e.g., C is statically typed because every variable has a statically known actual type, but also weakly typed because of its casts)

Above, Jeff explicitly separates inference and dynamic-ness. Unfortunately, later on he proceeds to imply that inference is a small step toward dynamic typing, which is stylistically true in principle but might mislead some readers into thinking inference has something to do with dynamic-ness, which it doesn’t.

Type Inference

Many languages, including C# (as shown above) and the next C++ standard (C++0x, shown below), provide type inference. C++0x does it via the repurposed auto keyword. For example, say you have an object m of type map<int,list<string>>, and you want to create an iterator to it:

How many times have you said to your compiler, “Compiler, you know the type already, why are you making me repeat it?!” Even the IDE can tell you what the type is when you hover over an expression.

Well, in C++0x you won’t have to any more, which is often niftily convenient. This gets increasingly important as we don’t want to, or can’t, write out the type ourselves, because we have:

• types with more complicated names
• types without names (or hard-to-find names)
• types held most conveniently via an indirection

In particular, consider that C++0x lambda functions generate a function object whose type you generally can’t spell, so if you want to hold that function object and don’t have auto then you generally have to use an indirection:

Note that the last line above is more efficient than the C equivalent using a pointer to function, because C++ lets you inline everything. For more on this, see Item 46 in Scott Meyers’ Effective STL on why it’s preferable to use function objects rather than functions, because (counterintuitively) they’re more efficient.

Now, though there’s no question auto and var are great, there are some minor limitations. In particular, you may not want the exact type, but another type that can be converted to:

So C++0x auto (like C# var) only gets you the most obvious type. Still and all, that does cover a lot of the cases.

The important thing to note in all of the above examples is that, regardless how you spell it, every variable has a clear, unambiguous, well-known and predictable static type. C++0x auto and C# var are purely notational conveniences that save us from having to spell it out in many cases, but the variable still has one fixed and static type.

Static and Dynamic Typing

As Jeff correctly noted in the above-quoted part, this isn’t dynamic typing, which permits the same variable to actually have different types at different points in its lifetime. Unfortunately, he goes on to say the following that could be mistaken by some readers to imply otherwise:

You might even say implicit variable typing is a gateway drug to more dynamically typed languages.

I know Jeff knows what he’s talking about because he said it correctly earlier in the same post, but let’s be clear: Inference doesn’t have anything to do with dynamic typing. Jeff is just noting that inference just happens to let you declare variables in a style that can be similar to the way you do it all the time in a dynamically typed language. (Before I could post this, I see that Lambda the Ultimate also commented on this confusion. At least one commenter noted that this could be equally viewed as a gateway drug to statically typed languages, because you can get the notational convenience without abandoning static typing.)

Quoting from Bjarne’s glossary:

dynamic type – the type of an object as determined at run-time; e.g. using dynamic_cast or typeid. Also known as most-derived type.

static type – the type of an object as known to the compiler based on its declaration. See also: dynamic type.

Let’s revisit an earlier C++ example again, which shows the difference between a variable’s static type and dynamic type:

The static type of the variable says what interface it supports, so in this case wb allows you to access only the members of WidgetBase. The dynamic type of the variable is what the object being pointed to right now is.

In dynamically typed languages, however, variables don’t have a static type and you generally don’t have to mention the type. In many dynamic languages, you don’t even have to declare variables. For example:

Boost’s variant and any

There are two popular ways to get this effect in C++, even though the language remains statically typed. The first is Boost variant:

Unlike a union, a variant can include essentially any kind of type, but you have to say what the legal types are up front. You can even simulate getting overload resolution via boost::apply_visitor, which is checked statically (at compile time).

The second is Boost any:

Again unlike a union, an any can include essentially any kind of type. Unlike variant, however, any doesn’t make (or let) you say what the legal types are up front, which can be good or bad depending how relaxed you want your typing to be. Also, any doesn’t have a way to simulate overload resolution, and it always requires heap storage for the contained object.

Interestingly, this shows how C++ is well and firmly (and let’s not forget efficiently) on the path of Static Typing Where Possible, and Dynamic Typing When Needed.

Use variant when:

• You want an object that holds a value of one of a specific set of types.
• You want compile-time checked visitation.
• You want the efficiency of stack-based storage where possible scheme (avoiding the overhead of dynamic allocation).
• You can live with horrible error messages when you don’t type it exactly right.

Use any when:

• You want the flexibility of having an object that can hold a value of virtually “any” type.
• You want the flexibility of any_cast.
• You want the no-throw exception safety guarantee for swap.

While Bjarne and I were at SD for S&S, we took time out to do an interview together with Ted Neward for InformIT. I just got word that it went live… here are the links.

On the OnSoftware – Video (RSS):

• OnSoftware – Bjarne Stroustrup & Herb Sutter on the Future of C++ – Part 1
• OnSoftware – Bjarne Stroustrup & Herb Sutter on the Future of C++ – Part 2

On the OnSoftware – Audio (RSS):

• OnSoftware – Bjarne Stroustrup & Herb Sutter on the Future of C++ – Part 1
• OnSoftware – Bjarne Stroustrup & Herb Sutter on the Future of C++ – Part 2

Enjoy!

I’ll be giving a memory model talk at Gamefest in Seattle next month. Here’s a quick summary:

Memory Models: Foundational Knowledge for Concurrent Code
July 22-23, 2008
Gamefest 2008
Seattle, WA, USA

A memory model defines a contract between the programmer and the execution environment, that trades off:

• programmability via stronger guarantees for programmers, vs.
• performance via greater flexibility for reordering program memory operations.

The “execution environment” includes everything from the compiler and optimizer on down to the CPU and cache hardware, and it really wants to help you by reordering your program to make it run faster. You, on the other hand, you want it to not help you excessively in ways that will break the meaning of your code. In this talk, we’ll consider why a memory model is important, how to achieve a reasonable balance, detailed considerations on current and future PC and Xbox platforms, and some best practices for writing solid concurrent code.

Want to kill your parallel application’s scalability? Easy: Just add a dash of contention.

Locality is no longer just about fitting well into cache and RAM, but also about avoiding scalability busters by keeping tightly coupled data physically close together and separately used data far, far apart. …