The ISO C++ committee met in Santa Cruz, CA, USA on October 19-24. You can find the minutes here, which include the votes at the whole-group sessions but not the details of the breakout technical sessions where we spend most of the week.

The good news is that there’s little new technical news. We did a lot of work during the week, but it was mostly working on refining the standard, deciding integration questions of how two language features should work together in cases not clearly described, fixing bugs, and answering national body comments on our first public draft last fall (those are now nearly all answered). We expect to produce another public draft at our next meeting in March.

We did vote in one small feature that I and Lawrence Crowl in particular had been working on: a simple async() facility to launch asynchronous work easily without messing with packaged_tasks and raw threads. Here’s a sample use, also demonstrating a simple use of the futures library and a lambda function for kicks:

  future f = std::async( []{ OtherWork(); } );

  //... do our own work concurrently with OtherWork ...

  OkayNowWeNeedTheResult( f.get() );  // blocks if necessary until f is ready

If you’ve been following the futures library, you’ll notice a name change above: We also renamed unique_future<T> to just plain future<T> as part of recasting the futures wording to make it clearer and more consistent. That’s an example of the kind of cleanup work being done.

Near the end of the meeting, we also discussed deprecating export (as I reported earlier) and exception specifications other than throw()-nothing. There seemed to be significant support for deprecating both, and so we’ll probably see a concrete proposal at our next meeting.

In sad news, the convener (chair) of the committee for the past year, P.J. Plauger, stepped down at the end of the meeting. After I had been the convener for two three-year terms from 2002 to 2008, I decided it was time for someone else to have a go and so Plauger replaced me a year ago. He has done a really great job over the past year and his contributions in that role will be missed, but we won’t lose his services entirely as he remains an active participant in the committee. I will probably volunteer again to replace him.

That’s pretty much it. The next meeting of the ISO C++ standards committee is in March:

• March 8-13, 2010: Pittsburgh, PA, USA

(Edited to fix “2009” in the title and add a link to the Pittsburgh meeting invitation.)

This month’s Effective Concurrency column, Prefer structured lifetimes – local, nested, bounded, deterministic, is now live on DDJ’s website.

From the article:

Where possible, prefer structured lifetimes: ones that are local, nested, bounded, and deterministic. This is true no matter what kind of lifetime we’re considering, including object lifetimes, thread or task lifetimes, lock lifetimes, or any other kind. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

Use Thread Pools Correctly: Keep Tasks Short and Nonblocking (Apr 2009)

Eliminate False Sharing (May 2009)

Break Up and Interleave Work to Keep Threads Responsive (Jun 2009)

The Power of “In Progress” (Jul 2009)

Design for Manycore Systems (Aug 2009)

Avoid Exposing Concurrency – Hide It Inside Synchronous Methods (Oct 2009)

Prefer structured lifetimes – local, nested, bounded, deterministic (Nov 2009)

I mentioned yesterday that I’ll be involved in two sessions at PDC09, including a parallel patterns tutorial. I know many of you are interested in concurrency in general and on Microsoft platforms in particular, so I thought I’d share this more complete list of concurrency-related sessions at PDC, put together by my colleague Stephen Toub.

Overview:

• The State of Parallel Programming
• Patterns of Parallel Programming: A Tutorial on Fundamental Patterns and Practices for Parallelism

Native code in Visual Studio 2010:

• A Computing Platform That Scales
• Lighting up Windows Server 2008 R2 Using the ConcRT on UMS
• C++ Forever: Interactive Applications in the Age of Manycore

Managed code in Visual Studio 2010:

• Manycore and the Microsoft .NET Framework 4: A Match Made in Microsoft Visual Studio 2010
• PLINQ: LINQ, but Faster!
• F# for Parallel and Asynchronous Programming

HPC Server:

• Accelerating Applications Using Windows HPC Server 2008

Research and Incubation:

• Axum: A .NET Language for Safe and Scalable Concurrency
• Concurrency Fuzzing & Data Races
• Data-Intensive Computing on Windows HPC Server with the DryadLINQ Framework
• Rx: Reactive Extensions for .NET
• Future of Garbage Collection

For those of you coming to PDC’09 in Los Angeles a couple of weeks from now, I’ll be there for a few hours on Monday and Wednesday participating in two events:

• Patterns of Parallel Programming: A Tutorial on Fundamental Patterns and Practices for Parallelism. The full-day tutorial is full of useful information. I’ll be giving the first hour or so as an intro/overview; if you’ve seen my high-level concurrency talks you’ll recognize much of it, but it’ll also have a slant toward patterns of course to set up the day. The rest of the tutorial will be presented by my colleagues Richard Ciapata, Ade Miller, and Stephen Toub.
• Panel: Microsoft Perspectives on the Future of Programming. This one’s going to be a blast. You can judge just from the names of my heavyweight fellow panelists: Butler Lampson, Burton Smith, Don Box, Erik Meijer, and Jeffrey Snover. Anytime you get a chance to watch or attend a talk by any of these, do. If you get a chance to come to this panel when they’re all on one stage, definitely do. As Bill Shatner might put it: “Do. Not Miss!”

See you at PDC!

On the flight to the ISO C standards meeting this morning, I was reading this month’s issue of CACM, and found that Sir C.A.R. (Tony) Hoare wrote a nice piece called Retrospective: An Axiomatic Basis for Computer Programming.

Hoare has long been a noted proponent of axioms and formal proofs of program correctness. In that light, the following passage on testing and axioms struck me as well put and I thought I’d share it (emphasis added):

One thing I got spectacularly wrong. I could see that programs were getting larger, and I thought that testing would be an increasingly ineffective way of removing errors from them. I did not realize that the success of tests is that they test the programmer, not the program. Rigorous testing regimes rapidly persuade error-prone programmers (like me) to remove themselves from the profession. Failure in test immediately punishes any lapse in programming concentration, and (just as important) the failure count enables implementers to resist management pressure for premature delivery of unreliable code [or forces management to be explicitly unreasonable in the face of bug bar data and specific failure cases having an objective severity –hps]. The experience, judgment, and intuition of programmers who have survived the rigors of testing are what make programs of the present day useful, efficient, and (nearly) correct. Formal methods for achieving correctness must support the intuitive judgment of programmers, not replace it.

My basic mistake was to set up proof in opposition to testing, where in fact both of them are valuable and mutually supportive ways of accumulating evidence of the correctness and serviceability of programs. …

He also mentions many other useful observations and reminds, including the value of assertions to find, not run-time errors, but programming bugs. (See also C++ Coding Standards Item 68: Assert liberally to document internal assumptions and invariants.)

The whole article is good reading, and not long. Recommended.

How interesting.

I’m at the ISO C++ meeting in Santa Cruz, CA, USA this week. Ten minutes ago we had a committee straw poll about whether we should remove, deprecate, or leave as-is the export template feature for C++0x. The general sentiment was to remove or deprecate it, with deprecation getting the strongest support because it’s safer per ISO procedure. Deprecation would not remove it from C++0x, but would put the world on notice that the committee is considering removing it from a future standard.

Note: Nothing is happening to export at this meeting. The effect of this straw poll of the committee was to give guidance to the core language working group (CWG) on the committee’s general sentiment on this issue. The CWG is taking this as guidance to produce a detailed proposal to take some action on export at our next meeting in Pittsburgh in March.

I’ll write a longer trip report in the next week, but this was worth noting in real time because it’s the first time export template has been discussed in committee since the highly controversial discussion of my and Tom Plum’s proposal to remove export at the Oxford meeting in 2003.

I’ve opened up a short concurrency poll to get a sense of what concurrency issues are top-of-mind for programmers, and I’d appreciate it if you could take a few minutes to participate. Some questions are about what you want to learn more about, others about your tools of choice in specific areas, and a few are slightly whimsical. I plan to use the results as input to topics to cover in future Effective Concurrency articles and talks, so by participating you’ll help influence the direction of future EC topics.

There are about 28 questions, each asking for just a word or a phrase in answer. Here again is the link: http://www.surveygizmo.com/s/193214/apw36

Thank you in advance for taking a few minutes to participate. If you’re interested in receiving a summary of the survey results, please leave your email address at the end of the survey and I’ll send you a copy (your email will not be used for any other purpose).

I recently received the following reader question (slightly edited):

About the (Stroustrup) approach of implementing IsDerivedFrom at page 27 in your book More Exceptional C++: […] why the second pointer assignment in:

static void Constraints(D* p)
{
B* pb=p; // okay, D better inherit from B…
pb=p; // huh? why this again?
}

Isn’t the initialization ” B* pb=p ” enough? What does the second assignment bring to the picture?

Excellent question. Interestingly, here’s the exact text from the book, which contains an additional comment that was intended to answer this specific question (emphasis added):

static void Constraints(D* p)
{
B* pb = p;
pb = p; // suppress warnings about unused variables
}

The reason for the redundant assignment is to try to shut up some compilers that issue warnings about unused variables. In a normal function, this is a helpful warning because, dollars to donuts, it’s a mistake and the programmer meant to use the variable but forgot or accidentally used some other variable instead. In this case, without the second line, pb is declared but never mentioned again, and by adding the extra line that mentions it one more time after its declaration, some compilers stop complaining.

Sometimes, however, we’re intentionally not using the variable, such as in this case where the only purpose of the function is to ensure that code that tries to convert a D* to a B* is compilable. (Note: There are other/better ways to achieve this. See the rest of Item 27 for superior approaches.)

However, in practice even the line “pb = p;” doesn’t shut compilers up very much, because many will still notice that now the second assigned value isn’t used, and will helpfully remind you that you probably didn’t mean to write an apparently useless “dead write” that is never read again. Most compilers offer a compiler-specific #pragma or other nonportable extension to silence this warning, but wouldn’t it be nice to have a portable way to suppress the warning that will work on all compilers?

Here’s a simple one-liner that works on all compilers I tried, and should not incur any overhead: Define an empty function template that takes any object by reference, but doesn’t actually do anything with it and so the empty function body should compile down to nothing. Let’s call it “ignore” – you can put it into your project namespace to avoid collisions on that common name.

  template<class T> void ignore( const T& ) { }

Note that ignore<T>’s parameter does not need a name; in fact, it had better not have a name or else you’ll get the same compiler warning as before about an unused variables, only in a new place.

Using ignore is easy. When you want to suppress the compiler warning, just write:

  static void Constraints(D* p)

  {

    B* pb = p;

    ignore(pb); // portably suppresses the warning

  }

Now all compilers I tried agree that the variable is “used” and stop complaining about pb.

[Oct 20: Updated  to add "const" on ignore<T>'s parameter to also handle rvalues.]

Astute readers may have noticed that Terry Crowley’s name frequently crops up in the Acknowledgments section of my Effective Concurrency columns. Who is Terry? To answer, Mary-Jo Foley profiles him this week.

This month’s Effective Concurrency column, Avoid Exposing Concurrency – Hide It Inside Synchronous Methods, is now live on DDJ’s website.

From the article:

You have a mass of existing code and want to add concurrency. Where do you start?

Let’s say you need to migrate existing code to take advantage of concurrent execution or scale on parallel hardware. In that case, you’ll probably find yourself in one of these two common situations, which are actually more similar than different:

• Migrating an application: You’re an application developer, and you want to migrate your existing synchronous application to be able to benefit from concurrency.
• Migrating a library or framework: You’re a developer on a team that produces a library or framework used by other teams or external customers, and you want to let the library take advantage of concurrency on behalf of the application without requiring application code rewrites.

You have a mountain of opportunities and obstacles before you. Where do you start?

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

Use Thread Pools Correctly: Keep Tasks Short and Nonblocking (Apr 2009)

Eliminate False Sharing (May 2009)

Break Up and Interleave Work to Keep Threads Responsive (Jun 2009)

The Power of “In Progress” (Jul 2009)

Design for Manycore Systems (Aug 2009)

Avoid Exposing Concurrency – Hide It Inside Synchronous Methods (Oct 2009)

“What’s the best way to process a pool of work?” is a recurring question. As usual, the answer is “it depends” because the optimal answer often depends on both the characteristics of the work itself and the constraints imposed by run-time system resources.

For example, I recently received the following email from reader Sören Meyer-Eppler, where the key was to avoid oversubscribing system resources (in this case, memory):

I have an application that has multiple threads processing work from a todo queue. I have no influence over what gets into the queue and in what order (it is fed externally by the user). A single work item from the queue may take anywhere between a couple of seconds to several hours of runtime and should not be interrupted while processing. Also, a single work item may consume between a couple of megabytes to around 2GBs of memory. The memory consumption is my problem. I’m running as a 64bit process on a 8GB machine with 8 parallel threads. If each of them hits a worst case work item at the same time I run out of memory. I’m wondering about the best way to work around this.

1. plan conservatively and run 4 threads only. The worst case shouldn’t be a problem anymore, but we waste a lot of parallelism, making the average case a lot slower.

2. make each thread check available memory (or rather total allocated memory by all threads) before starting with a new item. Only start when more than 2GB memory are left. Recheck periodically, hoping that other threads will finish their memory hogs and we may start eventually. Still dangerous if the check happens when all threads are just starting out with their allocations.

3. try to predict how much memory items from the queue will need (hard) and plan accordingly. We could reorder the queue (overriding user choice) or simply adjust the number of running worker threads.

4. more ideas?

I’m currently tending towards number 2 because it seems simple to implement and solve most cases. However, I’m still wondering what standard ways of handling situations like this exist? The operating system must do something very similar on a process level after all…

I replied:

I don’t have time to write a detailed answer right now, but also consider two queues (one for big tasks and one for small tasks), or having work items give a rough size estimate (possibly by doing an extra lightweight pass over the data up front).

May I post an extract of your mail on my blog? Then others may comment and provide useful hints.

He said yes, and so here it is for your consideration.

Note also this similar question that came up a few days ago on comp.programming threads, but with different constraints — in that case, it was about avoiding idleness rather than avoiding oversubscription.

I just received a reader email that asked about GotW #42:

You write "Non-Problem: Zero-Length Arrays Are Okay", but both 14882:2003 and N2914 "[dcl.array]" say "If the constant-expression (5.19) is present, it shall be an integral constant expression and its value shall be greater than zero.". Shall we assume that you overrule the standard? :-) Or am I missing something, like the meaning of "derived-declarator-type-list" (I can’t find it anywhere in 14882:2003)?

I thought the answer might be of interest to other people, so I’m posting it here.

First, this reader gets kudos for consulting not only the ISO C++ standard, but also the current C++0x working draft from June (paper N2914), to do research before asking the question. Good stuff.

He also gets a small penalty point, though, for reading only the subhead text that he quoted from GotW #42, and not the actual short passage the heading introduces. The article already contains the answer to his qustion, and that answer is still the same in the current C++0x draft all these years later:

From 5.3.4 [expr.new], paragraph 7:

When the value of the expression in a direct-new-declarator is zero, the allocation function is called to allocate an array with no elements. The pointer returned by the new-expression is non-null. [Note: If the library allocation function is called, the pointer returned is distinct from the pointer to any other object.]

So the reader is quoting from 8.3.4 [dcl.array], which governs non-heap allocated arrays (e.g., T array[N];). In that case, a zero length is not allowed by the standard. You can’t rely on it in portable code because although it is allowed as an extension in some popular compilers (e.g., Gnu gcc) it is treated as an error in others (e.g., Visual C++).

But the case being discussed in GotW #42 is dynamically allocating an array using the array form of new (e.g., new T[n] where n is zero), which is governed by 5.3.4 [expr.new]. Here, zero length is okay for the reasons given in GotW #42:

The result of "new T[0]" is just a pointer to an array with zero elements, and that pointer behaves just like any other result of "new T[n]" including the fact that you may not attempt to access more than n elements of the array… in this case, you may not attempt to access any elements at all, because there aren’t any.

"Well, if you can’t do anything with zero-length arrays (other than remember their address)," you may wonder, "why should they be allowed?" One important reason is that it makes it easier to write code that does dynamic array allocation. For example, the function f above would be needlessly more complex if it was forced to check the value of its n parameter before performing the "new T[n]" call.

To make GotW #42 completely unambiguous, it could more specifically say that zero-length heap-allocated arrays are okay, which was the case being discussed in the article. But it always helps to read more than the subhead text.

This month’s Effective Concurrency column, Design for Manycore Systems, is now live on DDJ’s website.

From the article:

Why worry about “manycore” today?

Dual- and quad-core computers are obviously here to stay for mainstream desktops and notebooks. But do we really need to think about "many-core" systems if we’re building a typical mainstream application right now? I find that, to many developers, "many-core" systems still feel fairly remote, and not an immediate issue to think about as they’re working on their current product.

This column is about why it’s time right now for most of us to think about systems with lots of cores. In short: Software is the (only) gating factor; as that gate falls, hardware parallelism is coming more and sooner than many people yet believe. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

Use Thread Pools Correctly: Keep Tasks Short and Nonblocking (Apr 2009)

Eliminate False Sharing (May 2009)

Break Up and Interleave Work to Keep Threads Responsive (Jun 2009)

The Power of “In Progress” (Jul 2009)

Design for Manycore Systems (Aug 2009)

Someone just asked me about getting more proficient in C++, and with their permission I thought I’d share the question and my answer in case it’s of broader interest to folks wanting to improve their C++ skills.

Here’s the question:

I need to take my C++ knowledge up a notch – or two. On a scale of 1-10 I’d consider my C++ knowledge a 5 and would like to get to 7-8 in the next 4, or so, months. As C++ is a large language my quandary is what subject areas, using what study materials, should I focus on to get myself to this next level.

Questions (assuming that I’m going to study approx 2 hrs/day for 4 months):

1. How would you suggest I structure such a course of study (subject areas, 5 of time for each) ?
1.1 If you were in my shoes, how would you proceed?
2. If I was going to study and hopefully know cold one of your books, which do you think I should select?

Here’s my response, which I didn’t limit to my own books. These are the first two “next books” books I would recommend to anyone who already knew the basics of the language and wanted to improve their proficiency with using C++ in production code:

If you’re not already familiar with Scott Meyers’ Effective C++, the current third edition is the plate to start (it’s substantially reworked since the first and second editions).

After that, I’d say the most important is C++ Coding Standards which I wrote with Andrei Alexandrescu. Not only does it cover the top 100 things we felt were important to say about using C++ in production code, and had its guidance peer-reviewed in advance by a who’s-who of the C++ community more than any other C++ book I’ve ever heard of, but it also includes references in each Item for where to find more in-depth treatment. Most Items are just one or two pages, so if you know that Item already you can use it as a refresher; and if you discover ones that you’re less familiar with or rusty on the details, it tells you where to go to find out more.

I’d say those are the two most important that I can suggest. Best wishes with your studies!

A week ago, I attended the summer ISO C++ meeting in Frankfurt, Germany. The C++ committee made a lot of progress on addressing national body comments on the full committee draft published last year, and is well on the way to publishing a second and final CD this winter with a final draft international standard a year after that. To meet that schedule, the committee decided to defer a major feature, “concepts”, and not include it in this standard.

I’m surprised at some of the commentary I’ve been reading on the web about the deferral of concepts, including not just technical inaccuracies but also that their absence in C++0x is somehow a big deal for C++ as a language or is a major geopolitical firestorm. It may feel like a big deal because of the work that’s gone into concepts and the large number of pages of standardese that were needed to specify the feature, and there was a lot of discussion about important design details including heavy traffic on the committee reflectors this spring and early summer that probably made them look more controversial than they are.

But my opinion is that concepts’ presence or absence in C++0x just won’t make much difference to most users. Let me give some reasons why I feel that’s so, and maybe debunk some common misconceptions along the way, in the form of an unofficial FAQ. Note that I was never involved in the design of concepts and don’t speak for the designers, whose views might differ here and there; the following are observations from my point of view as a participant in the standards body (including in Frankfurt last week), and chair of the ISO C++ committee during the entire time concepts were being developed up to the meeting they were voted into the ISO C++0x working draft last fall.

Q: Bjarne Stroustrup is the creator of C++ and a primary designer of concepts. What’s his take?

A: Great question. See Bjarne’s coverage here, published earlier today in Dr. Dobb’s.

Q: Were concepts removed because of political reason X, Y, or Z?

A: No. Concepts were removed because of normal project management considerations. The basic constraint has been phrased in many ways, all of them hitting at the same underlying tension among time, scope, cost, and quality. Here are two examples: “You can pick what goes on the train, or you can pick when the train leaves the station, but you can’t pick both.” “You can have good, fast, and cheap – pick any two.”

Fundamentally, the committee members felt that concepts are desirable but many (though not all) believed that they were still in design mode and needed more “bake time” – a formal poll of committee members in Frankfurt put the expected time at between three and four years – to get up to the quality of a feature that belongs in an international standard. Therefore, the two major options, and the ones that people generally argued for, were to either:

• Prioritize (bigger) scope. Wait for concepts, and accept a delay to the rest of the standard including features that are already finalized and ready to ship.
• Prioritize (faster) schedule. Finish and ship the standard to get already-done features into users’ hands, and decouple concepts to continue work on them for a future standard.

The committee decided to prioritize schedule and ship C++0x sooner while still leaving the door wide open to also adopt concepts later.

Q: Isn’t removing concepts a lot of work? Concepts are used pervasively in the C++0x CD published last fall.

A: Yes, but it’s less work than you’d think. We won’t have a de-conceptized working draft for the post-meeting mailing, two weeks after the meeting, but should have one soon after that. (I would be remiss not to add: Thanks to our always-hardworking project editor, Pete Becker!)

Recall that, until last September, we had a complete draft of C++0x features specified entirely without the use of concepts. And many of the uses of concepts in the draft standard library were just replacements of preexisting non-concepts wording or implicit requirements (e.g., DefaultConstructible, MemberContainer), and will just revert to their previous wording or disappear back into implicitness.

The main C++0x feature that was always specified in terms of concepts was the new range-based for loop (e.g., for( x : collection ) ). I’ve seen some wonder what we’ll do about that, but it’s already done; the non-concepts text for that feature was already voted in at the same Frankfurt meeting.

Q: Wasn’t this C++0x’s one big feature?

A: No. Concepts would be great, but for most users, the presence or absence of concepts will make no difference to their experience with C++0x except for quality of error messages (see below).

C++0x is still a major revision with many new major features, including improvements to templates (e.g., variadic templates, template aliases). C++’s other major new features include lambda functions (which unfortunately for Java were rejected for Java 7), move semantics (aka rvalue references), a concurrency memory model, threading and atomics libraries, initializer lists, and more. Other handy new features include delegating constructors, inherited constructors, defaulted and deleted functions, explicit virtual overriding control, alignment support, static assertions, and more. Not to mention convenience features like range-based for loops and “auto” type inference that are small but will certainly get a lot of use (as they do in C# and Java).

A number of those features are available now or soon in real compilers, including Gnu gcc, Visual C++ 2010, and Intel C++.

Q: Aren’t concepts about adding major new expressive power to the language, and so enable major new kinds of programs or programming styles?

A: Not really. Concepts are almost entirely about getting better error messages.

Yes, concepts would add one truly new expressive capability to the language, namely the ability to overload on concepts. That’s inconvenient to simulate without language support. Concept-based overloading would have been used in a handful of places in the standard library. But that’s it.

By far the most visible benefit of concepts lies in clearer template error messages, including the ability to do separate checking – confirming that a template’s implementation type-checks correctly with all possible valid types without having to instantiate it, and confirming that a given type can be used to instantiate a template without knowing the template’s implementation.

Q: Weren’t concepts all about bringing templates into the 21st century? Where does this leave templates?

A: Templates are still king of the genericity hill, for all their faults in the area of opaque compiler messages.

It’s important to remember that, in 1994, C++ was the only major language whose type genericity capabilities were strong enough to create the Standard Template Library (STL). Today, 15 years later, that is still true; you can’t express the STL’s containers-algorithms design separation well, or at all, using generics facilities in Ada, Java, .NET, or any other significant commercial or research language that I know of; as we learned when doing STL.NET, you can do the containers well with other generics, but not the orthogonalization with algorithms that is the heart of STL design style. Concepts or not, that hasn’t changed.

So my personal view is that templates have been in the 21st century since about 1994. No other language has yet caught up to their expressive power. And C++0x is adding some further power to templates, in particular by adding variadic templates and template aliases, both of which will help to simplify template code.

This month’s Effective Concurrency column, The Power of “In Progress”, is now live on DDJ’s website.

From the article:

Don’t let a long-running operation take hostages. When some work that takes a long time to complete holds exclusive access to one or more popular shared resources, such as a thread or a mutex that controls access to some popular shared data, it can block other work that needs the same resource. Forcing that other work to wait its turn hurts both throughput and responsiveness.

…

Let’s look at the question from another angle, suggested by my colleague Terry Crowley: Instead of viewing partially-updated state with in-progress work as an ‘unfortunate’ special case to remember and recover from, what if we simply embrace it as the normal case? …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

Use Thread Pools Correctly: Keep Tasks Short and Nonblocking (Apr 2009)

Eliminate False Sharing (May 2009)

Break Up and Interleave Work to Keep Threads Responsive (Jun 2009)

The Power of “In Progress” (Jul 2009)

Someone emailed me today asking:

I’m writing because I’m somewhat conscious of what I would consider a rather large hole in the parallel programming literature.

… What if one or more of your tasks throws an exception? Should the thread that runs the task swallow it? Should the caught exceptions get stashed somewhere so that the "parent" thread can deal with them once the tasks are complete? (This is somewhat tricky currently in a language such as C++(98) where one cannot store an exception caught with the "catch(…)" construct). Perhaps all tasks should have a no-throw guarantee? Perhaps some kind of asynchronous error handlers might be installed, somewhat like POSIX signals? The options are many, but choosing a strategy is hard for those of us with little parallel programming experience.

I thought I’d share my response here:

That’s an excellent question. Someone asked that very question in Stockholm last month at my Effective Concurrency course, and my answer started out somewhat dismissive: "Well, it’s about the same as you do in sequential code, and all the same guarantees apply; nothrow/nofail is only for a few key functions used for commit/rollback operations, and you’d usually target the basic guarantee unless adding the strong guarantee comes along naturally for near-free. So it’s pretty much the same as always. Although, well, of course futures may transport exceptions across threads, but that’s still the same because they manifest on .get(). And of course for parallel loops you may get multiple concurrent exceptions from multiple concurrent loop bodies that get aggregated into a single exception; and then there’s the question of whether you start new loop bodies that haven’t started yet (usually no) but do you interrupt loop bodies that are in progress (probably not), and… oh, hmm, yeah, I guess it would be good to write an article about that."

So the above is now adding to my notes of things to write about. :-) Maybe some of that stream-of-consciousness may be helpful until I can get to writing it up in more detail.

I pointed him to Doug Lea’s Concurrent Programming in Java pages 161-176, "Dealing with Failure", adding that I haven’t read it in detail but the subtopics look right. Also Joe Duffy’s Concurrent Programming on Windows, pages 721-733.

If you know of a good standalone treatise focused on error handling in concurrent code, please mention it in the comments.

This month’s Effective Concurrency column, “Break Up and Interleave Work to Keep Threads Responsive”, is now live on DDJ’s website.

Sorry for the long title; suggestions welcome. I always try to word the title to make it (a) short, (b) active, and (c) advice, but sometimes I’ll settle for two of those, or just one, until a better suggestion comes along.

From the article:

What happens when this thread must remain responsive to new incoming messages that have to be handled quickly, even when we’re in the middle of servicing an earlier lower-priority message that may take a long time to process?

If all the messages must be handled on this same thread, then we have a problem. Fortunately, we also have two good solutions, both of which follow the same basic strategy: Somehow break apart the large piece of work to allow the thread to perform other work in between, interleaved between the chunks of the large item. Let’s consider the two major ways to implement that interleaving, and their respective tradeoffs in the areas of fairness and performance.

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

Use Thread Pools Correctly: Keep Tasks Short and Nonblocking (Apr 2009)

Eliminate False Sharing (May 2009)

Break Up and Interleave Work to Keep Threads Responsive (Jun 2009)

This afternoon I was just finishing up my next Effective Concurrency article (it’ll be up in a few days), when some spam email arrived. Just as my fingers’ auto-delete macro was about to fire, I noticed something odd about the name of the attachment and did a double-take:

Cool! There must be some kind of new truth-in-advertising laws for spammers.

Yes, I know that as programmers we could argue about naming all day long. We could point out that maybe “virusLoader.gif” or “exploit_exploit_muhaha.gif” would be a little better, and argue about the relative merits of camel case and underscores. But there’s no need; I think “runnable.gif” is short, clear, and definitely good enough. (Evidently someone else thought so too, and just shipped it.)

An amusing hearse, seen on a neighborhood street:

Here’s a close-up of the license plate:

Made my morning.

For those of you who are interested in using or trying Microsoft development tools, I’m happy to report that Visual Studio 2010 Beta 1 is now available.

If you’re interested in:

• concurrency and parallel computing, check out the new concurrency runtime (ConcRT) that implements efficient work stealing for scalable code, the Asynchronous Agents Library and the Parallel Patterns Library (PPL) for C++, and the Task Parallel Library (TPL) and Parallel LINQ (PLINQ) for .NET. And some nifty concurrency debugging and profiling tools, too.
• C++0x features, check out the lambda functions (my fave), move-semantic rvalue references, the new auto, its good friend decltype, and the ever-helpful static_assert.
• functional programming in VS, enjoy the F# programming language.

Remember this is just a beta and not intended for production use, but there’s a lot of cool stuff to play around with and it should run fine side-by-side with VS2008. Feedback via forums or filing a bug/suggestion is always appreciated. Enjoy!

This month’s Effective Concurrency column, “Eliminate False Sharing”, is now live on DDJ’s website.

People keep writing asking me about my previous mentions of false sharing, even debating whether it’s really a problem. So this month I decided to treat it in depth, including:

• A compelling and realistic example where just changing a couple of lines to remove false sharing takes an algorithm from zero scaling to perfect scaling – even when many threads are merely doing reads. Hopefully after this nobody will argue that false sharing isn’t a problem. :-)
• How your performance monitoring and analysis tools do and/or don’t help you uncover the problem, and how to use them effectively to identify the culprit. Short answer: CPU activity monitors aren’t very helpful, but cycles-per-instruction (CPI) and cache miss rate measurements attributed to specific lines of source code are your friend.
• The two ways to correct the code: Reduce the frequency of writes to the too-popular cache line, or add padding to move other data off the line.
• Reusable code in C++ and C#, and a note about Java, that you can use to use padding (and alignment if available) to put frequently-updated objects on their own cache lines.

From the article:

In two previous articles I pointed out the performance issue of false sharing (aka cache line ping-ponging), where threads use different objects but those objects happen to be close enough in memory that they fall on the same cache line, and the cache system treats them as a single lump that is effectively protected by a hardware write lock that only one core can hold at a time. … It’s easy to see why the problem arises when multiple cores are writing to different parts of the same cache line… In practice, however, it can be even more common to encounter a reader thread using what it thinks is read-only data still getting throttled by a writer thread updating a different but nearby memory location…

A number of readers have asked for more information and examples on where false sharing arises and how to deal with it. … This month, let’s consider a concrete example that shows an algorithm in extremis due to false sharing distress, how to use tools to analyze the problem, and the two coding techniques we can use to eliminate false sharing trouble. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

“Use Thread Pools Correctly: Keep Tasks Short and Nonblocking” (Apr 2009)

“Eliminate False Sharing” (May 2009)

Seen at a gas station:

You know your UI has usability issues when people tape multiple signs on your gas pump to help people get through the intricate and error-prone process of purchasing fuel.

Why does the upper note exist? The trouble is that there’s a Debit button but not a Credit button, and so since Credit is the default, users need to remember to override the default before swiping the card. People will sometimes naturally forget the special step because there was originally no reminder that this needed to be done, and because most other pumps don’t work that way. One solution would be to print the information prominently near the card reader, thus standardizing the note and making it more visible. A better and simpler solution would be to do what most pumps do: Avoid the opportunity for forgetting to specify the right thing by having both buttons, Debit and Credit, and simply prompting the user to press one or the other after they swipe their card.

But the trouble behind the lower note is even more blatant and, frankly, inexcusable: Is there any good reason not to conditionally print “ENTER PIN” or “ENTER ZIP,” instead of just always printing “ENTER DATA” which is not only unclear but also one of the classic geeky words to avoid in a consumer-oriented UI?

Then again, I’m amazed that in this day and age I still see output like “1 item(s) purchased.” Apparently it’s still more important to write the programmer-friendly printf( “%d item(s)”, count ) than the user-friendly printf( “%d item%s”, count, (count==1 ? “” : “s”) ).

This month’s Effective Concurrency column, “Use Thread Pools Correctly: Keep Tasks Short and Nonblocking”, is now live on DDJ’s website.

From the article:

… But the thread pool is a leaky abstraction. That is, the pool hides a lot of details from us, but to use it effectively we do need to be aware of some things a pool does under the covers so that we can avoid inadvertently hitting performance and correctness pitfalls. Here’s the summary up front:

1. Tasks should be small, but not too small, otherwise performance overheads will dominate.

2. Tasks should avoid blocking (waiting idly for other events, including inbound messages or contested locks), otherwise the pool won’t consistently utilize the hardware well — and, in the extreme worst case, the pool could even deadlock.

Let’s see why. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

“Use Thread Pools Correctly: Keep Tasks Short and Nonblocking” (Apr 2009)

In my travels, I recently came across this empty store with an almost-empty box beside the front door. As seen in Monterey, CA:

Evidently some character had also noticed the empty store with its empty box, and decided to do a little walk-by wry economic commentary via repurposed quotation. Zooming on the once-empty box:

Now that I’m over the icky flu that forced me to postpone the seminar two weeks ago, I’m happy to say that we have new dates: Effective Concurrency (Europe) will be held on May 27-29, 2009, in Stockholm, Sweden. I’ll cover the following topics:

• Fundamentals: Define basic concurrency goals and requirements • Understand applications’ scalability needs • Key concurrency patterns
• Isolation — Keep work separate: Running tasks in isolation and communicate via async messages • Integrating multiple messaging systems, including GUIs and sockets • Building responsive applications using background workers • Threads vs. thread pools
• Scalability — Re-enable the Free Lunch: When and how to use more cores • Exploiting parallelism in algorithms • Exploiting parallelism in data structures • Breaking the scalability barrier
• Consistency — Don’t Corrupt Shared State: The many pitfalls of locks–deadlock, convoys, etc. • Locking best practices • Reducing the need for locking shared data • Safe lock-free coding patterns • Avoiding the pitfalls of general lock-free coding • Races and race-related effects
• High Performance Concurrency: Machine architecture and concurrency • Costs of fundamental operations, including locks, context switches, and system calls • Memory and cache effects • Data structures that support and undermine concurrency • Enabling linear and superlinear scaling
• Migrating Existing Code Bases to Use Concurrency
• Near-Future Tools and Features

I hope to see some of you there!

The latest casualties in the technical education world are the Software Development conferences – SD West, SD Best Practices, and Architecture & Design World – which are being discontinued effective immediately, making the SD West that was just held earlier this month the last of its kind. The conferences were run by the same company that ran some important but now-defunct magazines including C/C++ Users Journal and of course Software Development Magazine, as well as Dr. Dobb’s which still exists but recently went web-only.

SD, especially SD West, has been a mainstay of the programming conferences world since 1985. I’m going to miss these shows. They were a wonderful gathering place for talks and tutorials by A-list speakers covering many technologies, from C++ to Java to AJAX and more. More personally, Bjarne Stroustrup and I immensely enjoyed giving our two-day Stroustrup & Sutter seminar there several times in recent years, and skipped this year’s event in anticipation of doing a fresh one again next spring. Alas, it’s not to be (at least not with SD).

SD, you will be missed.

This month’s Effective Concurrency column, “Use Threads Correctly = Isolation + Asynchronous Messages”, is now live on DDJ’s website.

From the article:

Explicit threads are undisciplined. They need some structure to keep them in line. In this column, we’re going to see what that structure is, as we motivate and illustrate best practices for using threads — techniques that will make our concurrent code easier to write correctly and to reason about with confidence.

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns and the DDJ print magazine issue in which they first appeared:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

Use Threads Correctly = Isolation + Asynchronous Messages (Apr 2009)

Right now I should be at 40,000 feet somewhere over Baffin Island on my way to Stockholm for Effective Concurrency Europe, but instead I’m in bed with a fever that I’ve had since Wednesday night and still unable to talk. The organizer and I have been staying in touch with flu updates every few hours to see if lots of “TLC and OTC” (tender loving care and over-the-counter drugs) could get me well enough to fly and push through teaching for three days; that usually works, but this time it’s a really nasty one, and we have to postpone.

I’m sorry to have to disappoint this week and for any inconvenience this will surely cause some of you, and I hope that you’ll understand. The organizer has already emailed everyone who has registered to notify you of the cancellation of this week’s dates (this post partly serves as a backup to that email), and we’ll keep you posted as we reschedule. The response has exceeded expectations by a good margin, and I’m grateful for your strong interest and planning to take time away from your schedules to come and talk concurrency with me. I’m still very much looking forward to seeing you there in the near future. (And minus potentially spreading wicked germs.)

Best wishes to you and yours,

Herb

Like many areas in the United States, Seattle has recently been hit with layoffs and downsizing in our industry. So it’s quite timely that Steve McConnell’s company Construx, in the Seattle area, is offering free training for laid-off software workers:

After listening to doom and gloom economic reports for the past few months, we decided we would try to do something to brighten our little corner of the world. Here’s our official press release about it:

Construx Software has designated 25% of its public seminar seats free of charge to software workers who have been laid off. Construx seminars help software professionals improve their technical and managerial skills. Seminar attendees will be more effective when they reenter the workforce. Construx hopes this program will help laid-off software workers reenter the workforce more quickly.

Please read the full article for more details.

I’m very pleased to report that this program will include my next U.S. Effective Concurrency seminar which will be held in Bellevue, WA on July 27-30, 2009. I’ll blog about that seminar again as we get closer, but it’s on the Construx calendar now.

I hope that this will be helpful to some people, and I look forward to seeing some of you in Bellevue.

Aside: Please Be Professional

One problem with offering things for free is that people don’t always value them. I was surprised to learn that a surprising number of people who’ve asked for and received this free admission at Construx have been no-shows.

The following should go without saying, but here it is: As a courtesy to Construx and to other attendees, please don’t sign up unless you intend to come, and if something comes up that prevents you from attending please notify Construx. Otherwise, you may be taking the place of someone who would have liked to attend in your place (seating is limited), and Construx is out the cost of food (they provide breakfast and lunch) and printed color binders and such that go to waste.

Other Free Materials

Most of you know that virtually all the articles and columns I’ve ever written have always been available for free on my own website or via magazine websites (modulo some link rot, sigh). These include all the Effective Concurrency articles, and my C++-focused “Guru of the Week” and magazine articles.

The only things I’ve written that aren’t legally freely available are the final texts of my books, including translations, to which the publishing company owns the copyright; but the books are based on the same freely-available English articles, and Addison-Wesley/Pearson has never had any issue with those staying available, so the basic material is all there.

But there’s news on the “final book contents” front, too, at least in English: Once my current book(s) on Effective Concurrency is done, Jim Hyslop and I plan to compile a book of the C++ Conversations articles we coauthored in C++ Report and C/C++ Users Journal (thanks again for your patience, Jim!) and I also plain to write a second edition of Exceptional C++ – and both books will be updated for C++0x, which is now feature-complete and undergoing its first round of international review. It’s quite interesting just how often using C++0x language and library features (from lambdas to shared_ptrs and concurrency) really help solve the old problems in even more elegant and robust ways… and I’m sure we’ll throw in a couple of new problems and solutions too. My goal is to post these books’ draft Items, perhaps in the form of blog entries, as we write them. But I’m also working with the publisher to see if perhaps we could even post the very final Items in that format with their permission. I’ll post more news about that as there’s something to say.

This month’s Effective Concurrency column, “Sharing Is the Root of All Contention”, is now live on DDJ’s website.

This article aims to address the root cause behind some frequently made assertions: Statements like “locks kill scalability” and “CAS kills scalability” are mostly true but focus on symptoms rather than causes; and others such as “reader/writer mutexes are great for scalability” at minimum need some heavy qualification.

From the article:

Quick: What fundamental principle do all of the following have in common?

• Two children drawing with crayons.
• Three customers at Starbucks adding cream to their coffee.
• Four processes running on a CPU core.
• Five threads updating an object protected by a mutex.
• Six threads running on different processors updating the same lock-free queue.
• Seven modules using the same reference-counted shared object.

Answer: In each case, multiple users share a resource that requires coordination because not everyone can use it simultaneously — and such sharing is the root cause of all resulting contention that will degrade everyone’s performance. (Note: The only kind of shared resource that doesn’t create contention is one that inherently requires no synchronization because all users can use it simultaneously without restriction, which means it doesn’t fall into the description above. For example, an immutable object can have its unchanging bits be read by any number of threads or processors at any time without synchronization.)

In this article, I’ll deliberately focus most of the examples on one important case, namely mutable (writable) shared objects in memory, which are an inherent bottleneck to scalability on multicore systems. But please don’t lose sight of the key point, namely that "sharing causes contention" is a general principle that is not limited to shared memory or even to computing.

The Inherent Costs of Sharing

Here’s the issue in one sentence: Sharing fundamentally requires waiting and demands answers to expensive questions. …

Aside: This will be in the first electronic-only Dr. Dobb’s Report, and I’m going to have to get used to Dobb’s being electronic-only. One consequence of this change that I just discovered today is that an article can go online hours after I correct the proofs. That may seem like it shouldn’t be surprising in the days of blogs, but a magazine isn’t (or at least wasn’t) like a blog. When I wrote for magazines like C++ Report in the late 1990s, the time between submitting the final column manuscript and seeing it in print was about two to three months, depending on the magazine. Since the mid-2000s with Dobb’s, it’s been only one month from proofs to print — and Dobb’s could have posted the online versions even sooner, but held them back to be semi-synced with the print magazine. Now that the print constraint has disappeared, apparently I can (and, today, did) start work on a Friday morning to find the Dobb’s proofs I corrected the night before already on the web. Nifty. (And thanks again, Deirdre!)

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns and the DDJ print magazine issue in which they first appeared:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

Sharing Is the Root of All Contention (Mar 2009)

I just saw a CNN headline that read: “Young workers scrimp to live on $15/wk.” Before reading further, what do you think: Is that stunning and shocking? Or shockingly typical?

The story turned out to be a piece about white-collar workers in China trying to live frugally, spending only 100 Yuan on travel and food during the workweek to conserve funds. Of course, the workers’ actual total expenses and income are higher, because that $15/week figure doesn’t include weekend expenses and other major costs like rent. Even so, the story is considered newsworthy here, and is probably a shock to a number of readers in the western world.

But the headline wouldn’t surprise readers who are familiar with the approximate distribution of income/GDP/wealth in the world.

To illustrate, here are two personal data points from 2006, when my wife and I traveled to Kenya and Zambia to visit friends:

• Income: In Kenya, we were told that being a staff worker at a safari lodge is considered a good job. What does it pay? About $2 per day, for long hours and six-day weeks. This isn’t unusual; in about 30 countries, including Kenya, more than half of the population earns under $2 per day. An estimated two billion people – 30% of the world’s population – live on an income of less than $3 per day. And $3 per day is about what the attention-grabbing CNN headline implies, though the actual story behind that headline is much less bad.
• Cost of living: But what happens when we consider, not just dollar-for-dollar comparisons, but purchasing power? Isn’t it less expensive to live in less-developed countries? Yes, it usually is, especially for shelter and services – there’s been some talk lately on the U.S. news about retiring in Mexico as a way for older people to save money in this economy – but the difference for the same quality goods is often less than one might think. In Lusaka, the capital of Zambia, we found there were only three grocery stores [*] having similar goods to what we would expect to find in a U.S.-style Safeway, although of course the Zambian stores were much smaller than U.S. stores (more the size of a medium Trader Joe’s) and offered far less selection diversity (something like a factor of 20 fewer varieties or brands). When we visited one of the stores, I picked up a few Western-style items, totaled the price and converted to U.S. dollars in my head, and found that those comparable products in Lusaka cost nearly the same as we would have paid for the same items in Seattle. Our local friend replied: “Right, nobody who lives here would ever think of buying a can of soda pop.” Certainly not when a can of Coke costs a day’s wage for many people, and doesn’t confer any significant nutritional benefit.

The gulf between the western- and world-median standards of living is, simply put, vast – and growing. The standard of living that’s normal for most of the planet’s population is well nigh unimaginable to many of us in the western world, and even for those of us who’ve been there, it’s one thing to see it and quite another to really understand what such a life would be like. I don’t claim to.

[*] They might well be the only such stores in the country, not just the capital.

A number of people have asked whether I will be teaching my Effective Concurrency seminar in Europe. The answer is yes:

Effective Concurrency (Europe) will be held on March 16-18, 2009, in Stockholm, Sweden. This is my only public European seminar in 2009. I’ll cover the following topics:

• Fundamentals: Define basic concurrency goals and requirements • Understand applications’ scalability needs • Key concurrency patterns
• Isolation — Keep work separate: Running tasks in isolation and communicate via async messages • Integrating multiple messaging systems, including GUIs and sockets • Building responsive applications using background workers • Threads vs. thread pools
• Scalability — Re-enable the Free Lunch: When and how to use more cores • Exploiting parallelism in algorithms • Exploiting parallelism in data structures • Breaking the scalability barrier
• Consistency — Don’t Corrupt Shared State: The many pitfalls of locks–deadlock, convoys, etc. • Locking best practices • Reducing the need for locking shared data • Safe lock-free coding patterns • Avoiding the pitfalls of general lock-free coding • Races and race-related effects
• High Performance Concurrency: Machine architecture and concurrency • Costs of fundamental operations, including locks, context switches, and system calls • Memory and cache effects • Data structures that support and undermine concurrency • Enabling linear and superlinear scaling
• Migrating Existing Code Bases to Use Concurrency
• Near-Future Tools and Features

I hope to see some of you there!

While walking our dogs recently, we came across several of these signs — ironically, in front of our neighborhood school.

This month’s Effective Concurrency column, “volatile vs. volatile”, is now live on DDJ’s website and also appears in the print magazine. (As a historical note, it’s DDJ’s final print issue, as I mentioned previously.)

This article aims to answer the frequently asked question: “What does volatile mean?” The short answer: “It depends, do you mean Java/.NET volatile or C/C++ volatile?” From the article:

What does the volatile keyword mean? How should you use it? Confusingly, there are two common answers, because depending on the language you use volatile supports one or the other of two different programming techniques: lock-free programming, and dealing with ‘unusual’ memory.

Adding to the confusion, these two different uses have overlapping requirements and impose overlapping restrictions, which makes them appear more similar than they are. Let’s define and understand them clearly, and see how to spell them correctly in C, C++, Java and C# — and not always as volatile. …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns and the DDJ print magazine issue in which they first appeared:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

volatile vs. volatile (Feb 2009)

A few days ago I posted a challenge to name the researcher/team and approximate year each of the following 16 important technologies was first demonstrated. In brief, they were:

• The personal computer for dedicated individual use all day long.
• The mouse.
• Internetworks.
• Network service discovery.
• Live collaboration and desktop/app sharing.
• Hierarchical structure within a file system and within a document.
• Cut/copy/paste, with drag-and-drop.
• Paper metaphor for word processing.
• Advanced pattern search and macro search.
• Keyword search and multiple weighted keyword search.
• Catalog-based information retrieval.
• Flexible interactive formatting and line drawing.
• Hyperlinks within a document and across documents.
• Tagging graphics, and parts of graphics, as hyperlinks.
• Shared workgroup document collaboration with annotations etc.
• Live shared workgroup collaboration with live audio/video teleconference in a window.

A single answer to all of the above: Doug Engelbart and his ARC team, in what is now known as “The Mother of All Demos”, on Monday, December 9, 1968.

Last month, we marked the 40th anniversary of the famous Engelbart Demo, a truly unique “Eureka!” moment in the history of computing. 40 years go, Engelbart and his visionary team foresaw — and prototyped and demonstrated — many essential details of what we take for granted as our commonplace computing environment today, including all of the above-listed technologies, most of them demonstrated for the first time in that talk.

This talk would be noteworthy and historic just for being the first time a “mouse” was shown and called by that name. Yet the mouse was just one of over a dozen important innovations to be compellingly presented with working prototype implementations.

Note: Yes, some of the individual technologies have earlier theoretical roots. I deliberately phrased the question to focus on implementations because it’s great to imagine a new idea, but it isn’t engineering until we prove it can work by actually building it. For example, consider hypertext: Vannevar Bush’s Memex, vintage 1945, was a theorectical “proto-hypertext” system but it unfortunately remained theoretical, understandably so given the nascent state of computers at the time. Project Xanadu, started in 1960, pursued similar ideas but wasn’t demonstrated until 1972. The Engelbart Demo was the first time that hypertext was publicly shown in a working form, together with a slew of other important working innovations that combined to deliver an unprecedented tour de force. What made it compelling wasn’t just the individual ideas, but the working demonstrations to show that the ideas worked and how they could combine and interact in wonderful ways.

Recommended viewing

You can watch the 100-minute talk here (Stanford University) in sections with commentary, and here (Google Video) all in one go.

We enjoy such an abundance of computing riches that it’s easy to take wonderful technological ideas for granted. Yet so many of the pieces of our modern computing experience that we consider routine today were at one time unimaginable. After all, back in the early days of computing, we were still discovering what these newfangled room-filling gadgets might eventually become capable of — who could have known then what using computers would be like today?

Of course, we have these technologies today because some visionaries did know, did imagine them… and, best of all, built and demonstrated them.

Hence today’s challenge:

Quiz: For each of the following 16 technologies that have become commonplace in our modern computing experience, give the researcher/team and approximate year that a working prototype was first demonstrated. How many can you answer without a web search?

• The personal computer for dedicated individual use, that one person can have at their disposal all day long. (Hint: Before the Altair in 1975 and Apple I in 1976.)
• Mouse input with a graphical pointer. (Hint: Before the Xerox Alto at Xerox PARC in 1973.)
• Internetworks across campuses and cities. (Hint: Before Ethernet at Xerox PARC (again) in 1973.)
• Discovery of ‘who’s got what service’ in an internetwork.
• Using internetworks for live collaboration, not just file sharing. (Hint: Before RDP and others.)
• Hierarchical structure within a file system and within a document. (Hint: Before Unix.)
• Cut/copy/paste, with drag-and-drop.
• Paper metaphor for word processing, starting with a blank piece of paper and the applying  formatting and navigating levels in the structure of text.
• Advanced pattern search and macro search within documents. (Hint: Before MIT’s Emacs.)
• Keyword search and multiple weighted keyword search. (Hint: Long before Google (alternate link).)
• Information retrieval through indirect construction of a catalog.
• Flexible interactive formatting and line drawing.
• Hyperlinks within a document and across documents, and “jumping on a link” to navigate. (Hint: Before Tim Berners-Lee invented the World Wide Web in 1989-1990.) (Hint’: Yes, before HyperCard too.)
• Tagging graphics, and parts of graphics, as hyperlinks. (Hint: Before Flickr.)
• Workgroup collaboration on a document, including collaborative annotations, allowing members of a group to use and modify a document. (Hint: Before Lotus Notes and Ward Cunningham’s Wikis.)
• The next step up from that: Live collaboration on a document with screen sharing on the two writers’ computers so they can see what the other is doing — with live audio/video teleconference in a window at the same time. (Hint: Not Skype or LiveMeeting.)

[edited 2009.01.15 to add link to DDJ's announcement]

2008 was quite a year, full of landmark events that were certainly historic, if not always welcome.

If I had to pick one technology-related highlight from the past year, it would be this: A notable inflection point in the ongoing shift from traditional media to the web. Given that that tide is still in progress, why single out 2008? I think we’ll look back at 2008, especially the fourth quarter, as a turning point when the web became an A-list media outlet and first started to beat up, and even replace, major legacy competitors in newspapers, technical magazines, movies, and TV. An inflection point, if you will, where the web clearly stopped being the pencil-necked upstart, and visibly emerged as the new gorilla flexing its advertising-revenue-pumped-up muscles and kicking sand on the others.

In 2008, and particularly in the last month, the web began to outright replace some existing newspapers and technology and programming magazines.

• December 2008: The first two major city newspapers go web-only or web-mostly. The Detroit News and The Detroit Free Press announced truncated print editions and reduced print delivery. Newspapers in several other cities seem likely to follow fairly soon.
• As of January 2009: PC Magazine is going “digital only.” After the January issue, the print magazine will disappear. Instead, the content will appear exclusively on the web.
• As of January 2009: Dr. Dobb’s Journal is permanently suspending print publication and going web-only. Some of the content will be available as a new “Dr. Dobb’s Report” section of InformationWeek. My Effective Concurrency column will continue, and I’ll continue to blog when new columns go live on the web, so if you’re reading the column via this blog nothing will significantly change for you.

As of next month, the only major technical programmer’s trade magazines still available in print, that I know of, are platform- and technology-specific ones like asp.netPRO and MSDN Magazine — and even those increasingly feature online-only content. For example, from MSDN Mag’s January 2009 editor’s note:

As we continue to grow our coverage to keep pace with the rapidly expanding set of technologies, we will often offer content exclusively online at msdn.microsoft.com/magazine. So please check in frequently!

Gotta love RSS (and Atom etc.): Every text feed is like a magazine or newspaper column, every blogger a columnist. Every audio/video podcast feed is like a radio or TV series, or a radio station or TV channel. And our feed reader is the new magazine/newspaper, as we subscribe to columnists to make our personal custom newsmagazine. But RSS readers, along with RSS-consuming clients like iTunes, are more — they’re our personal selection, not only of the columns we want to read (on whatever topics we want, including the funnies section), but also of the media we want to hear and watch. It’s increasingly our way to choose the text, audio, and video we want all together. Who knew that a large chunk of the coming media convergence would come in the shape of RSS readers?

And other media are feeling the pressure from Dr. Web, the new gorilla:

• This week (late December 2008), even cable operator Time Warner pushed web delivery for TV. Time Warner has had various contract disputes with Viacom and some local stations. But as part of the dispute:
  Time Warner will respond to Viacom’s advertisement, [Time Warner spokesman] Mr. Dudley said, by highlighting the availability of television content on the Internet.

  “We will be telling our customers exactly where they can go to see these programs online,” Mr. Dudley said. “We’ll also be telling them how they can hook up their PCs to a television set.”

• During October-December 2008, Netflix’s Watch Instantly has started to turn into a juggernaut. It’s interesting enough that you can watch 12,000+ movies and TV shows streamed over the net to your PC at high quality. As of October 2008, you can get them on your Mac. As of November 2008, on your Xbox. As of December 2008, on your TiVo. As noted in one recent article:
  “It’s a good strategic move,” said Andy Hargreaves, an analyst with Pacific Crest Securities. “Netflix sees the world will go digital, no matter what they do. They realize there is more to be lost by waiting than doing it early.”

And I won’t even get started on SaaS: hosted rich GUI apps served up on the web, for example the December 2008 (again) announcement about Office Online.

Yes, don’t forget 2008, especially December 2008: The month the first major newspapers moved mostly to the web and abandoned print at least partly; the month that PC Magazine and Dr. Dobb’s suspended print publication and went web-only; the month Netflix Watch Instantly arrived in the living room on TiVos after hitting Xboxes a fortnight before; the month cable provider Time Warner threatened to tell people to watch TV on the net; and the month even Microsoft announced their intent to deliver significant Office web applications.

I for one welcome Dr. Web, our new Gorilla and media overlord!

Besides, what choice do I have?

Lest people get the wrong idea, I enjoy reading Jeff Atwood’s blog and agree with much of what he writes so entertainingly and provocatively. So far I’ve only responded when I strongly felt differently about something, which has been a grand total of twice now.

So let me also offer an example of something I wholeheartedly agree with. Yesterday, Jeff cited what is also my own favorite Programming Pearls figure:

Despite the enduring wonder of the yearly parade of newer, better hardware, we’d also do well to remember my all time favorite graph from Programming Pearls:

Everything is fast for small n.

Spot on. If you’re a professional programmer and haven’t read Programming Pearls yet, “run don’t walk” to your bookstore of choice.

Incidentally, just to tie this in to parallel computing as well, Jeff’s article also cites a nice graph of optimizations that improved NDepend:

Patrick Smacchia’s lessons learned from a real-world focus on performance is a great case study in optimization.

Patrick was able to improve nDepend analysis performance fourfold, and cut memory consumption in half. As predicted, most of this improvement was algorithmic in nature, but at least half of the overall improvement came from a variety of different optimization techniques.

As I’ve said many times, measure twice, optimize once: Know when and where to optimize. Profilers are your friend. As Patrick writes:

When it comes to enhancing performance there is only one way to do things right: measure and focus your work on the part of the code that really takes the bulk of time, not the one that you think takes the bulk of time.

And of course, in our ever-more-multicore world, the contribution of parallelization gain will continue to grow and dominate the optimization of CPU-bound code. But as Patrick also notes, realizing that gain is not always trivial:

While we get the 15% gain from between 1 and 2 processors, the gain is almost zero between 2 and 4 processors. We identified some potential IO contentions and memory issues that will require more attention in the future. This leads to another lesson: Don’t expect that scaling on many processors will be something easy, even if you don’t share states and don’t use synchronization.

Yesterday, the ever-popular Jeff Atwood (of Coding Horror fame) wrote an article [*] on how not to write Web 2.0 UIs. Unfortunately, it’s exactly backwards: What he identifies as a problem is in fact not only desirable, but necessary.

• [*] Aside: Jeff, I know you love pictures, but is that particular gratuitous one really necessary? Yes, I know it’s CGI, but it made me really hesitate about linking to your post and has nothing to do with your technical point.

Jeff observes correctly that when you write an application to run on a platform like Windows and/or OS X, your application should follow the local look-and-feel. Fine so far. But he then repeats a claim that I believe is incorrect, at least today, and based on a fallacy — and adds another fallacy:

[Quoting Bill Higgins]

• … a Windows application should look and feel like a Windows application, a Mac application should look and feel like a Mac application, and a web application should look and feel like a web application.

Bill extends this to web applications: a web app that apes the conventions of a desktop application is attempting to cross the uncanny valley of user interface design. This is a bad idea for all the same reasons; the tiny flaws and imperfections of the simulation will be grossly magnified for users.

…

• When you build a “desktop in the web browser”-style application, you’re violating users’ unwritten expectations of how a web application should look and behave.

There are actually two fallacies here.

Fallacy #1: “Look and feel like a web application”

The first fallacy is here:

a web application should look and feel like a web application.

…

violating users’ unwritten expectations of how a web application should look and behave.

These assertions beg the question: What does a web application “look and feel like,” and what do users expect? Also, are you talking about Web 1.0, where there is an answer to these questions, or Web 2.0, where there isn’t?

For Web 1.0 applications, the answer is fairly easy: They look like hyperlinked documents built on technologies like HTML and CSS. That’s what people expect, and get.

But the examples Bill uses aren’t Web 1.0 applications, they’re Web 2.0 applications. For Web 2.0 applications, there are no widely accepted UI standards, and applications are all over the map. Indeed, the whole point of Ajax-y/Web2.0-y applications is to get beyond the current 1.0-standard functionality.

Not only are there are no widely-accepted UI standards, there aren’t even many widely-accepted UI technologies. Consider how many dissimilarities there are among just Flash, Silverlight, and JavaFX as these technologies compete for developer share. Then consider that even within any one of these technologies people actually build wildly diverse interfaces.

Here’s the main example these bloggers used:

Consider the Zimbra web-based email that Bill refers to.

It’s pretty obvious that their inspiration was Microsoft Outlook, a desktop application.

But what’s wrong with Zimbra?

Here’s a Better Question #1: How could you do better?

And for bonus points, Still Better Question #2: What about OWA? Consider that Microsoft already provides essentially the same thing, with the same approach, in the form of Outlook Web Access, which looks remarkably like the usual Outlook [caveat: this is an example of why I write above and below that 'most' Web 2.0 apps don't try to emulate a particular OS look-and-feel; this one does]. For example (a couple of sample shots taken from this brief video overview):

The rich UI isn’t a bug, it really is a feature — a killer feature we’re going to be seeing more of, not less of, because this is what delivering software-as-a-service (SaaS) is all about. Although I use the desktop Outlook most of the time, I like OWA and think it’s the best web-based email and calendaring I’ve seen especially when I’m away from my machine (and I’ve tried several others, though granted you do need to be using Exchange). I suspect its UI conventions are probably pretty accessible even to non-Windows users, though that’s probably debatable and your mileage may vary.

Fallacy #2: “A web app that apes the conventions of a desktop application is [wrong]“

The second fallacy is Jeff’s comment (boldface original):

a web app that apes the conventions of a desktop application is attempting to cross the uncanny valley of user interface design.

I think this is flawed for three reasons.

First, the “uncanny valley” part makes the assumption that people will find it jarring that the app tries to look like a desktop application. (This concern is related to, but different from, the concern of fallacy #1.) But most such apps aren’t doing that at all, because they know their users will access them from PCs, Macs, and lots of different environments, and they have to look reasonable regardless of the user’s native environment. They’re usually not trying to duplicate a given platform’s native look and feel.

Second, what they are doing is borrowing from UI components and conventions that already work well in desktop GUI environments, and are common across many of those environments. When you have no standards for Web 2.0 look and feel, then doing the best you can by borrowing from ideas we already know work pretty well isn’t just okay, it’s necessary. What else can you do?

Finally, the worst part is this: The whole point of SaaS is to deliver desktop-like rich-GUI applications on the web. So what is being labeled ‘wrong’ above is the whole point of what we’re doing as an industry.

“SaaS/Web 2.0 on Web 1.0″: The new “GUI on DOS”

Most SaaS/Web 2.0 applications today look and feel pretty much the way GUI applications looked and felt like on DOS, before technologies like Windows and OS/2 PM existed. Around the late 1980s, people wrote lots of GUI applications that ran on DOS, but we didn’t have a widely-used common GUI infrastructure that handled basic windows and menus and events, much less standards like CUA that tried to say how to use such a common infrastructure if we had it. So they each did their own thing, borrowing where possible from what seemed to work well for GUIs on other platforms.

Twenty years ago, everyone writing GUIs on DOS designed the UIs as best they could, borrowing where possible from what they saw worked on platforms like the Macintosh and Xerox Alto and Star — but the results were all over the map, and would stay that way until a standard environment, followed by standard guidelines, came into being.

Today, everyone writing rich Web 2.0 applications is doing their own thing, borrowing as best they can from Macs and Windows and others — but the results are all over the map, and will continue to be until there actually is such a thing as a UI standard for rich-GUI web applications. You can see that in the differences between Zimbra and Outlook Web Access. In the meantime, it’s not just okay to borrow from what we’ve learned on the desktop; it’s necessary.

And the question isn’t whether metaphors users already understand on the desktop will migrate to the web, but which ones and how soon, because it’s the whole point of SaaS. The industry will soon be going well beyond Google Apps; with offerings like Office Online already announced for the short term, which puts still more rich-client GUI apps like word processors and spreadsheets in the browser (with functionality somewhere between Google Apps and the desktop version of Office).

Zimbra and Outlook Web Access aren’t examples of poor web app design, but exactly the opposite: They’re just the beginning of the next wave of rich web apps.

This month’s Effective Concurrency column is special — it turned into a feature-length article. (I don’t know whether it’ll officially be called a “feature” or a “column” in the print issue.) “Measuring Parallel Performance: Optimizing a Concurrent Queue” just went live on DDJ’s site, and will also appear in the print magazine.

From the article:

How would you write a fast, internally synchronized queue, one that callers can use without any explicit external locking or other synchronization? Let us count the ways…or four of them, at least, and compare their performance. We’ll start with a baseline program and then successively apply three optimization techniques, each time stopping to measure each change’s relative performance for queue items of different sizes to see how much each trick really bought us.

The goal of the article is to see how to measure and understand our code’s parallel performance and measure the actual effect of specific optimizations. Disclaimer: The goal of this article is not to write the fastest possible queue in the world (though it’s pretty good). I’ve already had plenty of email on recent queue-related columns from people who sent me their “faster” implementations; writing lock-free queues seems to be a popular indoor sport. Interestingly, for well of half of the ones I received, a 30-second glance at the code was enough to determine that the code had to be incorrect. Why? Because if it doesn’t do any synchronization on the shared variables — if there aren’t any locks, atomics, fences or other synchronization in the code — then it has races, which will manifest in practice even on forgiving platforms like x86/x64, and there’s no need to look further. (For more details, see the September 2008 column, Lock-Free Code: A False Sense of Security. Even some code submissions I received in response to that very article were broken for the same reasons shown in that article.)

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns and the DDJ print magazine issue in which they first appeared:

The Pillars of Concurrency (Aug 2007)

How Much Scalability Do You Have or Need? (Sep 2007)

Use Critical Sections (Preferably Locks) to Eliminate Races (Oct 2007)

Apply Critical Sections Consistently (Nov 2007)

Avoid Calling Unknown Code While Inside a Critical Section (Dec 2007)

Use Lock Hierarchies to Avoid Deadlock (Jan 2008)

Break Amdahl’s Law! (Feb 2008)

Going Superlinear (Mar 2008)

Super Linearity and the Bigger Machine (Apr 2008)

Interrupt Politely (May 2008)

Maximize Locality, Minimize Contention (Jun 2008)

Choose Concurrency-Friendly Data Structures (Jul 2008)

The Many Faces of Deadlock (Aug 2008)

Lock-Free Code: A False Sense of Security (Sep 2008)

Writing Lock-Free Code: A Corrected Queue (Oct 2008)

Writing a Generalized Concurrent Queue (Nov 2008)

Understanding Parallel Performance (Dec 2008)

Measuring Parallel Performance: Optimizing a Concurrent Queue (Jan 2009)

Oops, I just noticed that I forgot to blog about one recent Effective Concurrency column: “Writing Lock-Free Code: A Corrected Queue” which also appeared in the October 2008 print issue of Dr. Dobb’s Journal.

From the article:

As we saw last month [1], lock-free coding is hard even for experts. There, I dissected a published lock-free queue implementation [2] and examined why the code was quite broken. This month, let’s see how to do it right.

Here is the complete-as-of-this-writing set of links to the published Effective Concurrency columns. As always, the months reflect the magazine print issue dates; they usually hit the web a bit sooner:

August 2007: The Pillars of Concurrency

September 2007: How Much Scalability Do You Have or Need?

October 2007: Use Critical Sections (Preferably Locks) to Eliminate Races

November 2007: Apply Critical Sections Consistently

December 2007: Avoid Calling Unknown Code While Inside a Critical Section

January 2007: Use Lock Hierarchies to Avoid Deadlock

February 2008: Break Amdahl’s Law!

March 2008: Going Superlinear

April 2008: Super Linearity and the Bigger Machine

May 2008: Interrupt Politely

June 2008: Maximize Locality, Minimize Contention

July 2008: Choose Concurrency-Friendly Data Structures

August 2008: The Many Faces of Deadlock

September 2008: Lock-Free Code: A False Sense of Security

October 2008: Writing Lock-Free Code: A Corrected Queue

November 2008: Writing a Generalized Concurrent Queue

December 2008: Understanding Parallel Performance

Wow, DDJ just posted the previous one a few days ago, and already the next Effective Concurrency column is available: “Understanding Parallel Performance” just went live, and will also appear in the print magazine.

From the article:

Let’s say that we’ve slickly written our code to apply divide-and-conquer algorithms and concurrent data structures and parallel traversals and all our other cool tricks that make our code wonderfully scalable in theory. Question: How do we know how well we’ve actually succeeded? Do we really know, or did we just try a couple of tests on a quad-core that looked reasonable and call it good? What key factors must we measure to understand our code’s performance, and answer not only whether our code scales, but quantify how well under different circumstances and workloads? What costs of concurrency do we have to take into account?

This month, I’ll summarize some key issues we need to keep in mind to accurately analyze the real performance of our parallel code. I’ll list some basic considerations, and then some common costs. Next month, I have a treat in store: We’ll take some real code and apply these techniques to analyze its performance in detail as we successively apply a number of optimizations and measure how much each one actually buys us, under what conditions and in what directions, and why.

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns (based on the magazine print issue dates):

August 2007: The Pillars of Concurrency

September 2007: How Much Scalability Do You Have or Need?

October 2007: Use Critical Sections (Preferably Locks) to Eliminate Races

November 2007: Apply Critical Sections Consistently

December 2007: Avoid Calling Unknown Code While Inside a Critical Section

January 2007: Use Lock Hierarchies to Avoid Deadlock

February 2008: Break Amdahl’s Law!

March 2008: Going Superlinear

April 2008: Super Linearity and the Bigger Machine

May 2008: Interrupt Politely

June 2008: Maximize Locality, Minimize Contention

July 2008: Choose Concurrency-Friendly Data Structures

August 2008: The Many Faces of Deadlock

September 2008: Lock-Free Code: A False Sense of Security

October 2008: Writing Lock-Free Code: A Corrected Queue

November 2008: Writing a Generalized Concurrent Queue

December 2008: Understanding Parallel Performance

The next Effective Concurrency column, “Writing a Generalized Concurrent Queue”, just went live on DDJ’s site, and also appears in the print magazine.

From the article:

Last month [1], I showed code for a lock-free queue that supported the limited case of exactly two threads—one producer, and one consumer. That’s useful, but maybe not as exciting now that our first rush of lock-free coding glee has worn off. This month, let’s tackle the general problem of supporting multiple producers and multiple consumers with as much concurrency as possible. The code in this article uses four main design techniques: …

I hope you enjoy it. Finally, here are links to previous Effective Concurrency columns (based on the magazine print issue dates):

August 2007: The Pillars of Concurrency

September 2007: How Much Scalability Do You Have or Need?

October 2007: Use Critical Sections (Preferably Locks) to Eliminate Races

November 2007: Apply Critical Sections Consistently

December 2007: Avoid Calling Unknown Code While Inside a Critical Section

January 2007: Use Lock Hierarchies to Avoid Deadlock

February 2008: Break Amdahl’s Law!

March 2008: Going Superlinear

April 2008: Super Linearity and the Bigger Machine

May 2008: Interrupt Politely

June 2008: Maximize Locality, Minimize Contention

July 2008: Choose Concurrency-Friendly Data Structures

August 2008: The Many Faces of Deadlock

September 2008: Lock-Free Code: A False Sense of Security

October 2008: Writing Lock-Free Code: A Corrected Queue

November 2008: Writing a Generalized Concurrent Queue

The ISO C++ committee met in San Francisco, CA, on September 15-20. You can find the minutes here, including the votes to approve papers.

The most important thing the committee accomplished was this:

Complete C++0x draft published for international ballot

The biggest goal entering this meeting was to make C++0x feature-complete and stay on track to publish a complete public draft of C++0x for international review and comment — in ISO-speak, an official Committee Draft or CD. As I predicted in the summer, the committee achieved that at this meeting. Now the world will know the shape of C++0x in good detail. Here’s where to find it: The September C++0x working draft document is essentially the same as the September 2008 CD.

This is “it”, feature-complete C++0x, including the major feature of “concepts” which had its own extensive set of papers for language and library extensions — I’ll stop there, but there are still more concepts papers at the mailing page and some more still to come during the CD phase. (If you get the impression that concepts is a big feature, well, it is indeed easily the biggest addition we made in C++0x.)

What’s next? As I’ve mentioned before, we’re planning to have two rounds of international comment review. The first of two opportunities for national bodies to give their comments is now underway; the second round will probably be this time next year. The only changes expected to be made between that CD and the final International Standard are bug fixes and clarifications. It’s helpful to think of a CD as a feature-complete beta, and we’re on track to ship one more beta before the full release.

And a new convener

On a personal note, I’m very happy to see this accomplished at the last meeting during my convenership. I’ve now served as secretary and then convener (chair) of the ISO C++ committee for over 10 years, and my second three-year term as convener ended one week after the San Francisco meeting. A decade is enough; I decided not to volunteer for another term as chair.

As of a few weeks ago, P. J. Plauger is the new convener of ISO/IEC JTC1/SC22/WG21 (C++). Many of you will know P.J. (or Bill, as he’s known within the committee) from his long service to the C and C++ communities, including that he has been a past convener of the ISO C standards committee, past editor of the C/C++ Users Journal, the principal author of the Dinkumware implementation of the C++ standard library, and recipient of the 2004 Dr Dobb’s Journal Excellence in Programming Award, among various other qualifications and honors. He has been a regular participant at ISO C++ meetings for about as long as they’ve been held, and his long experience with both the technology and the ISO standards world will serve WG21 well.

I’m very happy to have been able to chair the committee during the development of C++0x. Now as we move from “develop mode” into “ship mode” it will be great to have his experienced hand guiding the committee through the final ISO process. Thanks for volunteering, Bill!

This spring at SD West in Santa Clara, Bjarne and I did a fresh-and-updated S&S event with lots of new material.

We don’t usually repeat the same material, but this time there’s been such demand that we agreed to do a repeat… four weeks from today, in Boston. More information and talk descriptions follow.

CONTENT ADVISORY

Again, usually our S&S events feature mostly new material, but this one is almost identical to the material we did in spring 2008 in Santa Clara. Bjarne is substituting one talk, and will present “C++ in Safety-Critical Systems” instead of his talk on C++’s design and evolution; and we’ll both be updating to the material to reflect the current state of the draft ISO C++0x standard. But otherwise it’ll be identical.

So:

• If you missed our event this spring, here’s your second chance! It was our highest-rated S&S ever, and in the post-conference survey we asked the question “Would you recommend this course to a colleague?” and 100% said yes.
• If you already attended this spring and came to all our sessions, you’ve seen nearly all this material already, but feel free to encourage a friend or colleague to attend who you think would benefit from the material.

The Talks

Wednesday, October 29, 2008

C++0x Overview (Bjarne Stroustrup)

Safe Locking: Best Practices to Eliminate Race Conditions (Herb Sutter)

How to Design Good Interfaces (Bjarne Stroustrup)

Lock-Free Programming in C++—or How to Juggle Razor Blades (Herb Sutter)

Grill the Experts: Ask Us Anything! (Bjarne Stroustrup & Herb Sutter)

Thursday, October 30, 2008

["Best of Stroustrup & Sutter"] Update of talk voted “Most Informative” at S&S 2007: Concepts and Generic Programming in C++0x (Bjarne Stroustrup)

What Not to Code: Avoiding Bad Design Choices and Worse Implementations (Herb Sutter)

C++ in Safety-Critical Systems (Bjarne Stroustrup)

How to Migrate C++ Code to the Manycore “Free Lunch” (Herb Sutter)

Discussion on Questions Raised During the Seminar (Herb Sutter & Bjarne Stroustrup)

Registration

This two-day seminar is getting billing on two different conferences that are running at the same time in the Hynes Convention Center: SD Best Practices and the Embedded Systems Conference. S&S is technically part of both conferences, which means you can attend S&S via either one… either

• register via SDBP, or
• register via ESC

and both ways will include options that include our two-day seminar.

I look forward to seeing many of you in Boston! Best wishes,

Herb

Even genuinely newsworthy topics can get distorted when commentators exaggerate or use data selectively. Here are two recent examples I noticed.

“This is the worst financial crisis since the Great Depression.” It’s true that it’s bad and even historic, and this sound bite correctly doesn’t actually claim it’s as bad as the Depression. I hope it doesn’t turn out to be in the same league as that; then, people were lining up at soup kitchens. For now, however, Apple is still on track to sell 10 million iPhone 3Gs this year, which says something.

“Yesterday [Monday, September 29, 2008] saw the worst single-day plunge in Dow Jones history.” “It’s a new Black Monday.” Well, these got my attention, because I remember Black Monday on October 19, 1987 very well. I was working in IT at a major bank, doing software application support for traders and related departments. When I went up to the trading floor that day, I immediately knew something was badly wrong because of the eerie sound as the elevator doors opened — a sound you never hear during trading hours, and certainly not from a room full of traders standing at their desks: silence.

Yesterday’s loss of 777 points was stunning as the largest single-day point loss in Dow history. But as a percentage loss that’s not even in the top 10 Bad Dow Days, all but two of which occurred before 1935. Those two since the Depression occurred on October 19 and 26, 1987, when the Dow lost 22.6% and then another 8% of its total value in single sessions, respectively #2 and #9 on All-Time Bad Dow Days list. For perspective, as of this writing the Dow is down 19.8% so far this entire year, and it surely hasn’t been a good year. Today’s crisis is already historic and could well get worse yet, of course, but some of us do remember some pretty bad ones in the past.

As good old Sam Clemens said (approximately), there are lies, darned lies, and statistics. Even when the statistics are true, always cross-check them for perspective. Even the best news and the worst news can be overstated, and viewing the same data from multiple angles helps ensure we understand it properly.

[Edited 9/30 to add: At 22.6%, Black Monday in 1987 was actually the worst Dow Day ever if you don't count "reopening days" after unusual market closures when the markets catch up with events that happened while they were closed. So when was the all-time worst Dow Day ever? Perhaps surprisingly, the answer is not in 1929, though several of the all-time top 10 were in that year. Rather, it was December 12, 1914, when the Dow dropped 24.4% after the markets reopened after being closed entirely for over four months due to the outbreak of World War I. (The markets closed on July 30, two days after Austria-Hungary declared war and a day before Germany did.) That helps put in perspective just how bad October 1987 was, and of course that today's crisis is also pretty bad even if it hasn't beaten those prior records, yet.]

A few days ago at UIUC, Ralph Johnson gave a very nice talk on “Parallel Programming Patterns.” It’s now online, and here’s the abstract:

Parallel programming is hard. One proposed solution is to provide a standard set of patterns. Learning the patterns would help people to become expert parallel programmers. The patterns would provide a vocabulary that would let programmers think about their programs at a higher level than the programming language. The patterns could steer programmers away from common errors and towards good design principles.

There have been a number of papers about parallel patterns, and one book Patterns for Parallel Programming. None of them have become popular. I think the problem is that parallel programming is diverse and requires more design expertise than traditional software design. Thus, parallel programming experts use more patterns than parallel programming expert. I’ll critique the existing patterns and explain what I think should be done to make a set of patterns that can be as effective for parallel programming as patterns have been for object-oriented design.

If Johnson’s name sounds familiar, it should: He’s one of the “Gang of Four” authors of the seminal book Design Patterns.

Recommended viewing.

The first offering of the three-day Effective Concurrency course in May went very well. We’re doing it again later this month — this will be the last offering this year.

Here’s the brief information (more details below):

3-Day Seminar: Effective Concurrency

September 22-4, 2008
Bellevue, WA, USA
Developed and taught by Herb Sutter

This course covers the fundamental tools that software developers need to write effective concurrent software for both single-core and multi-core/many-core machines. To use concurrency effectively, we must identify and solve four key challenges:

• Leverage the ability to perform and manage work asynchronously
• Build applications that naturally run faster on new hardware having more and more cores
• Manage shared objects in memory effectively to avoid races and deadlocks
• Engineer specifically for high performance

This seminar will equip attendees to reason correctly about concurrency requirements and tradeoffs, to migrate existing code bases to be concurrency-enabled, and to achieve key success factors for a concurrent programming project. Most code examples in the course can be directly translated to popular platforms and concurrency libraries, including Linux, Windows, Java, .NET, pthreads, and the forthcoming ISO C++0x standard.

Here’s a summary of what we’ll cover during the three days:

Fundamentals

• Define basic concurrency goals and requirements
• Understand applications’ scalability needs
• Key concurrency patterns

Isolation: Keep Work Separate

• Running tasks in isolation and communicate via async messages
• Integrating multiple messaging systems, including GUIs and sockets
• Building responsive applications using background workers
• Threads vs. thread pools

Scalability: Re-enable the Free Lunch

• When and how to use more cores 
• Exploiting parallelism in algorithms 
• Exploiting parallelism in data structures 
• Breaking the scalability barrier

Consistency: Don’t Corrupt Shared State

• The many pitfalls of locks–deadlock, convoys, etc.
• Locking best practices
• Reducing the need for locking shared data
• Safe lock-free coding patterns
• Avoiding the pitfalls of general lock-free coding
• Races and race-related effects

Migrating Existing Code Bases to Use Concurrency

Near-Future Tools and Features

High Performance Concurrency

• Machine architecture and concurrency
• Costs of fundamental operations, including locks, context switches, and system calls
• Memory and cache effects
• Data structures that support and undermine concurrency
• Enabling linear and superlinear scaling

I hope to get to meet some of you in the Seattle area!

The adage, quoted again this week by Bruce Schneier:

The plural of “anecdote” is not “data.”

But lest we enshrine raw data as holy in itself, another perspective:

And the plural of “datum” is not “proof.”