In the beginning (of Windows), there was C. Then came the (non) flood of applications. Then Bill begat Visual Basic, C begat C++ and the development environments for all of the languages got merged into Visual Studio. With each version release, each of the languages became more powerful, faster and easier to use. Along the way the Internet and Java appeared. Microsoft struggled with them in the code, in the market, and in the courts. Then, after much pique, and more than the usual amount of thunder, Redmond begat .Net inscribed on a silver DVD tablet. At least that was true for the Visual Studio.Net. Visual Studio.Net is Microsoft's latest revolution in software development. It provides a very rich environment in which to develop programs.

Customers want features more than performance. This point finally got through to me when I attended a seminar at the Computer History Museum. The topic was the origin of spreadsheets. The panel consisted of Dan Bricklin and Bob Frankston, the creators of VisiCalc, and Mitch Kapor, the creator of 1-2-3. The VisiCalc team conceded that 1-2-3 won the marketplace battle because VisiCalc focused on running fast on every computer marketed, whereas 1-2-3 provided many features and only ran on one computer and required large amounts of memory. Microsoft learned this lesson early. You may remember the experience of installing a new version of Microsoft Word and being able to type faster than it could display characters on the screen. You pouted and got a faster computer. This characteristic has begun to appear in some surprising places.

"Rich" is a software marketing word for lots of features, and the .Net development environment provides that. There are so many APIs, reserved words, etc. that it makes you think of the old saw that "high technology is low magic"; it will do anything as long as you can know the magic reserved words. This brings me back to spreadsheets. Remember the feature-rich one won the race even though it was slower.

The questions I am pursuing are:

How much of a performance penalty is there to execute in the .Net environment?
What is the relative speed of the languages?
How much optimization is done?

After several conflicting responses, non-responses and unintelligible responses, your humble reporter is reduced to working it out on his own. I enlisted the help of .Net programming guru, Odis Wooten, in developing a test case.

To get a reasonably indicative benchmark, we needed a compute-intensive application of non-trivial complexity. It then needed to be implemented in several languages exactly the same way, statement by statement. During the course of development, we frequently discovered statement groups that could be improved. We had to fight off these temptations in the interest of completing precisely equivalent applications in a reasonable amount of time. I think we could double the speed now, but that isn't the point. We implemented in the following languages:

Visual Basic.Net - Microsoft transmogrified Visual Basic, their most popular language ever, into this dialect of C++.
C# - It's pronounced C-sharp. This is a new language derived from C++ specifically to take advantage of the .Net environment.
C++ - This has been Microsoft's workhorse development language for several years.

A key part of .Net is the Common Language Runtime, CLR. Visual Basic.Net and C# run in this environment. Here the source language is converted to an Intermediate Language, IL. The IL is compiled to machine code the first time it is required to execute. This just-in-time compilation needs to be taken into account when benchmarking the implementations. C++ runs outside of this environment, executing very much like pre-.Net languages.

The application chosen is a tree-searching program where the tester can select the complexity of the task. Over 4000 lines of code are in the test assembly containing the three implementations and the hosting program. The program was run on four PCs at three levels. There were two desktops, one notebook and a dual processor. The timing results are in the table in seconds. The chart displays only the timings at Level 10 in seconds.

Chart

C# runs around 15% faster than VB.Net. C++ runs around three times the speed of VB.Net or C#! Differences are substantial all the way around. The difference between C# and VB.Net are larger than we expected before this experiment. The C++ speed advantage has to be considered stunning! Keep in mind that this is a statement-by-statement transliteration of the code. The initial rendering of the code was done in VB.Net. The nature of the code is such that non-computational OS services are not used, i.e. no input or output during the timed interval. Speed in this case should be simply a function of the quality of the code generated. I strongly suspect that VB6 will produce code whose performance is more comparable to that of C++ than VB.Net.

Is it time to abandon the move to .Net? No. The vast majority of programs written today have their performance limited by things other than compute speed. Unless you are developing real-time simulations, e.g. Doom, or transforming huge arrays of data, e.g. photo editing, you are probably better off with the vast collection of resources provided by the .Net environment. But until Microsoft gets serious about high quality code generation in .Net, there will remain a substantial community of developers who aren't going to migrate. The other major VB.Net migration impediment is that the convenience of programming associated with earlier versions of Visual Basic has been sacrificed in the implementation of VB.Net. There is hope that both of these characteristics will be rectified in future releases. The best way to insure this is to tell Microsoft that they are important to you.

Graph

Fred Thorlin is a writer and contract software developer with experience in compiler development now working in Visual Basic environments. He also writes columns on computer industry events and computing on the road. He can be contacted at thorlin@swbell.net.

Odis Wooten is president of NanoSoft Corporation and chairman of the HAL C# SIG. He does contract software development in the C languages. He can be contacted at odis@msn.com.