What the market will bear

Very frequently I see the idea that capitalism (or market economy) will somehow ensure that the end-users get the best possible products with the lowest prices. Yet many times common wisdom contradicts this. How is it possible? (Disclaimer: I'm no economist, these are just some personal ideas and observations).

Personally I think that there are two causes of this phenomenon in the software industry: vendor lock-in and the over focus on short term profit.

Vendor lock-in can mean several things, but usually it refers to data lock-in (where it is very difficult to extract the already introduced data) and/or interface lock-in (where the staff has learned the current interface and it would take serious time - mind you, many of these people are not tech-junkies - for them to re-learn a new interface - which also may not be the most intuitive or usable one).

Concrete example: on the Romanian market there are several accounting products. All of them I've seen so far are poorly written (some of them are still based on an MS-DOS version of FoxPro for example). But the software vendors are not the only ones to blame, mind you, because legislation here changes almost daily (I think we hold the record for a law which existed for less than 48 hours!). Also, an other aspect of our legislation system is that after the law appears, you have to wait (on average a month, but longer periods are not unusual) until the application methodology appears. Two things follow from this: (a) the law is so confusing that nobody knows how to apply it without further clarification and (b) you can be fined because you didn't respect the law, which nobody knew how to apply! Anyway, back to the accounting software: the one which is accepted to be the best is written in Delphi (nothing wrong with that) with a look which reminds you of the first applications written for Windows (big, clunky buttons with an red x or green a check mark on them for yes and no and so on). The back-end is based on BDE (Borland Database Engine), which, although it has good integration with the programming language, it completely inadequate for storing anything even remotely complex. No foreign keys, stored procedures, etc. It has been discontinued for a reason. The business I'm talking about is a small-to-medium one, and even there it takes around 10 minutes to bring up some reports. The company selling the product tries to blame this on the network, but this is a laughable excuse (all the computers are connected to the local LAN with 100MBit speeds). And did I mention that the database re-indexing must be done manually and there is no command line tool so that it can be scheduled! If you thing that this is something for the DailyWTF, you're right. But it gets even better: as I mentioned earlier, the laws change almost daily here and accounting products need to adapt. So, after yet an other law change, the company managed to produce an upgrade which theoretically respected the law. I say theoretically, because their clients never got to use it, since it broke horribly when installed, taking down the whole accounting system. And mind you, these are the best on the market!

The other example I want to give is not so clear-cut (because of government involvement you can make the argument that it's the fault of bureaucracy and corruption not a failure of free marked): a local company (which again won't be named to protect the guilty) prides itself with producing an European level e-Learning system (as a side-note: European level seems to be a keyword around here lately). This given company also buys full page ads in many of the local computer magazines where they write long texts about how they won different ads. I never used the system, but was blow away just by the specs: written in Java with an Oracle back-end. Now I'm sure that many of the schools have such huge amount of data that it couldn't be handled by PostgreSQL <sarcasm>. Nice way to sell some more Oracle licenses!

A final example (again, this is somewhat irrelevant, since I'm talking about the government here): check out the programs published by the Ministry of Finance to help companies making their monthly reports. They look like some VB apps written by 14 year olds who just read the first half of a how to program in VB book. And most probably big bucks were paid for them, much bigger than I make in a year.

The point of all this rant is that pure economical forces won't create the needs for technological excellence, or create it very slowly (and I mean very slowly - think decades). One of the factors is that if you are just a little bit better than the others, you can dominate the market. And that little bit better can be your bug ridden antique FoxPro based accounting system! An other problem is that customers don't know that it can be done better, nor do they have a way to find it out, since there is no better solution! So forget agile programming, test driven development, source control and all these nice things you only hear about in podcasts and say hello to big bucks made by selling cheap software!