[blackhole]

"Programmer" is an Overgeneralization

Earlier today, I came across a post during a google-fu session that claimed that no one should use the C++ standard library function make_heap, because almost nobody uses it correctly. I immediately started mentally ranting about how utterly ridiculous this claim is, because anyone whose gone to a basic algorithm class would know how to properly use make_heap. Then I started thinking about all the programmers who don't know what a heap is, and furthermore probably don't even need to know.

Then I realized that both of these groups are still called programmers.

When I was a wee little lad, I was given a lot of very bad advice on proper programming techniques. Over the years, I have observed that most of the advice wasn't actually bad advice in-of-itself, but rather it was being given without context. The current startup wave has had an interesting effect of causing a lot of hackers to realize that "performance doesn't matter" is a piece of advice riddled with caveats and subtle context, especially when dealing with complex architectures that can interact in unexpected ways. While this broken telephone effect arising from the lack of context is a widespread problem on its own, in reality it is simply a symptom of an even deeper problem.

The word programmer covers a stupendously large spectrum of abilities and skill levels. On a vertical axis, a programmer could barely know how to use vbscript, or they could be writing compilers for Intel and developing scientific computation software for aviation companies. On a horizontal axis, they could be experts on databases, or weeding performance out of a GPU, or building concurrent processing libraries, or making physics engines, or doing image processing, or generating 3D models, or writing printer drivers, or using coffeescript, HTML5 and AJAX to build web apps, or using nginx and PHP for writing the LAMP stack the web app is sitting on, or maybe they write networking libraries or do artificial intelligence research. They are all programmers.

This is insane.

Our world is being eaten by software. In the future, programming will be a basic course alongside reading and math. You'll have four R's - Reading, 'Riting, 'Rithematic, and Recursion. Saying that one is a programmer will become meaningless because 10% or more of the population will be a programmer on some level. Already the word "programmer" has so many possible meanings it's like calling yourself a "scientist" instead of a physicist. Yet, what other choices do we have? The only current attempt at fixing this gave a paltry 3 options that are just incapable of conveying the differences between me and someone who graduated from college with a PhD in artificial intelligence. They do multidimensional mathematical analysis and evaluation using functional languages. I write incredibly fast, clever C++ and HLSL assembly while juggling complex transformation matrices to draw pretty pictures on the screen. These jobs are both extremely difficult for completely different reasons, and neither person can do the other persons job. What is good practice for one is an abhorration for the other. We are both programmers.

Do you know why we have pointless language wars, and meaningless arguments about what is good in practice? Do you know why nobody ever comes to a consensus on these views except in certain circles where "practice" means the same thing to everyone? Because we are overgeneralizing ourselves. We view ourselves as a bunch of programmers who happen to specialize in certain things, and we are making the mistake of thinking that our viewpoint applies outside of our area of expertise. We are industrial engineers trying to tell chemists how to run their experiments. We are architects trying to tell English majors how to design an essay because we both use lots of paper.

This attitude is deeply ingrained in the core of computer science. The entire point of computer science is that a bunch of basic data structures can do everything you will ever need to do. It is a fallacy to try and extend this to programming in general, because it simply is not true. We are forgetting that these data structures only do everything we need to do in the magical perfect land of mathematics, and ignore all the different implementations that are built for different areas of programming, for completely different uses. Donald Knuth understood the difference between theory and implementation - we should strive to recognize the difference between theoretical and implementation-specific advice.

It is no longer enough to simply ask someone if they are a programmer. Saying a programmer writes programs is like saying a scientist does science. The difference is that botanists don't design nuclear reactors.

[Goober5000]

Good article.

[Aardwolf]

Lets not forget the fact that some programmers can't program.

[Ghostavo]

While I agree programming covers a width arrangement of topics, I feel the author is somehow confusing a skill, e.g. programming, with a profession, e.g. programmer.

I find amusing he touches a similar example of this,  "We are architects trying to tell English majors how to design an essay because we both use lots of paper.", but fails to find a parallel.

That is, we all know how to write and yet few of us call ourselves writers. Another example is many of us can do math to varying degrees, and yet few of us call ourselves mathematicians.

Unless he is, of course, saying people who know how to program shouldn't call themselves that, unless that is specifically what they do for a living. You know, like writers or mathematicians.

[The E]

The problem outlined in the article is that the generic term "programmer" covers a lot of different skillsets and specializations. And while there are certain things common to all programmers (just as there are things common to all scientists, or musicians), it does not mean that someone working on, say, a business reporting application can seamlessly transfer to do operating system coding or vice versa.

The other big point he makes is that methodologies that work in one field are not easily applicable to others, even though the tools used are the same. If, for example, somebody working on a word processor is asking somebody who specializes in high-performance parallel computing for advice, it is highly probable that the methods the second person uses will be irrelevant to the first one, even though both use C++. Yet both are programmers, and yet, this literally happens all the time.

[Ghostavo]

I'm not seeing the problem. One is an application developer the other is an HPC developer.

It's like complaining romance novel writers and technical book writers are both called writers even if they cannot do each other's jobs.

[The E]

It's like complaining romance novel writers and technical book writers are both called writers even if they cannot do each other's jobs.

That is exactly the problem. Or rather, romance writers asking tech writers for advice and vice versa. A technique that works in HPC may be the completely wrong thing to do when developing an AJAX application, and yet, people keep trying to implement stuff that they've seen working in other fields, or heard about working in other fields, even though it may be the completely wrong approach to doing business.
The point mentioned in the article was about a function used wrong by many people who do not understand how it works (and probably do not need to understand how it works), but who got told by others that it should be used. It's an advanced form of cargo-cult programming, a trap that is very easy to fall into.

[Ghostavo]

It's like complaining romance novel writers and technical book writers are both called writers even if they cannot do each other's jobs.

That is exactly the problem. Or rather, romance writers asking tech writers for advice and vice versa. A technique that works in HPC may be the completely wrong thing to do when developing an AJAX application, and yet, people keep trying to implement stuff that they've seen working in other fields, or heard about working in other fields, even though it may be the completely wrong approach to doing business.
The point mentioned in the article was about a function used wrong by many people who do not understand how it works (and probably do not need to understand how it works), but who got told by others that it should be used. It's an advanced form of cargo-cult programming, a trap that is very easy to fall into.

If someone doesn't use a function correctly, he's either a bad programmer or the documentation is faulty (or the third option of the function's call being retarded, but let's ignore that one).

If bad romance novel writers continuously mangled the language they use, we wouldn't modify the language to accommodate them. (Even though it seems that's how language evolution works...  )

This isn't a HPC vs normal programming thing or a contrast between some other different programming domains. Hence, that is not the problem. It's basically a non sequitur by part of the author.

[The E]

You are either not reading my posts correctly or dodging my points.

[Ghostavo]

Then please rephrase them, since I'm not understanding them then.

I'm not seeing the leap in logic of "Some programmers don't understand how to use this function." to "Programmers is a vague label.". To me the whole issue he presents in the beginning is totally unrelated.

[karajorma]

Let me put it this way. If you asked a particle physicist to explain how to dissect a lab rat and he said he didn't know how would you say he was a bad scientist? Would you blame the physics textbook he's looking at? But both particle physics and animal biology are sciences. People who study them are both scientists.

In this case he can easily say "I'm a physicist, I know nothing about biology" and everyone will understand why he knows nothing about the subject. The author however is talking about the fact that we use the term programmer to refer to two jobs as different as the physicist and biologist. But worse that because we use the same term for both you get physicists explaining to biologists why their methodology is better and that the biologist should use it.

[Polpolion]

Not entirely sure what the big deal here is. Programmer is a general term for a profession, just like lawyer, doctor, farmer, etc. I don't think anyone was trying to claim that when you say you're a programmer, you're saying that you know literally everything about making computers do what you want.

[karajorma]

You're missing the main point. Which is that even though it is a general word programmers too often assume that what works in their particular field is a general case which works for all programmers. The whole point of the article was that this simply isn't true.

[Polpolion]

Still not clear about things; every field has its specializations, why would people assume programming is any different? Especially with such a diverse array of things that need to be programmed in tremendously varying levels of abstraction, I'd think it's plenty clear that some situations call for different use of CS ideas, not that this isn't a CS idea in and of itself. The article stresses so much that "programmer" is a term insufficient in detail to adequately describe what someone does, but can anyone think of a single word to describe what they do without any ambiguity?

Anyway, that aside this is as good a spot as any to ask about any of your experiences with programming at the university level. The Electrical Engineering and Computer Science dept at my university only offers B.S.Es in computer science, computer engineering, and electrical engineering (ie no software engineering, networking, etc). You'd think that this wouldn't be fine enough of selection having read this article. Not that I'm in much of a position to comment, being just shy of halfway done. For those of you that have finished uni in a computer related field, how wide of an area did (or could with the electives offered) you cover for what degree?

[Ghostavo]

So can anyone give an actual example of "a technique which works on a specific field that programmers assume work for all"?

Because what the author presented wasn't that at all.

[jr2]

Am I understanding correctly that the underlying problem is that some programmers think that because they know how to program perfectly well for their job, that everyone should do things the way they do, when in fact, programmers with other focuses need to do things differently?

[LHN91]

So can anyone give an actual example of "a technique which works on a specific field that programmers assume work for all"?

Because what the author presented wasn't that at all.

For example, a programmer that specializes in object oriented programming, i.e. Java, writing interfaces for large systems like banks, would have absolutely no idea what to do when coding for embedded systems. Embedded systems require careful memory management and programming as efficient;y and close to bare metal as possible, usually in Assembly or ANSI C. Someone used to coding in Java wouldn't have spent any time managing memory, for example, as Java handles that for you; also, situations where you would generally be using Java are generally not ones that require realtime speeds, so they likely don't know much about optimizing for execution speed.

[Ghostavo]

And what programmers assume that embedded systems' programming is not different from, say... application programming (which is what I assume you were trying to make as a counterpart)? Or are we just confusing programmers with people who dabble into programming?

[LHN91]

Am I understanding correctly that the underlying problem is that some programmers think that because they know how to program perfectly well for their job, that everyone should do things the way they do, when in fact, programmers with other focuses need to do things differently?

Basically, this.

If an embedded systems programmer has never seen a piece of application code, or vice versa, then yes, they may believe that things that apply to one apply to the other. In all honesty, many of those who 'dabble in programming' (IMO) probably have a better view of the differences between types than someone who took a 6 month College course in, say, Embedded Systems, who went straight into a job writing embedded systems.

[Ghostavo]

Let me try to present my argument in another form. If the programmer label is that vague and it's related to that "problem", how would making it less vague solve it?

In the case you presented we can already narrow down the specialization those programmers have and their areas. Hell, we just did, embedded systems programmer and "dude who dabbles into programming" (or for those of you who want a serious title, lets just call him applications programmer).

The fact that we already have names and labels for the various things we have been talking about would suggest that this labelling business is not the problem, whatever that problem is.

And coming back to the author, using the programmer label is not the same as using a scientist one. One programmer can with some training do the job of another programmer. The same can't be said of a physicist to a botanist.


P.S.
Joining Polpolion, I've got a Masters in Computer Science, where the course work was supposedly network heavy, my thesis was about AI/game theory, and now I'm trying to do research on OS architecture and trying to extend stuff about my thesis on my spare time.