Hard Light Productions Forums
Off-Topic Discussion => Programming => Topic started by: blackhole on February 06, 2012, 08:30:50 am
-
"Programmer" is an Overgeneralization (http://blackhole12.blogspot.com/2012/02/programmers-are-overgeneralized.html)
Earlier today, I came across a post during a google-fu session that claimed that no one should use the C++ standard library function make_heap, because almost nobody uses it correctly. I immediately started mentally ranting about how utterly ridiculous this claim is, because anyone whose gone to a basic algorithm class would know how to properly use make_heap. Then I started thinking about all the programmers who don't know what a heap is, and furthermore probably don't even need to know.
Then I realized that both of these groups are still called programmers.
When I was a wee little lad, I was given a lot of very bad advice on proper programming techniques. Over the years, I have observed that most of the advice wasn't actually bad advice in-of-itself, but rather it was being given without context. The current startup wave has had an interesting effect of causing a lot of hackers to realize that "performance doesn't matter" is a piece of advice riddled with caveats and subtle context, especially when dealing with complex architectures that can interact in unexpected ways. While this broken telephone effect (http://en.wikipedia.org/wiki/Chinese_whispers) arising from the lack of context is a widespread problem on its own, in reality it is simply a symptom of an even deeper problem.
The word programmer covers a stupendously large spectrum of abilities and skill levels. On a vertical axis, a programmer could barely know how to use vbscript, or they could be writing compilers for Intel and developing scientific computation software for aviation companies. On a horizontal axis, they could be experts on databases, or weeding performance out of a GPU, or building concurrent processing libraries, or making physics engines, or doing image processing, or generating 3D models, or writing printer drivers, or using coffeescript, HTML5 and AJAX to build web apps, or using nginx and PHP for writing the LAMP stack the web app is sitting on, or maybe they write networking libraries or do artificial intelligence research. They are all programmers.
This is insane.
Our world is being eaten by software. In the future, programming will be a basic course alongside reading and math. You'll have four R's - Reading, 'Riting, 'Rithematic, and Recursion. Saying that one is a programmer will become meaningless because 10% or more of the population will be a programmer on some level. Already the word "programmer" has so many possible meanings it's like calling yourself a "scientist" instead of a physicist. Yet, what other choices do we have? The only current attempt at fixing this gave a paltry 3 options (http://www.skorks.com/2010/03/the-difference-between-a-developer-a-programmer-and-a-computer-scientist/) that are just incapable of conveying the differences between me and someone who graduated from college with a PhD in artificial intelligence. They do multidimensional mathematical analysis and evaluation using functional languages. I write incredibly fast, clever C++ and HLSL assembly while juggling complex transformation matrices to draw pretty pictures on the screen. These jobs are both extremely difficult for completely different reasons, and neither person can do the other persons job. What is good practice for one is an abhorration for the other. We are both programmers.
Do you know why we have pointless language wars, and meaningless arguments about what is good in practice? Do you know why nobody ever comes to a consensus on these views except in certain circles where "practice" means the same thing to everyone? Because we are overgeneralizing ourselves. We view ourselves as a bunch of programmers who happen to specialize in certain things, and we are making the mistake of thinking that our viewpoint applies outside of our area of expertise. We are industrial engineers trying to tell chemists how to run their experiments. We are architects trying to tell English majors how to design an essay because we both use lots of paper.
This attitude is deeply ingrained in the core of computer science. The entire point of computer science is that a bunch of basic data structures can do everything you will ever need to do. It is a fallacy to try and extend this to programming in general, because it simply is not true. We are forgetting that these data structures only do everything we need to do in the magical perfect land of mathematics, and ignore all the different implementations that are built for different areas of programming, for completely different uses. Donald Knuth understood the difference between theory and implementation - we should strive to recognize the difference between theoretical and implementation-specific advice.
It is no longer enough to simply ask someone if they are a programmer. Saying a programmer writes programs is like saying a scientist does science. The difference is that botanists don't design nuclear reactors.
-
Good article. :yes:
-
Lets not forget the fact that some programmers can't program (http://www.hard-light.net/forums/index.php?topic=68350.0).
-
While I agree programming covers a width arrangement of topics, I feel the author is somehow confusing a skill, e.g. programming, with a profession, e.g. programmer.
I find amusing he touches a similar example of this, "We are architects trying to tell English majors how to design an essay because we both use lots of paper.", but fails to find a parallel.
That is, we all know how to write and yet few of us call ourselves writers. Another example is many of us can do math to varying degrees, and yet few of us call ourselves mathematicians.
Unless he is, of course, saying people who know how to program shouldn't call themselves that, unless that is specifically what they do for a living. You know, like writers or mathematicians.
-
The problem outlined in the article is that the generic term "programmer" covers a lot of different skillsets and specializations. And while there are certain things common to all programmers (just as there are things common to all scientists, or musicians), it does not mean that someone working on, say, a business reporting application can seamlessly transfer to do operating system coding or vice versa.
The other big point he makes is that methodologies that work in one field are not easily applicable to others, even though the tools used are the same. If, for example, somebody working on a word processor is asking somebody who specializes in high-performance parallel computing for advice, it is highly probable that the methods the second person uses will be irrelevant to the first one, even though both use C++. Yet both are programmers, and yet, this literally happens all the time.
-
I'm not seeing the problem. One is an application developer the other is an HPC developer.
It's like complaining romance novel writers and technical book writers are both called writers even if they cannot do each other's jobs.
-
It's like complaining romance novel writers and technical book writers are both called writers even if they cannot do each other's jobs.
That is exactly the problem. Or rather, romance writers asking tech writers for advice and vice versa. A technique that works in HPC may be the completely wrong thing to do when developing an AJAX application, and yet, people keep trying to implement stuff that they've seen working in other fields, or heard about working in other fields, even though it may be the completely wrong approach to doing business.
The point mentioned in the article was about a function used wrong by many people who do not understand how it works (and probably do not need to understand how it works), but who got told by others that it should be used. It's an advanced form of cargo-cult programming, a trap that is very easy to fall into.
-
It's like complaining romance novel writers and technical book writers are both called writers even if they cannot do each other's jobs.
That is exactly the problem. Or rather, romance writers asking tech writers for advice and vice versa. A technique that works in HPC may be the completely wrong thing to do when developing an AJAX application, and yet, people keep trying to implement stuff that they've seen working in other fields, or heard about working in other fields, even though it may be the completely wrong approach to doing business.
The point mentioned in the article was about a function used wrong by many people who do not understand how it works (and probably do not need to understand how it works), but who got told by others that it should be used. It's an advanced form of cargo-cult programming, a trap that is very easy to fall into.
If someone doesn't use a function correctly, he's either a bad programmer or the documentation is faulty (or the third option of the function's call being retarded, but let's ignore that one).
If bad romance novel writers continuously mangled the language they use, we wouldn't modify the language to accommodate them. (Even though it seems that's how language evolution works... :rolleyes:)
This isn't a HPC vs normal programming thing or a contrast between some other different programming domains. Hence, that is not the problem. It's basically a non sequitur by part of the author.
-
You are either not reading my posts correctly or dodging my points.
-
Then please rephrase them, since I'm not understanding them then.
I'm not seeing the leap in logic of "Some programmers don't understand how to use this function." to "Programmers is a vague label.". To me the whole issue he presents in the beginning is totally unrelated.
-
Let me put it this way. If you asked a particle physicist to explain how to dissect a lab rat and he said he didn't know how would you say he was a bad scientist? Would you blame the physics textbook he's looking at? But both particle physics and animal biology are sciences. People who study them are both scientists.
In this case he can easily say "I'm a physicist, I know nothing about biology" and everyone will understand why he knows nothing about the subject. The author however is talking about the fact that we use the term programmer to refer to two jobs as different as the physicist and biologist. But worse that because we use the same term for both you get physicists explaining to biologists why their methodology is better and that the biologist should use it.
-
Not entirely sure what the big deal here is. Programmer is a general term for a profession, just like lawyer, doctor, farmer, etc. I don't think anyone was trying to claim that when you say you're a programmer, you're saying that you know literally everything about making computers do what you want.
-
You're missing the main point. Which is that even though it is a general word programmers too often assume that what works in their particular field is a general case which works for all programmers. The whole point of the article was that this simply isn't true.
-
Still not clear about things; every field has its specializations, why would people assume programming is any different? Especially with such a diverse array of things that need to be programmed in tremendously varying levels of abstraction, I'd think it's plenty clear that some situations call for different use of CS ideas, not that this isn't a CS idea in and of itself. The article stresses so much that "programmer" is a term insufficient in detail to adequately describe what someone does, but can anyone think of a single word to describe what they do without any ambiguity?
Anyway, that aside this is as good a spot as any to ask about any of your experiences with programming at the university level. The Electrical Engineering and Computer Science dept at my university only offers B.S.Es in computer science, computer engineering, and electrical engineering (ie no software engineering, networking, etc). You'd think that this wouldn't be fine enough of selection having read this article. Not that I'm in much of a position to comment, being just shy of halfway done. For those of you that have finished uni in a computer related field, how wide of an area did (or could with the electives offered) you cover for what degree?
-
So can anyone give an actual example of "a technique which works on a specific field that programmers assume work for all"?
Because what the author presented wasn't that at all.
-
Am I understanding correctly that the underlying problem is that some programmers think that because they know how to program perfectly well for their job, that everyone should do things the way they do, when in fact, programmers with other focuses need to do things differently?
-
So can anyone give an actual example of "a technique which works on a specific field that programmers assume work for all"?
Because what the author presented wasn't that at all.
For example, a programmer that specializes in object oriented programming, i.e. Java, writing interfaces for large systems like banks, would have absolutely no idea what to do when coding for embedded systems. Embedded systems require careful memory management and programming as efficient;y and close to bare metal as possible, usually in Assembly or ANSI C. Someone used to coding in Java wouldn't have spent any time managing memory, for example, as Java handles that for you; also, situations where you would generally be using Java are generally not ones that require realtime speeds, so they likely don't know much about optimizing for execution speed.
-
And what programmers assume that embedded systems' programming is not different from, say... application programming (which is what I assume you were trying to make as a counterpart)? Or are we just confusing programmers with people who dabble into programming?
-
Am I understanding correctly that the underlying problem is that some programmers think that because they know how to program perfectly well for their job, that everyone should do things the way they do, when in fact, programmers with other focuses need to do things differently?
Basically, this.
If an embedded systems programmer has never seen a piece of application code, or vice versa, then yes, they may believe that things that apply to one apply to the other. In all honesty, many of those who 'dabble in programming' (IMO) probably have a better view of the differences between types than someone who took a 6 month College course in, say, Embedded Systems, who went straight into a job writing embedded systems.
-
Let me try to present my argument in another form. If the programmer label is that vague and it's related to that "problem", how would making it less vague solve it?
In the case you presented we can already narrow down the specialization those programmers have and their areas. Hell, we just did, embedded systems programmer and "dude who dabbles into programming" (or for those of you who want a serious title, lets just call him applications programmer).
The fact that we already have names and labels for the various things we have been talking about would suggest that this labelling business is not the problem, whatever that problem is.
And coming back to the author, using the programmer label is not the same as using a scientist one. One programmer can with some training do the job of another programmer. The same can't be said of a physicist to a botanist.
P.S.
Joining Polpolion, I've got a Masters in Computer Science, where the course work was supposedly network heavy, my thesis was about AI/game theory, and now I'm trying to do research on OS architecture and trying to extend stuff about my thesis on my spare time.
-
Let me try to present my argument in another form. If the programmer label is that vague and it's related to that "problem", how would making it less vague solve it?
In the case you presented we can already narrow down the specialization those programmers have and their areas. Hell, we just did, embedded systems programmer and "dude who dabbles into programming" (or for those of you who want a serious title, lets just call him applications programmer).
The fact that we already have names and labels for the various things we have been talking about would suggest that this labelling business is not the problem, whatever that problem is.
And coming back to the author, using the programmer label is not the same as using a scientist one. One programmer can with some training do the job of another programmer. The same can't be said of a physicist to a botanist.
P.S.
Joining Polpolion, I've got a Masters in Computer Science, where the course work was supposedly network heavy, my thesis was about AI/game theory, and now I'm trying to do research on OS architecture and trying to extend stuff about my thesis on my spare time.
I suppose that's entirely fair. I guess what I feel is the part that maybe the article would have been better served to focus on would be the societal perception of the term "programmer". Those who don't know anything about programming would be quite likely to not know of the division in the discipline, and assume that all programmers are just programmers.
I'm in the 3rd year of my Uni's Bachelor of Computing degree (which has SoftEng and CompSci as sub-disciplines). EDIT: Thus, admittedly I have far less experience than you. Ours focuses on covering paradigms, in a way; they start us off in straight ANSI C, and then in second year give you Java, 68k Assembly, C99, Python, and PostgreSQL. I'm also currently in a course on Legacy systems, and in there we're work with Fortran, Ada, and Cobol, primarily modernizing say Fortran77 to Fortran95.
-
But don't we treat other disciplines the same way?
Physics is a large field and yet if someone says they're a physicist we don't bat an eye.
The same for biology, medicine, etc.
-
Physics are enough of a "dark art" that most people don't inquire further unless they're also interested in physics. Also, the "basics" are simple enough that you could expect most physicists to be familiar with them. I'm only running into "wrong specialization" problems when asking my teacher about really obscure/recent/complicated topics (well, I guess you shouldn't expect a high school physics teacher to be an expert astrogator, even if it's an elite school).
Programming is different, in that there's a lot of languages and specializations, sometimes completely unconnected. While you could expect a dentist to know how to patch up a leg wound, or from a microbiologist to have a general knowledge of how a digestive tract works, a programmer could easily specialize in C++, but know next to nothing about LUA of Java.
-
Programmers also have "basics" as you call them. Every programmer knows (or should know) basic flow control mechanisms or data structures for example.
Also, I assume that the more people are familiar about a subject, the wider they feel the area really is. Since everyone in this thread seems to have familiarity computer science, it's natural that we all feel that computer science is a vast vast place, while other disciplines may not feel as vast. And what you consider simple is also effected by your familiarity with the area.
Try asking your dentist about dermatology.
-
But that's the thing, dentists know not to try to give advice to dermatologists.
-
Programmers also have "basics" as you call them. Every programmer knows (or should know) basic flow control mechanisms or data structures for example.
On the other hand, the implementation of these basics can vary widely, much more than in physics. A better analogue for programming might be linguistics. There's a "basic theory", which includes how tenses, word gender, declination (a concept unknown in English and difficult to explain, but present in Latin and Slavic languages) and other such things work. The precise implementation, however, varies. For example, in Polish there are only three times, but you have to wrap your head around declination. Same with programming. The basic concept of a data structure might be the same across various languages, but how do you define it depends on the language.
But that's the thing, dentists know not to try to give advice to dermatologists.
Perhaps it's because medicine is a very old science, while programmable machines exist only for about 90 years, and that's including the Colossus (the very first one, used during WWII).
Programming as we know it (for a quite liberal value of "as we know it") exists for about 40 years at most. It's a science that didn't had too much time to evolve and set any rules in stone. Also, in medicine the specializations are quite clearly defined, not to mention it's nature means it has to be deadly serious. I would be less surprised if an astrophysicist tried to advice a nuclear physicist on how he should approach his research though. People who think "If you don't do it my way, you're doing it wrong." might happen in any science.
-
Programming predates computers.
Ada Lovelace comes to mind.
-
Programming predates computers.
Ada Lovelace comes to mind.
Well, programming sort of equals creating flowcharts for devices, following certain rules of communication (language) that the device / or compiler flowchart software on the device is made to follow.
So, I imagine that flowcharts have been around for quite a while, just now they are being used for machines, and machines aren't too aweful good at inferring what is actually meant (although, even with human intelligence, you still run into language barriers and interpretation issues, which is why you can get two different building from the same set of blueprints if two different architects build them.
Perhaps what the problem is is that some programmers are a bit closed-minded? Experts in any area can become that way. Life is a process of learning, and even when someone is an expert, that does not entitle them to close their mind to new information and theories and simply spout their brilliance. Even if they are 100% correct, there are still things to be gained from learning others' perspectives of the same truth. In addition, it is hard to learn from someone who does not try to break things down to your level and instead, simply lectures on 'the way things are' without bringing you through at least an outline of how they came to that conclusion. (I suppose, sometimes a teacher doesn't have enough time to even give a bare skeleton of the reasoning behind their teachings, however, one would hope that at least there would be references in the textbook / lesson notes).
...Rambling, sorry. :nervous:
-
One programmer can with some training do the job of another programmer. The same can't be said of a physicist to a botanist.
I think you are oversimplifying it quite a bit. For example, imagine you'd been writing whatever scripts for web sites the last 10 years, and then apply for a job as hardware driver developer for modern GPUs. The botanist could well have a degree in physics before you'd be producing something useful..
-
I don't think think it's that big of a difference, although to be honest, I can't conceive how a programmer cannot have experience with lower level languages, even if it's just making an hello world program.
Maybe I expect programmers to be (much more) educated and held to a higher standard.
-
i approach programming from a hobbyist point of view. but im just as at home programming a mirocontroller in asm as i am programming a game engine in lua. and even thats not enough, i have to get into how a cpu works, how digital logic works, how to build a flipflop from discrete transistors or how to use logic gates to do math. i probibly dont count as a programmer because im not formally trained in that discipline and i do not work in that field. im not really all that intrested in web programming or java, but im pretty sure i could pick it up if i ever had the desire to, or a project demanded it (internet accessible robots anyone?). so im not a programmer, but i do have a knack for writing code that works.
-
One programmer can with some training do the job of another programmer. The same can't be said of a physicist to a botanist.
I think you are oversimplifying it quite a bit. For example, imagine you'd been writing whatever scripts for web sites the last 10 years, and then apply for a job as hardware driver developer for modern GPUs. The botanist could well have a degree in physics before you'd be producing something useful..
I don't think think it's that big of a difference, although to be honest, I can't conceive how a programmer cannot have experience with lower level languages, even if it's just making an hello world program.
Scripting languages are as different to hardware drivers as it's possible to get!
Nuke is very good with the Lua scripting language used in FSO, yet I bet he doesn't think he could learn the necessary to write notable parts of a functional and stable hardware driver for a modern GPU in a year.
I've also got experience in both low-level MCU and high-level Windows application development - and I know I couldn't write notable parts of a modern GPU driver without several years of training.
This is exactly the kind of thing that the original article is talking about.
You have stated the belief that scripting languages and hardware driver development are sufficiently similar to cross-train someone in a short course.
This is bunk, as anyone who has done extensive work in either field should know.
However, unfortunately the media, and many others who ought to know better still hold that belief - and because they repeatedly say that kind of thing, the general public come to believe it as well.
The core ideas do transfer - arrays, loops, decisions and pointers (Except that most scripting languages don't have pointers!) - but all the detail is different.
A good idea in one context in utterly insane in another - eg recursion on an MCU, directly accessing hardware in an application.
-
One programmer can with some training do the job of another programmer. The same can't be said of a physicist to a botanist.
I think you are oversimplifying it quite a bit. For example, imagine you'd been writing whatever scripts for web sites the last 10 years, and then apply for a job as hardware driver developer for modern GPUs. The botanist could well have a degree in physics before you'd be producing something useful..
I don't think think it's that big of a difference, although to be honest, I can't conceive how a programmer cannot have experience with lower level languages, even if it's just making an hello world program.
Scripting languages are as different to hardware drivers as it's possible to get!
Nuke is very good with the Lua scripting language used in FSO, yet I bet he doesn't think he could learn the necessary to write notable parts of a functional and stable hardware driver for a modern GPU in a year.
I've also got experience in both low-level MCU and high-level Windows application development - and I know I couldn't write notable parts of a modern GPU driver without several years of training.
This is exactly the kind of thing that the original article is talking about.
You have stated the belief that scripting languages and hardware driver development are sufficiently similar to cross-train someone in a short course.
This is bunk, as anyone who has done extensive work in either field should know.
However, unfortunately the media, and many others who ought to know better still hold that belief - and because they repeatedly say that kind of thing, the general public come to believe it as well.
The core ideas do transfer - arrays, loops, decisions and pointers (Except that most scripting languages don't have pointers!) - but all the detail is different.
A good idea in one context in utterly insane in another - eg recursion on an MCU, directly accessing hardware in an application.
I didn't say they were similar, I said that I can't believe any programmer doesn't have experience with any low level language.
For instance, Nuke IIRC, said somewhere he programed micro controllers.
And if you know the core ideas, as you called them, the hard part is just checking references and the respective documentation. Sure, you have to be careful with some stuff because of optimizations and other limitations, but that's with practice.
The original article states that these core ideas are not known. The issue he first tries to tackle is that apparently some [sic]programmers don't know what a heap is.
-
I didn't say they were similar, I said that I can't believe any programmer doesn't have experience with any low level language.
I have no experience with assembly. Many other self-taught programmers will also have no experience with it for the same reason I don't, they didn't see any point in learning it.
The original article states that these core ideas are not known. The issue he first tries to tackle is that apparently some [sic]programmers don't know what a heap is.
I didn't have any need to learn what the heap was for the entire time I was a Java programmer. It so happens that I actually knew what the term meant before I even started learning Java but while I was coder in that language it was nothing I had any need to know.
-
I believe that every programmer *should* have experience with at least one lower level language, like ANSI C, but that doesn't mean that every one does. Someone who only does GUI programming in say Java would have no explicit need to know much about low level data structures and the like short of knowing how to call their library functions. Even the most basic structures, like lists, are handled by a built in data structure like ArrayLists, that require no knowledge whatsoever of the low level code behind it to use it. Why would a Java programmer, for example, feel a need to explore how to build linked lists if it's always been abstracted away?
-
I didn't say they were similar, I said that I can't believe any programmer doesn't have experience with any low level language.
I have no experience with assembly. Many other self-taught programmers will also have no experience with it for the same reason I don't, they didn't see any point in learning it.
The original article states that these core ideas are not known. The issue he first tries to tackle is that apparently some [sic]programmers don't know what a heap is.
I didn't have any need to learn what the heap was for the entire time I was a Java programmer. It so happens that I actually knew what the term meant before I even started learning Java but while I was coder in that language it was nothing I had any need to know.
I apologize for you not knowing the key data structure Java uses. And you know, how the garbage collector works and stuff...
Also, a programmer using Java should know how a LinkedList and an ArrayList work because of efficiency reasons. If you are trying to repeatedly access a LinkedList by indexes for instance...
I think I need to repeat myself, not everyone who programs is a programmer, just as not everyone who writes is a writer and not everyone who does math is a mathematician. And just as there are examples of bad writers, there are examples of bad programmers. Hell, there are whole websites (http://thedailywtf.com/) dedicated to this stuff.
-
I did write and post a response but you know what? **** it.
The rest of us have understood the point under discussion. If you wish to flail around in ignorance and display contempt to those who are only trying to explain something to you, so be it.
-
I'd like to do some flailing, actually. Thanks for giving me this opportunity.
I didn't say they were similar, I said that I can't believe any programmer doesn't have experience with any low level language.
I have no experience with assembly. Many other self-taught programmers will also have no experience with it for the same reason I don't, they didn't see any point in learning it.
The original article states that these core ideas are not known. The issue he first tries to tackle is that apparently some [sic]programmers don't know what a heap is.
I didn't have any need to learn what the heap was for the entire time I was a Java programmer. It so happens that I actually knew what the term meant before I even started learning Java but while I was coder in that language it was nothing I had any need to know.
1) Experience with assembly in and of itself is exactly 0% helpful when writing applications, for the most part. Knowing how your computers work, on the other hand, can pretty useful if you know enough about your compiler to know what it will do with it. Most of the time that happens to come from knowing assembly or having taken a computer organization course at one point or another. Still, compilers work well enough for you to not care about this unless you really need to squeeze out the performance.
2) The article wasn't talking about the heap, it was talking about a heap. Though I could certainly see the confusion the make_heap function would cause if someone knew what the heap was but not what a heap was. :p
I believe that every programmer *should* have experience with at least one lower level language, like ANSI C, but that doesn't mean that every one does. Someone who only does GUI programming in say Java would have no explicit need to know much about low level data structures and the like short of knowing how to call their library functions. Even the most basic structures, like lists, are handled by a built in data structure like ArrayLists, that require no knowledge whatsoever of the low level code behind it to use it. Why would a Java programmer, for example, feel a need to explore how to build linked lists if it's always been abstracted away?
When programmers write a data structure abstraction to be used by everyone everywhere they just guess what kind of workload it will have and choose the most optimal structure for that workload. With a linked list, if you spend all your time doing random accesses you're wasting loads of time. That's the problem with abstracting things. Unless you know exactly how it's going to be used, someone is going to be using it sub-optimally, when a different structure would work much better. Every programmer that has a degree should have taken a data structures and algorithms class at one point and should know generally which structures work when, but if they're stuck with a mystery box structure that doesn't work so well then they can't be helped. So really the only time you should be using abstracted structures is if performance isn't your biggest concern. tl;dr different situations calling for different approaches is why abstraction can't work in the most optimal way.
So if you don't know how to write optimal Java code and you're a java coder, you definitely have things to learn that require breaking abstraction.
-
I'd like to do some flailing, actually. Thanks for giving me this opportunity.
Since you asked nicely. :)
1) Experience with assembly in and of itself is exactly 0% helpful when writing applications, for the most part. Knowing how your computers work, on the other hand, can pretty useful if you know enough about your compiler to know what it will do with it. Most of the time that happens to come from knowing assembly or having taken a computer organization course at one point or another. Still, compilers work well enough for you to not care about this unless you really need to squeeze out the performance.
I won't deny that there are times when I've found knowledge of what goes on under the hood very useful. What I find objectionable is that you need that knowledge in order to be a good coder. Or that everyone who is a programmer has it.
Besides if you're at the point where you're playing around with compiler flags in order to squeeze out performance in Java, maybe you need to think long and hard about why you ended up in that position in the first place. :D
The article wasn't talking about the heap, it was talking about a heap. Though I could certainly see the confusion the make_heap function would cause if someone knew what the heap was but not what a heap was. :p
That should teach me to re-read the article again before replying to it. Still the point about abstraction is still valid.
As is my point that sometimes a little knowledge can actually be dangerous. I've seen no end of times when someone has written a convoluted function because they wanted to optimise the speed based on some abstraction-breaking rule they knew. And of course when you profile that function against the one written by someone ignorant of that rule you find the speed difference to be minimal or even that the complicated function is actually slower.
When programmers write a data structure abstraction to be used by everyone everywhere they just guess what kind of workload it will have and choose the most optimal structure for that workload. With a linked list, if you spend all your time doing random accesses you're wasting loads of time. That's the problem with abstracting things. Unless you know exactly how it's going to be used, someone is going to be using it sub-optimally, when a different structure would work much better. Every programmer that has a degree should have taken a data structures and algorithms class at one point and should know generally which structures work when, but if they're stuck with a mystery box structure that doesn't work so well then they can't be helped. So really the only time you should be using abstracted structures is if performance isn't your biggest concern. tl;dr different situations calling for different approaches is why abstraction can't work in the most optimal way.
So if you don't know how to write optimal Java code and you're a java coder, you definitely have things to learn that require breaking abstraction.
But do you? Why is it breaking abstraction to simply learn linked lists are slow for random access but good for sequential access? I'm not saying knowing that stuff isn't useful, it certainly helps you remember why it is slower to insert in the middle of a vector compared with a linked list. But why do you have to know anything beyond whether an operation is O(n) or O(1)? Why do you have to understand the internal structure?
Edit : some days I really should proof read. :rolleyes:
-
As is my point that sometimes a little knowledge can actually be dangerous. I've seen no end of times when someone has written a convoluted function because they wanted to optimise the speed based on some abstraction-breaking rule they knew. And of course when you profile that function against the one written by someone ignorant of that rule you find the speed difference to be minimal or even that the complicated function is actually slower.
Yeah. The thing about this is that if people do these crazy optimization we're talking about and get a good change in performance they're usually either in the main part of the program or in really common functions, which should scream "reconsider your design" to the people trying to do the optimization.
But do you? Why is it breaking abstraction to simply learn linked lists are slow for random access but good for sequential access? I'm not saying knowing that stuff isn't useful, it certainly helps you remember why it is slower to insert in the middle of a vector compared with a linked list. But why do you have to know anything beyond whether an operation is O(n) or O(1)? Why do you have to understand the internal structure?
You really don't have to know exactly how it's implemented, but personally I'd say that if things are implemented in such a way that you yourself need to remember how to access your data in the structure then you're breaking the abstraction. If people want to index my linked lists they can write their own functions to do it.
-
I'm not saying you have to remember how to access the data yourself, just that when you're picking between a vector and a linked list all the information you really require is how fast they are at certain tasks. That sort of information tends to appear in the documentation. For instance, the docs for ArrayList here (http://docs.oracle.com/javase/1.4.2/docs/api/java/util/ArrayList.html).
The size, isEmpty, get, set, iterator, and listIterator operations run in constant time. The add operation runs in amortized constant time, that is, adding n elements requires O(n) time. All of the other operations run in linear time (roughly speaking). The constant factor is low compared to that for the LinkedList implementation.
Why would I need to know more than that?
-
For what you are asking, most probably not. But if a someone knows what O(n) is, most likely they are not the ones the article author is referring to.
I apologize for my tone earlier, sometimes it's hard being the only one on one side of the discussion.
-
Fair enough. I understand it's easy to lose tempers.
When it comes to stuff like which collection class to choose though, you don't even need to understand O(n), just that one is better at one thing while the other is better at something else will get you pretty far.
-
Eight years of teaching myself how to code in as many mainstream C-like languages as possible, and I'm already considering diverting my college major altogether after reading this thread. And I haven't even graduated high school yet >_>
On the topic of the original post: I've noticed this in the coding community like myself, but I wonder if the confusion doesn't arise from the fact that the same basic skill is shared across the board: we all have to know how to create an abstraction of a problem so that we can solve it in a precise, logical manner, whether we're scripting in PLSQL for a giant corporate database or sitting in our parent's basement, pounding away at the keyboard to create a Java library that can perform arithmetic operations on dates and times.
Perhaps, because we all have the same core approach to figuring out a problem (abstraction), we think we're all equally qualified to do each other's job, even if we're not specialized the way they are?
-
The problem here is that there is no end to programmers that foolishly insist that you just need to read the documentation to know how to code something. This is demonstrably false as soon as you start solving anything other than easy problems. There is no way anyone with no experience in graphics can do my job, and I know this is true because I've seen idiots attempt it and fail miserably over and over and over. There is an endless stream of morons who continue to try to tell me to do stuff that simply doesn't work because they aren't aware of memory bandwidth bottlenecks or cache coherency. No programmer can just go and write a lockless data structure without an incredibly deep knowledge of low level CPU operations because you have to know where to enforce memory barriers to prevent the CPU from re-ordering instructions in a way that breaks the data structure. This isn't stuff you can just pick up in a month. I've tried. All I can do is write a one-sided locking mechanism without using semaphores, which is pathetically easy in comparison to writing an entire lockless data structure. And then there's the fact that lockless data structures are only sometimes useful and you have to know when to use them properly. This stuff matters.
That didn't stop approximately 300 people from telling me that I am a complete idiot and should never program anything ever again.
-
Wait, you also have graphics skills? :eek2:
You never cease to amaze me, Blackhole.
Are you a modeler, or a 2D artist? Because if your art is as good as your music, you could really help some projects in here.
-
By "graphics" I mean I'm a graphics programmer :P I'm a terrible artist in both 2D and 3D. I can do some manipulation, but that's purely because I know the inner workings of all the graphical theory behind the programs.
I'd be too busy to help, anyway.
-
Ever considered tackling SCP? If your coding is as good as your music, FSO would look better than X: Rebirth in no time. :)
-
I actually did try once. I went in and attempted to upgrade DirectX only to be repulsed by the horrendous Cthulhu-code monstrosity that still haunts my nightmares. It would quite literally be easier for me to just write an entire new engine than try to fix that mess. Also I hear you have another graphics programmer putting in stuff like shadows already. Plus, still don't have time to do it anyway.
-
blackhole, maybe it would be easier to port the functional OpenGL code that we have into a new DirectX code? :lol: Ok, ok, I don't know what I'm talking about, just throwing the idea out there.
-
I'd have to look at the code again. The last time I looked at it was 5 years ago. This is unlikely to happen due to me being entirely too busy to help with the project anyway.
-
blackhole, maybe it would be easier to port the functional OpenGL code that we have into a new DirectX code? :lol: Ok, ok, I don't know what I'm talking about, just throwing the idea out there.
I don't think that's a good idea because FSO is cross-platform.
DirectX is only useful on Windows and XBOX platforms (and possibly Windows Phone if anybody cared) - ok, WINE support is getting quite good but still.
OpenGL does everything (graphical) that DirectX does and is also cross-platform.
Thus re-implementing DirectX would be a waste of a good graphics programmer. (To be honest, unless you're targeting the XBOX I think even learning DirectX is a waste of time these days.
However, updating the OpenGL implementation, cleaning it up, removing the deprecated calls etc?
Now that's worthwhile - and will probably result in notable performance improvements for everybody.
-
If the openGL already exists I tend to agree with Tomo. I personally prefer directX over openGL because its less retarded, but I usually think most graphics engines should implement both for a variety of reasons. It's still irrelevant because you guys will never get me to help :P
-
how is direcx less retarded to opengl? directx has has piss poor reverse compatibility, ms doesnt want to support more than one or two generations of legacy applications. and on top of that ms has been deprecating parts of it and replacing it with lesser code from the xbox line (like direct input, getting replaced by xinput, not really graphics related but is still a major issue with me). i have not touched direct3d but opengl is elegant and easy to learn. i have not seen any of this retardation that you speak of.
-
Meanwhile OpenGL has a crapton of legacy cruft too. This answer (http://programmers.stackexchange.com/questions/60544/why-do-game-developers-prefer-windows/88055#88055) is more of a history lesson but it points out a lot of the issues I had with openGL back when I started building my engine in DX9. Since modern engines now use DX10/DX11 vs. openGL 4, I don't know which issues have been solved and what new issues have arisen, but I have little faith in either openGL or directX, since both of them have certain issues that keep coming up and making things extremely annoying, and they are almost never addressed, or they are addressed very badly. DirectX still ends up being easier for me to use due to its object-oriented nature meshing nicely with my engine design, which is why I prefer it over openGL. If you don't, then more power to you. I don't really care.
-
id rather have a bunch of legacy cruft than piles of old games that i can only run on a hand full of operating systems. i have a distinct loathing of directx 5,6,7 and sometimes 8 games, because of how many bugs they have when you try to run them on a modern operating systems. i have an entire collection of games that i just cannot play because they were made for those versions of direcx. i can run even older opengl and even glide games just fine. not to say ive had any issues running 9 through 11 games. i just dont trust ms to preserve more than a couple years (or a couple windows versions) of backwards compatibility, while opengl has proven itself reliable in the long term.
any other reasons i might not like directx, really have nothing to do with d3d at all, but the other aspects of directx (like the deprecation of direct input for the inferior xinput, as i mentioned earlier). im sure both have their quirks, so im not looking to start an api war. i should also add that i played glide games long before i ever played d3d or opengl games.
anyway im gonna read that article because im sure its got sone important stuff in it, but the second it starts being biased im closing it. :D
meh it was an ok read. i can say that i like opengl for the same exact reason why everyone else hates it. opengl is still important, because without it all those other platforms wouldnt have an api (i doubt ms is going to make any api open enough to be used on all platforms). il just stick to opengl and sdl for all my 3d development in the near future.
-
The reason I think you're daft to learn DirectX (Direct3D) instead of OpenGL is very simple - it's the small things.
Portable devices like phones and tablets almost exclusively use OpenGL ES 2.0, which is Open GL with nearly all the legacy cruft removed.
Thus if you learn OpenGL ES, you've learnt the vast majority of what you need for all PC (Windows, OSX and Linux) environments and the rather large mobile segment.
- Yep, you heard right. In theory, it should be possible to port FSO to your phone.
(Whether it's feasible or anyone has the time to do it is quite another matter.)
-
Obviously if you want to be portable you should be learning openGL. That doesn't mean openGL is easier to use or better designed than DirectX. It's just a reason to suck it up and learn all its stupid, ****ty ways of doing things because it works on everything.
It's kind of like using Java instead of C# because you want to torture yourself with its incredibly bad tools so your program works on everything.
-
Just try out both and let us know what ya think of it. Done deal. :yes: