Industry


Ads by TechWords

See your link here


Joyce Carpenter's picture
Joyce Carpenter

Philosophy of Technology

Developers, it's all your fault

Grady Booch, an IBM fellow and inventor of UML, is using his position to preach the gospel of moral responsibility to software developers. During an interview with Charles Cooper at Cnet, he says that it's not simply a question of whether or not something can be built, but whether it should be built.

Good point, so far as it goes. And it doesn't go very far.

He uses the standard analogy of nuclear power and nuclear weapons to remind us of technologies that may not be morally neutral, although he doesn't commit himself to the view that they're morally bad. He only commits himself to the possibility that their moral status is open to question. Of course, it is. So is the moral status of every piece of technology. A nuclear weapon may throw the issue into high relief, but it does not tell me that the inventors or builders of a nuclear weapon are morally blameworthy. If he thinks they are, he should say so.

According to Wade Jolson, Booch referenced consulting at the Department of Defense during a speech in September. Apparently Booch doesn't stop himself from working for the war industry. One wonders, then, where he would set his personal limits.

And according to Booch, it is an individual's responsibility to form the questions and the answers on these topics. He told Cooper that "at the ultimate level, the software developer can say, 'Do I want to actually build a system that potentially could violate human rights?'" What doesn't "potentially" do harm? Too many marshmellows may lead to diabetes, even if they are fluffy and delicious.

A reasonable person might think it's okay to write software that controls the video camera at the convenience store, which gives the clerk a false sense of safety. But when the same software is used to track individuals on public streets, is it okay? Booch raises the question -- which I don't deny is an interesting one -- but gives no hint of the answer. Nor does he tell us whether the original developer should have foreseen the problematic usage of the software.

Booch jumps from technologies to individuals without stopping at the organizations or collections of people who commission  the projects he questions. Why is it each developer for him or herself? Why is there no guidance as to principles that individuals and organizations should follow? Are there any clear cases of immoral software? Any clear cases of software that can't be used to harm others?

Booch has the visibility to get people thinking. Can he use it for good?

What People Are Saying

Grady, thanks very much for

Grady, thanks very much for the reply.

I agree that raising the issue of responsibility is important. And thank you for using your position to do that. However, I don't agree with your focus on persons rather than organizations, and I would very much like to know what the principles are behind your decision making processes.

Whether you are for or against some particular practice is interesting, but I think the education comes from an analysis of the reasoning behind those positions. You are generally for defense; not so much offense. Why? Should developers not work on offensive weapons systems? Sometimes the same weapons systems can be used either way, so should developers not work on any system that could have an offensive use? What would you say at the meeting where the proposal is under consideration?

I can hear someone objecting, "It's up to the developer to decide." I don't buy that cop out. If there are reasons for rejecting the offensive weapons development project, let's hear 'em.

You say that you're opposed to technology that erodes personal rights. I wonder how you know in advance which technologies those are. Surveillance systems, for example, might be like weapons that can have both good and bad uses. The surveillance system you use to check on your cats might be pretty similar to that used in London where you worry about it being used to "track individuals as they go through the city." Is one technology acceptable and the other not? Is it the technology that's questionable or just its use?

And finally I disagree with one last thing. You say that just talking is not enough. Sometimes it is; I don't necessarily need to know about your actions. And to be honest, whether or not you have always acted according to your own principles is none of my business. I'd be amazed to meet anyone who had a perfect track record. I can argue with and learn from mere mortals (like myself). Thanks for the argument. :-)

I was just looking at an advance copy of our Monday print edition, which contains The Grill: Software guru Grady Booch. I was hoping one of my colleagues might have raised some of these issues, given our cover story, Ethics in IT: Dark secrets, ugly truths -- and little guidance. Great interview, even if it's not on topic here.

Joyce, I very much

Joyce, I very much appreciate your comments; you and some of the posters - quite legitimately - called me on a couple of issues: do i put my values in action? would i be more precise about my stand?

first, keep in mind, that what you read was just part of a longer interview, and thus you've seen only a part of the dialog and thus part of my position and actions.

second, as one commenter pointed out, a teacher has to begin somewhere, and that place for me is just raising personal responsibility as a relevant issue.

third, also as various people here have pointed out, just talking is not enough; my actions must speak. so, to be clear, i have personally chosen to not engage in certain projects that i deem immoral (and sorry....the vast bulk of these are classified projects); i support a variety of defensive efforts; i'm less keen on offensive one. in the commercial world, i have tissue rejection for projects that are of the let's-make-piles-of-money-and-flip-an-ipo, but rather have an affinity for projects that work to build enduring value.

fourth, let me be clear also that the argument that the individual has no power (for someone else will do it) is vacuous. in that light, mikem is right in that there is a clear arrow of scientific progress (read, for example, fukuyama's "end of history and the last man") and that arrow is irreversible...but mike, for me, it is the essence of the human experience that we can (and in some cases should) seize the decision to not do something. i'm in favor of stem cell research; i'm opposed to technology that erodes personal rights. so, a critical point here: you and i may disagree about we should do/not do, but the very fact we can have a dialog...that journey, that discussion may be more valuable than our end decision

Technology does not erode

Technology does not erode personal rights. It only provides the means to do it. It also provides the means not to do it. The reason I say it's a "social" problem is because I cannot think of any tool that cannot be used both for good and bad. Name some "technology" that can only be used for good or only be used for bad.

"Good" or "bad" are social values and can only be answered in social terms. I believe in the yin-yang aspects of reality, and thus if you abandon or refuse to develop a technology for fear of its misuse, then you also lose all possible uses of it.

Knowledge is like a sphere of light. And that is why myths like Frankenstein and expulsion from Eden endure. No matter how much brighter the light gets, the darkness gets even pi times bigger. Ignorance increases with knowledge. But letting the light of knowledge die means the darkness encroaches which is even scarier.

Booch is wrong, wrong,

Booch is wrong, wrong, wrong. If something _can_ be built, then it _should_ be built.

Unless you are the only person in the world who could possibly think up and build something, then someone else will. Even if the US had never built the A-bomb, someone else would have. (And note that the reason the Manhattan Project was on a fast timeline was because we knew the Nazis were building one.)

Or what about the US Republican pogrom against stem cell research. Yes. Let's shut all that down and pray to God. Keep your eyes on the Bible and not on the Chinese who will find cures for diseases using stem cell research and then probably get money from rich, old Americans who will give _everything_ they own for another year of life. But at least one-half of the government got to live by its principles.

Technology cannot be put back in a box, and unless you actually are the only person in the world smart enough to think of building something, then it will get built.

The real question, the one Booch refuses to talk about, is social. Just because we can do something, should we? That is unanswerable in any sort of fundamental philosophical terms, thus it can only be answered in terms of real situations. Should we bomb Afghanistan even farther back into the Stone Age? Should we drop the Twin Towers of America to the ground?

A hammer is a tool. I can build things with it or tear them down. I can hit nails with it or I can beat people to death with it. The use of a tool is independent of its development. We should always build all tools just so we can discover the pros and cons of them. Unless you can envision some ideal reality where we can control the actions of _everyone_ else.

Anonymous, I agree that good

Anonymous, I agree that good teachers don't provide answers, but rather they get students to find answers themselves. However, good teachers have to do enough to point the students in the general direction of an answer. I don't think Booch is doing that so far. I hope he will.

RCA, I have to wonder whether you aren't countering your own position. If the average philosophical maturity-level is so low, and a teacher must teach to the student's level, then maybe Booch is starting at the right level after all? ;-)

Thanks for the comment! Very nicely put.

A teacher, or rather a good

A teacher, or rather a good teacher, does not pose questions for his students and then answer those questions for them. A good teacher puts forth problems, then leaves their solution as an exercise for the student. More vexing perhaps, there may not be a "right" answer. That's the price of personal growth.

Indeed, it is preferable for

Indeed, it is preferable for a good teacher to provide their students an opportunity for an "Aha!" moment. However, epiphanies do not come easily, and without insightful guidance they can be specious or logically inconsistent.

A good teacher needs to teach to the philosophical maturity-level of his audience, and unfortunately, the philosophical maturity-level of the general public is puerile at best. When given only questions with no guidance, many are incapable of logical conclusions in isolation from the wisdom of those who have contemplated the matter at hand at great length.

A good teacher, therefore, needs to provide insights which stimulate the student to discover truth, either by accepting or rejecting the teacher's propositions.

It is quite easy to recognize problems. Just listen to water cooler or coffee shop conversations and you will quickly realize that recognizing problems and complaining about them is lightweight stuff. Wisdom, however, is quite precious and rare.

Grady Booch should have been able to offer, if not answers, points to ponder. To decry a situation and not be prepared to offer insights to a solution is gratuitous at best.

I concur that just because

I concur that just because something "can" be done, does not imply that it "should" be done. A cogent case in point is the recent announcement from Microsoft that they are developing technology to allow everyone to store their personal medical records at a Microsoft site where this information can be shared with virtually anyone. The intent is to make your records readily avasilable to all of your medical and insurance providers, with no effort required to transfer records should you change providers.

I can't imagine what the strategists at MS were smoking when they came up with this one! It is a very bad idea for several reasons, but chief among these is the privacy implications! Can you say "HIPAA"?

With hacked sites and stolen personal information in the news almost daily, who is going to believe that Big Broth... I mean Microsoft can verify the absolute security of everyone's medical records. And this idea will only be practical if MS can convince the vast majority of Americans to use this "service." My strong feeling is that few will trust it enough to use it.

Even if MS does get a critical mass of users, it would only take one publicized incident of lost and/or stolen medical records to cripple the system, and indeed cripple Microsoft's viability as a company! I truly believe that they would be "betting the farm" on this proposition because of the HIPAA laws (and ensuing fines) and a loss of public trust in the company!

Just because you can, Microsoft, does not mean you should! This is an extremely foolish idea!

word

word