Extracting user requirements: Like pulling teeth?

What is missing in the

What is missing in the discussion is consideration of prototyping or simulation software to quickly and accurately identify requirements. Particularly (but not only) for web app development iRise and Simunication both offer very useful software (and the latter also gives option to build online as a service). I think simulation is the wave of the future in dealing with requirements definition.

Just checked back to see how

Just checked back to see how this conversation, looks pretty good.
However, I must challenge the "Business Requirements change constantly" statement. In stating that, you are saying that all Business Requirements are the same in form or shape, which is absurd. Business Requirements vary in their purpose and perspective, and I classify them into at minimum the following categories:
- Business Processes
- Business Rules
- Function / System Use (Use case)
- Data Requirements
This is a ranked list, from the requirements that change the most to the ones that change the least.

Business Processes change frequently: new steps, re-ordering of steps, new restrictions. If you are building today's business process into your information system, you have built an instant legacy system. The business will start to work-around your system almost immediately. This is what workflow and and business process management systems are for. They may be computerized, but they are not 'Information' Systems.

Business Rules change less frequently, but often enough that they now have their own automation systems, known as Business Rule Engines. Do a web search to learn what you need to know about those.

That leaves your core Information system requirements, the data you store and the main functions that CRUD the data. These do change, but not very often. Creating a new customer with name and address is the same now as it was 20 years ago and will be 20 years from now.

So, understand your Business Requirements and that development of software and databases is not the only method available in meeting those requirements.

I think the role of a

I think the role of a business analyst in an enterprise is a must. What happen usually is that we let IT professionals like Systems analyst, programmers, designers, IT Architects and so on to deal directly with the users to get/ identify their requirements. However, the business analyst will know in advance all processes and will identify exactly the system requirements in a way that IT professionals will be able to understand and create an excellent RFP. Furthermore, the business analysts are also able to foresee any requirements which may appear in the near future as well as to classify the requirements in phases based on business priority.

One of the issues is the

One of the issues is the confusion between technical requirements and business rules as requirements. Management of these needs to be different with business rules being treated as "first class citizens" as recommended by Daryl Kulak in Use Cases. These business rules typically change continually and are the source of many of the business' complaints that their systems never do what they need. Better requirements management will not help - only empowering the business to collaborate with IT by letting them manage their business rules in a controlled way.
This obsession with trying to "fix" requirements has been going on for years. It simply does not work.

Great comments all but

Great comments all but everyone failed to mention the obvious. It seems business requirements change constantly. End users know this and developers know this. Both parties struggle with the frustration created by such an environment.

My experience has been that

My experience has been that the business user knows what they need inside their heads but are not always very good at organizing the information or articulating those needs. Business analysts serve the essential role of helping the business to clearly define the requirements at an appropriate level of detail. Tools and templates help to structure requirements but does not ensure higher quality. You can still populate a template with ambiguous requirements.

There is a false assumption

There is a false assumption that this line of reasoning is built on, namely that users are capable of defining a set of requirements for developers to implement. They aren't going know exactly what they want until they see it. Any attempt to fix this problem that does not accept the interative nature of development will fail.

You mention developers and

You mention developers and users, but not the key role in gathering, analysing and documenting Requirements: the Business Analyst.
Long derided and misunderstood, Business Analysts are now coming together to bring rigor and recognition to their profession through the International Institute of Business Analysis ( www.iiba.com ) If you want an external view of Business Analysts, see
http://www.eweek.com/article2/0,1759,1241173,00.asp (an eweek article). Developers are skilled as developers, so they should focus on doing that using Requirements documented by a Business Analyst as input.
As to some of your points:
- most Requirements Tools have been fairly useless list managers with change control, but some new products are coming into play, some of which I saw at the latest Project Management and Business Analysis World conference in Toronto. The new focus seems to be making use cases more rigorous so you can generate tests or simulations with them. Of course, if IT hadn't abandoned tools like IEF and IEW in the nineties, we would not still have this problem today.
- as for users who are too busy (or their management won't allow them to be involved), then the business unit does indeed not deserve any IT support. As we are constantly reminded, IT staff are expensive and should only work on projects that the business has committed to

Part of the problem is that

Part of the problem is that "requirements management tools" don't really solve the challenge of "requirements definition." They'll manage any requirements - good or bad.

I liked your use of the word "extracting" and would suggest the following additional activities: asking, digging, wrenching, pulling, cajoling, wringing, bargaining, negotiating, begging, pleading, arm-twisting...(I could go on). Getting the right requirements is harder than it would initially appear.

At the end of the day, words (when used alone) are a cumbersome and abiguity-ridden approach to define systems.

Users, whether they agree on requirements or not, don't really understand what they're getting until they try it. You can't try a document or specification. Generally, the first time users can try something is with alpha software - or, worst case, at UAT time. Generally, it's at a point where it's prohibitively expensive to make the changes end users would want to see.

iRise provides non-technical users with the ability to create a realistic simulation of the software they'd like built. It allows end users to test-drive an application before coding begins. What if you could go through several iterations letting end users try a new application, with rapid feedback loops, before coding ever began? This early prototype then becomes a visual agreement between business and IT for what the users really need.

If you're interested in the concept of letting end users test drive their applications before you build them - check out a visual approach to application definition. Have a look at iRise (www.irise.com).