Recent experience at a client showed me that Agile can be successful for a project, but maintaining a level of success across projects and organizations is difficult.The client had one team using Scrum that was delivering sucessfully, but several others that were not.
A manager of one of these other teams told me that their biggest problem was turnover in the Product Owner role; a business person would take on the role, get better and better at it, then change jobs. The team would have to acquire a new person and start from scratch again.
The other most common problems were scope creep, and excessive change during dev sprints in a project. To me, these are another sign that the Product Owner role was not being performed well. Overall, the discipline needed to make Scrum effective was not being practiced.
The client's response to these problems was to focus on a process for requirements definition, to have stable and clear requirements available for use by a dev team, irrespective of the development process that team may use. Yes, this can be seen as a step backward from an Agile point of view, but what else could the client do?
As a consultant involved in implementing the new requirements process, I have to acknowledge that some readers would say I am biased against Agile/Scrum. What I am biased against is any process that is not working. The client's original use of Scrum is certainly before my time, but it can be safe to assume that it was done for good reasons, one of them likely being that up-front requirements was not working for them.
Over time, the above agile fragilities became apparent and something needed to be done. Rather than try to improve their use of Scrum, the client looked back to its original problems and determined that if Scrum did not solve their requirements problems, what else might? …leading to me coming to this client to do a Requirements Process Improvement program.
That does leave the one team that is successful with Scrum; how can a Requirements Process fit into what they do? What their team leader and I have worked out together is that functional requirement statements (e.g. The system must have the ability to create an Order) can be matched to User Stories. A good set of Requirements helps deffne a much better and stable Product Backlog to use going forward with Scrum.
In the end, it is about the process, whatever it is, that works for an organization, delivers successful results, is repeatable, and can be learned by others with the same results; anything else causes more problems than it solves.