One thing that I think has been really good recently at the Guardian is the promotion of the idea that what matters is understanding what is critical to our business and valuing quality only in these critical areas. For everything else the thing we should strive to improve is the time to fix. I.e. how long it takes from a problem being reported to it being fixed in production.
If we have a short or tiny time to fix then we can relax a lot of the traditional fixtures of software development like regression testing and metrics like bugs found in production.
What is also interesting is that when you do fix problems you can also look at how long the problem took to be reported and who reported it. If a problem was reported very quickly by a user then you have an indicator that a feature is perhaps more important than you thought.
If on the other hand a problem was reported a week after it occurred and not by the assumed consumers of the feature but by another group or department you have saved yourself a lot of time and effort in having to verify a feature that perhaps does not deserve to exist at all.
Product managers should know where the customer value is in the products they are responsible for. I wonder how many of them pay attention to this kind of feedback?
I do know that some of ours do look at this. It definitely has identified gaps in reporting for example and also gave some insight into what really are significant revenue flows. Mostly however product management is still “vision” rather than evidence-based.