I skimmed a Wired.com article, and then stopped.
I read it more closely. The article was A/B Test: Inside the Technology That’s Changing the Rules of Business.
Mentioned in the article was Google Web Optimizer. I found
The ideas are:
Choose everything.
Don't make a decision – take every idea (or at least two of 'em) and test
and see which creates the effect you are trying to achieve –
in the case of websites this is traffic patterns, clicking, purchases…
Data makes the call.
Not "HiPPO", not ”highest-paid person’s opinion.” Rather, let actual measurable results make the decision.
The risk is making only tiny improvements.
The success of this model can reinforce making only incremental changes –
and failing to make big changes when they are needed.
The model can apply to bigger changes, larger chunks of code, but that tendency to narrow focus has to be
taken into account. It is so easy, and so effective to make those incremental changes,
the bigger changes needed over time can get lost…
Data can make the very idea of lessons obsolete.
You can get the right results without knowing what the lesson there was – the testing and measuring have become real time.
This has huge implications for not just web design, but in where real world concrete physical design goes.
Prototype on your 3D printer, including the electrical internals, and see which product succeeds,
then push marketing and development behind that to move it further toward ideas that do well.
Again, you would have to watch for a narrowing of genius and those products that are
not incremental changes in existing structure would have to have a route beyond the
tight narrow framework that could come about.
This also affects hiring – find and hire people who are creative rather than have a specific expertise,
and that also makes sense looking over my experiences both hiring and being hired.
The jobs that work well are not necessarily the specific job I was hired for –
it is the job the challenges evolved into…
— doug