Building a Bridge Across the Research Valley of Death

In the business of research and development, the valley of death is the place where potentially great science and technology goes to die on the way to the marketplace. I’ve been thinking a lot about how one goes about building a bridge across this valley. Here are some readings I found to be of value and some of my thoughts on this topic.

“The Valley of Death” is a metaphor, attributed to Congressman Vern Ehlers that is used to illustrate the gap between research on one hand, and operational use of the technology on the other.

If you are not able to cross this valley, your research and development ends up being nothing more than a science experiment - something that is of value to other scientists and academics but with little to no commercial impact or value to operational entities.

A strategic framework for building the bridge

A good treatment of the challenges and a strategy to build a bridge across the valley can be found in the IEEE Security and Privacy Paper, Crossing the “Valley of Death”: Transitioning Cybersecurity Research into Practice (PDF).

It outlines an approach that starts with active customer engagement to identify challenges and requirements, followed by the building of a pre-R&D research agenda that aligns with overall community priorities. This prioritized research agenda in turn drives the R&D investments.

The paper provides strategies to ensure effective “technology transition” that incorporate specific actions in both the pre-R&D and R&D phases. Technology transition, a.k.a. successful crossing of the valley of death, could be any one or more of the following:

• Deployment of the developed technology within the organization
• Deployment of the developed technology outside the organization
• Adoption of the developed technology into the product roadmap of an existing company
• Incorporation of the developed technology into another project
• Formation of a company that is interested in commercializing the developed technology

The paper, while providing an overall strategic framework, left me wanting more.

Minimum effective dose

As a fan of the concept of the “minimum effective dose” a la Tim Ferriss, which he defines as “the smallest dose that will produce the desired outcome”, I was fascinated to read a recent Government Accountability Office (GAO) study on technology transition at DARPA called Key Factors Drive Transition of Technologies, but Better Training and Data Dissemination Can Increase Success.

GAO’s analysis of DARPA data since 2010 showed that successful technologies that crossed the valley of death had the following common characteristics (I re-wrote the factors to make them more generic and less DOD-centric):

• Organizational or commercial demand for the technology
• Linkage to a research area of sustained interest to the organization
• Active collaboration with potential transition partners
• Achievement of clearly defined technical goals

What gets measured gets done

The above two pieces give a solid foundation to stand on, but the challenge comes in applying the old maxim “What gets measured gets done” to the above four factors.

For that I am turning to the two types of metrics (lag and lead measures) from The 4 Disciplines of Execution: Achieving Your Wildly Important Goals.

Lag measures describe the thing you are trying to improve, so they map well to the above four factors. The problem with lag measures is that they come too late to change your behavior: “When you receive them, the performance that drove them is already in the past.”

Lead measures, on the other hand, “measure the new behaviors that will drive success on the lag measures.” So, proper lead measures focus your attention on improving behaviors under your control so that will then have a positive impact on the lag measures (long term goals).

However, my sense is that there may not be a one-size fits all answer here, and that each project will have to define an appropriate set of lead measures that map to the above success factors.

I would be very interested in discussing and learning more about the metrics that R&D organizations are using to measure their rate of success.

RELATED INFO

• Crossing the “Valley of Death”: Transitioning Cybersecurity Research into Practice (PDF)
• GAO:Key Factors Drive Transition of Technologies, but Better Training and Data Dissemination Can Increase Success
• The 4 Disciplines of Execution: Achieving Your Wildly Important Goals