Director of DevOps at ADP UK

Keith Watson, Director of DevOps at ADP UK

One of the benefits of software engineering is the ease with which prototypes and proofs of concepts (PoCs) can be created in code. These PoCs help demonstrate the technical and business potential of new ideas and new technologies. This encourages and enables innovation and ideation. They also help to prepare other disciplines in the business, such as marketing and sales, for how these concepts can benefit the product (and hence the customer experience) as well as how they can grow the business in terms of mindshare and market share.

The disadvantage is that these PoCs do not surface the true engineering and business cost of taking these ideas from concept to revenue realization (often called commercialization). In addition, once the executives, sales and marketing are shown these PoCs, the excitement can create a high expectation within the business that these new features can, or must, be delivered quickly to take advantage of first mover advantage or existing competitive conditions. This dissonance in expectations between the business and the software engineering organization is illustrated in the PoC iceberg. This diagram shows just some of the 'below the surface’ engineering work required to deliver a functional, ready, production resilience feature. The costs of achieving this robustness are often far higher than the organization understands.

This expectation can pressure the software engineering teams to deliver components too quickly. This can result in a focus on the ‘golden path’ features demonstrated in the PoC, which are usually the user workflows or features that are most used. This is often at the expense of other less common user journeys (such as error scenarios) and other non-functional requirements necessary for making a feature production-ready and operationally resilient. These are some of the reasons why many software development products and components suffer from budget and schedule overruns or suffer major issues in production after release.

‘Agile development and continuous delivery aim to help improve the organization's ability internally to deliver features quickly to production.’

Typically, the business will already have processes to ensure business readiness requirements of new features from a sales and marketing perspective. The PoC iceberg, however, shows the types of technical requirements that should be costed into functional deliveries to provide a better estimate of true project costs. In an agile environment, these should result in stories in the backlog that should be sized, prioritized, and planned into an appropriate release cycle.

Agile development and continuous delivery aim to help improve the organization’s ability internally to deliver features quickly to production. Another element of innovation often underestimated is the concept of diffusion of features into the marketplace. This diffusion includes the ability of the client and users to adopt and use these new features. This is often described as the innovation adoption cycle where types of adopters are described as innovators, early adopters, the early majority, the late majority, and laggards. If users cannot easily consume new features, then this will delay their adoption by the early majority, which is often the break-even point to recover the engineering investment of new features. In addition, client organizations in the innovation adoption cycle need to be helped to adopt these new features through appropriate client readiness features such as ease of implementation and ease of migration.

In summary, any demonstration of a proof of concept should be handled carefully. The organization should endeavor to fully understand the additional engineering work required to deliver a fully functional, robust, production-ready feature. In addition, it needs to consider what additional work can be done to enable customers to easily consume these features to improve the pace through the innovation adoption cycle.