How to run an effective pilot and advance on your digitalization journey!

Deploying technology solutions successfully has become the biggest stake of businesses around the world. With over 70% of technology transformation failing, organization really have to reflect on how to launch technology solutions effectively. What drives that high level of failure? In nearly all cases, the reason for the failure isn’t technological (the technology does work as intended) but rather a problem of adoption of the solution.

For me, one key reason for low adoption and failed digitalization is a poor approach to project management and especially in the piloting phase.

Some organizations may skip piloting altogether because they expect people to adopt the new tech immediately and launching globally may seem like a good idea, but it never is. When pilots are run, the vast majority of organizations will underestimate the need for structure in their pilot and it will fail. In most cases, the transformation will fail because of piloting resulting in poor adoption and not the technology! A good pilot will serve the purpose of i) verifying the technology works as intended, ii) making sure that your organization will adopt this solution and iii) verifying that the expected benefits are delivered, with a limited investment in time and money.

So how can we run better pilots and make sure that adoption is high enough to make informed decision?

I. Define the scope:

Defining the perimeter of the project is the first important step in running a successful pilot. It is important to define a scope that is big enough to get representative data but small enough to keep some control over it. To define a scope, you will need to identify “who” and “what”.

Defining the people who are going to be part of the pilot is perhaps the most important consideration. The idea is to identify a pilot population that will be representative of the global population and avoid falling into the trap of piloting your solution with a cohort of your best performers. This will skew the results positively during the pilot and may translate into a failed rollout.

It’s also important to avoid two traps when defining your pilot population:

• Include detractors as well as supporters in the project. As Machiavelli wrote, “keep your friend close and your enemies closer”. Including the detractors in the projects is a great way to sway them over, or at least make sure that their arguments are well-founded rather than opposed in principles.
• Don’t create divisions between pilot users and non-pilot users. Try to keep your scope of users coherent with your business. It can create tensions when you designate pilot users and non-pilot users within a same office or division, especially if the tool does provide great value.

Finally, don’t neglect the other stakeholders, even if not directly using the solution, like the various finance controllers, operation leaders. Keep them at least informed with the periodic reviews (see last point).

The what, in procurement, would be to define clearly the categories of spend, or amounts from which the solution should or should not be used. A common failure at this stage is to launch a solution and expect that the users will figure out when or for what purchases they should use the solution. It should really work the other way around: you need to tell the users when they should be using it, otherwise they will just not use the solution.

II. Define the timeline:

Defining the timelines for the pilot is also an important step for success. That means that all steps, including before and after the pilot must be planned, but also that the pilot itself must have a clear start date and end date. How long is that pilot going to be? There is no clear and universal length to this, as the duration of the pilot should be dictated by how long do we need to get enough data to make an informed decision. This means that duration also depends on how many people you included in the pilot, the more people the shorter and vice versa.

A too-short pilot means you may not have enough data, or will be missing the peak of the adoption curve. All pilots (and technology changes) have a certain inertia, caused by the learning curve and change of habits so if you are cutting the project to early, you may miss that momentum and make wrong decision.

A too-long pilot means that you are either using resources for a project that should have been abandoned as not probing, or that you are delaying the benefits of the roll-out.

Once the ideal duration has been identified, it is important to schedule the Go/No Go call where the decision to roll out or abandon the pilot must be taken (see more on this in the section below).

A basic good practice to remind here is to have a project tracker with all milestones and ETA, and that is updated weekly by the project team and shared in the flash ( See periodic reviews section).

III. Set goals and KPIs:

Setting the goals and KPI of the project is also critical to do from the onset. The idea is to set a number of goals for the solutions: between 3 and 5 and use KPI to validate them. By identifying clearly what success looks like, you are facilitating the follow-up of the project (are we on the right track ?) and perhaps more importantly, the Go/ No Go decision (do we go to roll out?).h

I have seen a lot of organizations forget to set up these goals and KPI and arrive at the end of the pilote only to scramble to find some data supporting what they think is the right decision. This is just poor project management.

Another problem when timelines and goals haven’t be clearly defined is to get stuck into the “never-ending” pilot stage.

That’s why from the onset, you should define a Go/ No Go checklist, (included in you project plan!) so that you summarise all goals, KPIs, who is responsible for getting the underlying data etc.. and can update this list as the pilot goes.

When defining your goals, I recommend defining your methodology for savings calculation – that may save you some painful discussion later on.

Example : The solution must provide an improved experience compared with existing ways of buying.

• Improved time from order to delivery in days
• Time spend per order in minutes
• Reduction in errors and exception handling in %
• Ease of use (As per survey and focus group)

To add some Time-bound element to these SMART goals, you can set these goals to be attained at the end of the pilot, but can also add intermediary steps to make sure you are on track during your periodic reviews.

Also, while you should use quantitive KPI to suport your decision, don’t underestimate qualitative ones: run focus group, survey your users about adoption, satisfactions etc. . As adoption is the key factor, make sure that you are listening to your users in those sessions. A very common pitfall is for procurement to recommend what they think is best, but if it is not what users think is best, your solution won’t be adopted and the roll-out will fail.

Users’ quotes taken from focus groups are a powerful way to support a Go/ No Go decision. Don’t underestimate them, after all, success stories is why you are deploying the solution in the first place.

IV. Define the project team:

The core project team is in my opinion the most important important thing to get right to be successful. A successful pilot needs great alignment between your internal stakeholders, but also with your supplier.

Internal team :

Assigning a project manager is a key step in organizing your pilot, and most organizations are now understanding that project management is a job by itself and should not be someone’s second hat. Your employees aren’t batman and cannot have two identities…

On top of the project manager, who is going to focus on the project deliverables, the most successful organizations I have seen also had a dedicated change manager (or communication manager). Successful deployment have a ‘social’ strategy, where communication will be planned strategically, with a deep understanding of the key influencers, and possible organizational blockers. A mistake most companies make is to neglect this, and suffer the consequences.

Also, if relevant, constitute a network of “power users” with whom you will be communicating more intensely than with other classic users. These people should be a mix of advocates but also detractors so that you collect more objective feedback and get a population of hyper-engaged people to spread your word internally.

Define an executive sponsor:

The decision to start a pilot should be top-down, but the decision to roll out must be bottom-up. Follow this rule and you increase your chances of success significantly.

The common mistake that many organizations make is to use leadership incorrectly, or not at all. it is so prevalent that I am actually using a separate section for this. Any project should have a senior leader sponsoring the project assigned from the very first day. This means that their name will be associated with the project, and its outcome: the Go/ No Go decision for roll-out. Sponsors should have the power to escalate and enforce policies when required, i.e. if resistance to change is too high, processes aren’t followed etc.

Equally important, make sure you have identified (and preferably met with) your account manager’s manager. This person should be the mirror of your executive sponsor and be present in the key milestones of the project like the intermediary and Go/ No Go meeting (see below part 5)

External:

Don’t underestimate or forget to clarify what you expect from the supplier in terms of data, reporting, and other resources. you should also have defined the contractual framework for your collaboration during and post-pilot. It is acceptable to leave certain points open to start a pilot, but the resolution of these points should be tracked and added to the “Go/ No Go checklist”, but ideally, try to align the success (i.e. the adoption) of the pilot with the remuneration.

Also, procurement tends to underestimate the role of the supplier’ account manager: Don’t. Great account managers are invaluable, they will be proactive in solving issues and proposing improvements, data and save a lot of time and issues. If you are worried about the performance of your supplier’ team, make sure to escalate and obtain the right resource.

V. Review periodically:

Set up reviews and communicate updates internally throughout the pilot phase, balancing weekly update, intermediary reviews and the final decision. Most organizations are launching new solutions and waiting until the end of the pilot to see whether it has been successful or not – the best organizations review progress weekly and establish corrective actions to make sure they are on track. Make sure you identified and planned all of these steps from the start of the pilot, this creates a certain accountability toward your stakeholders and establishes a real two-way dialogue.

Weekly:

“Weekly Flash” are a wonderful way to communicate on your project regularly, set them up and see how everything will change. By sending flashes that summarise the vital few of your project and the key updates of your project since the last update, you are creating a channel for engagement that is amazing for both the people in the pilot and the senior stakeholders:

• For the people in the pilot, they will want good news to be shared and are made even more accountable for meeting timelines.
• It is also an easy way for the project to get visibility during the months of the pilot and make sure it remains on the radar. If your leadership hasn’t heard of your project in the last 2 or 3 months, you have made your job of getting the roll out approved much harder.

I also recommend to use these weekly flash to invite people to give feedback by advertising the survey and other feedback mechanisms (like ticketing system or mailbox) available to them.

The intermediary reviews and the final Go/No Go call are key steps to take, and should be planned ahead. These meetings should contain the project team including executive sponsors from both sides so that these meetings really are top to top meetings. These meetings require a lot of preparation as you will need to produce all KPIs and provide a comprehensive view of where we are and agree on next steps.

Don’t underestimate the power of the mid-review too. With mid-reviews, you are giving a chance for the leaders, who ultimately approve the project, to weight in, give their recommandations and be aware of how your pilot is going. Waiting until the final meeting to update your leadership is one surest way to delay your roll out sign off.