I’m often approached by customers – mainly project managers or process owners – asking me for assistance on assessing the current state of their capture processes (or former projects). Most interestingly, one thing that I hear all to often is them asking for accuracy. How many fields are processed without any user interaction? How many documents can be processed straight-through without any user interaction? How large is the OCR error rate? Several years ago I would have played along, started process benchmarks, and delivered them the figures they were asking for. But things have changed after a) David Wright made me aware of an important fact, b) I managed several projects on my own, and c) I keep asking the customer for their real goals (not only requirements they have and functions they need).

The goal of every capture process is user productivity.

This is what I reply nowadays. This is the sentence I usually start with in pre-sales. I make this statement several times during the project setup phase, and again after everything is running as specified. So – this article is about why accuracy only plays a little role when it comes to assessing a process’ quality.

Imagine the following scenario: you run a company manufacturing garden gnomes. On an average day you can produce 1,000 gnomes, with 10 employees working in your factory. Now one of the most likely KPIs you introduce is the amount of gnomes produced per day and employee – in your case that’s 100 gnomes/worker/day. This is user productivity, and remember that KPIs measure a process’ efficiency, not its effectiveness (doing the things right versus doing the right things). It tells you nothing about the fact that people don’t but gnomes any longer. That’s a different story.

Now, let’s imagine another, lesser abstract scenario: your company is processing invoices, and you’re responsible for running a shared service center. In that center invoices are processed before an accountant sees them in their ERP: capturing most of the relevant information from said documents is your job; with the ultimate goal to make the accountant’s lives easier (and because they pose quite expensive typists). The figures are the same as in our gnome factory: 1,000 invoices per day with 10 employees working in your shared service center, so 100 invoices/worker/day. You’ll call this KPI User Productivity, and it allows both measuring and benchmarking your day-to-day efficiency quite well.

Then, your company acquires additional organizations – and you are chosen to process their invoices as well. But instead of adding more people to your department, you decide to purchase an invoice solution from Kofax. Your ultimate goal is to process a larger amount of invoices per day with the same team. So, while talking to the officials the conversation comes to process efficiency, and that’s where things go south quite fast. All of a sudden every single person keeps mentioning OCR, field and document accuracy – and eventually they’ve got you, before even realize what just happened you think that your new KPI should  all be about accuracy. Before you thought accuracy was your level one goal, you had the perfect KPI already in place: how many documents per hour can an individual process? You knew exactly how efficient (and expensive) your process was. Why is everyone so obsessed with measuring accuracy when it is all but an indirect way to measure efficiency?

Let me give you an abstract example why measuring accuracy isn’t the best idea, we’re going back to or gnomes again. Someone found out that on some days you’re able to produce 20% more gnomes than on average days, and most interestingly that seems to happen almost exclusively when using blue paint for the gnome’s caps. It turns out that blue paint dries much faster than the regular red one, allowing for faster packaging, resulting in an overall higher throughout. So, the proportion of blue capped gnomes could be your new KPI, couldn’t it? The more blue caps there are, there faster you’ll clear your drying tables, resulting in higher overall efficiency. That sounds pretty silly, doesn’t it? All we see here are chains of cause and effect, and we’re looking at a single factor – the blue colour. You would not dream of introducing that KPI – and yet you were doing the same thing with the invoice process?

Think about it that way: accuracy and recognition rates definitely can and often will affect efficiency. But when we focus on them solely we fail to consider many other factors that drive efficiency as well! Here is a non-extensive list of factors that can affect user productivity positively or negatively. Some of them even are interdependent (OCR accuracy affects field accuracy, for example).

So, what usually happens when you measure accuracy alone is that you’ll focus on extraction, i.e. OCR profiles, locators, et cetera. And that’s the problem: there’s a limit to what you’ll be able to achieve. Maybe you’ll find another method of extracting that invoice number, maybe that one will be 20% faster than the old one – but there’s a limit to what you will be able to achieve: that is, close to 100% accuracy (which is, of course, impossible).  Now, let’s have a look at the figure above – is there anything else we can do in case we want to increase user productivity? Here are some examples on what you can do:

• Enable autocomplete (see image below). For example, you can key a n-digit number by just typing 2 characters and hitting enter;
• Keyboard shortcuts: do not only create them, but also make sure they know which commands there are;
• Use script formatters for default values: imagine a date field that should default to today. Let them type “d” and hit enter, and fill the field;
• Custom auto-complete: when your users need to type in a time period that most of the cases is from the 1st of January to the 31st of December of a specific year, then just let them type “15” and translate that to “01/01/2015”;
• Learning by Observation: reserve half a day to work along with them. Observe them, work with their documents and your system. You’ll quickly see where performance bottlenecks are;
• Make sure they aren’t dissatisfied when using your system. Slow performance can drive fast typists mad, and when they are forced to use that system on a daily basis, they easily become frustrated. Analyze what causes the performance issues, and get rid of them;
• And last, but not least: (now I’m talking about accuracy!) perform classification/extraction benchmarks. Focus on false positives first, followed by false negatives, and lastly try reducing true negatives.

Don’t get me wrong here, I’m not saying accuracy isn’t important. It is, and sometimes it can contribute a great share toward user productivity and hence efficiency. All I’m saying is that by focusing on it solely, sometimes one becomes blind for more important factors that quite often achieve greater results in less time.