Sometimes even a “good” result can be disappointing

Learn Data Science
Teradata Employee

Last week, Karthik posted some comments regarding pivoting and generalization, and within it made the point that “business folks want results targeted towards specific goals.” This post looks at a variation of what the business wants.


Of course analytics must solve a business problem to be worthwhile, but there are times when the business not only wants a problem solved, but also expects that the solution will take a particular form having a very specific outcome, and they want that outcome really, really bad – they just aren’t imagining any other possibility.


It’s like the Mirror of Erised from the first Harry Potter book, within which Harry sees himself enjoying the company of his parents, though they had been killed before he had ever known them. Properly decoded, the inscription on the mirror reads, “I show not your face but your heart’s desire.” Harry’s Headmaster warns him that people have wasted their lives away before the mirror, transfixed by images of their wishes attained.


So back in the real world, we had an urgent request from a business group involved with store locations. It had come to pass that a store slot was now available in a very desirable center in the same town with an existing store only a mile and an intervening highway away. The existing center was respectable, but had no major anchors, while the new location had a Macy’s and a Nordstrom and an array of banners that virtually shouted high end.


All the factors influencing a site selection delivered better scores for the new location than the existing one with two exceptions. The rent was much higher for the new location and the potential store traffic for the new location was still an unknown. And a decision needed to be made in less than two weeks.


We were fortunate to have both Aster & Teradata platforms available with a large box of tools. Two weeks gave us a short window, but all the store demographics and traffic history were available. We quickly assembled an analytic data set and ran a series of iterations to build a model that would predict store traffic. The model did predict all store traffic, but it was only going to be used once, and the only outcome that really mattered was the prediction for the new location.


Because the rent was so much higher, it became a requirement that traffic in the new location needed to be at least equal or higher than the highest trafficked stores in the region. But the store group had been gazing into the mirror, mesmerized by visions of exchanging high-fives at their wildly successful grand opening, and thus were very disappointed at our news – we could not guarantee the new location would have enough extra traffic.


Our delivery was good, but the specific outcome was disappointing. Our forecast model wasn’t perfect, none ever are, and might have been better if we’d had a little more time. But the result would have been the same. The model did quite well predicting the volume of existing stores with the same demographics as the new proposed location, but it just couldn’t be forced into producing the specific desired outcome - the highest volume in the region.


Any project manager, with or without a PMI certificate, would emphatically tell us that we hadn’t managed expectations. And that’s true. Even with a short duration, it’s still necessary. You make sure the business problem is well understood by everybody, but then you lead your client away from the mirror so they’re not expecting unreasonable dreams as a certainty.


So remember to manage expectations. We have the platforms and the tools to do great things, but it is still the data that determines the outcome. Even if you have managed expectations well, your client may still be disappointed in a specific outcome – you need to be ready for that. Providing a full explanation of what was done and the accuracy of the overall results should help mitigate that disappointment.


[Short disclaimer – I have not read all the Harry Potter books or seen all the movies. I just like what I remember and like the analogies they contain.]