I Think We Screwed Up Testing

Dear Buried beneath data in Buffalo.

Ok. Deep breaths. Testing is never easy and if one thing usually knocks it off course, it’s data.

It is a lesson learned. Your team is not the first to over-trust dummy data when you are reviewing development and conducting unit testing (testing the system in unit parts not as a whole)

But your team will remember this. That is a positive in the dilemma you currently face.

And don’t beat yourself up too much – you did have the right idea about ensuring your high-volume, financial processing got attention from the outset.

Testing is one of two areas in technology project management where you can find yourself deep underground for a period searching for a way out and through to Go Live.

Data is the other one.

Think about this as process not product for a (short) period. Both components must work well together, especially in your sector when one mistake can cost millions of dollars.

What I mean is: you are going to need to work like the amateur engineer, taking each system process apart with Salesforce to understand the root cause problems – why the engine isn’t going back together and purring into life. And you will then need to rebuild your testing pace again.

Senior management appreciate root cause thinking even if they don’t need the entire life story of what is failing. So, consider surfacing some of what you and Salesforce are discovering. This can reassure them you are the right people for the job, you are just hitting problems at depth.

Don’t refer too much back to your specification at this point, unless it is to pinpoint what you intended in any area of your system functionality. However, do ask more questions on what has been built, on its coding, on what it relies in the system. This is invaluable because with systems and data together, one extra option set value that has not been considered can cause a testing failure.

Accept that in certain areas this means more development to fix data-led problems and thus, more testing time, yet you should monitor each new change and its positive impact on fixing the problem. This is truly like untying a large knot – you find tiny moments where the problem loosens until you are holding the long piece of open rope in your hands.

Focus your team on more nor less discipline, cooler not hotter heads. A good way to is create if you did not create this before, or refine if you did, your acceptance criteria – what must work for the area of development to pass testing. And ask your team to cater for the edge cases in this work – those examples or scenarios you do not get every day which arise and are part of your business.

I say that as a piece of software with common data can often test out well, but it is when it hits the edge cases that it starts to produce failures.

And don’t be too humble to go back to your current system and if another 3^rd party to your current provider. Albeit old or behind your company’s expectations, that system works right now! If they have created system logic descriptions for these problems and this was not shared or was shelved, it may well reveal some answers. Your current provider solved these problems themselves once upon a time, remember or at least some of them.

Finally, and it is an abstract point – the world is telling you to simplify. You may simply be seeking to build in or make work, those data or those scenarios that are not worth the effort, not producing the revenue or impact, versus all those hours tinkering away to get them functioning in the new system.

Make sure you are not simply doing this to make your lives easier. However, sure, removing some of the problem errors, subtracting these data examples from your testing, may well be the best thing to do so you have a smaller list of core scenarios to prove out in your final testing.

Appreciate the high endeavour of what you are doing. You are putting an 1100cc engine back together. You are making a fortress out of little sets of Lego bricks. Sometimes this takes time.

I hope we’ve helped here.