QA for your MS Dynamics 365 BC application: why the time for automation is now

July 22, 2019

How to define the right QA strategy for your business and to stay cost-efficient

By now, Microsoft is pushing frequent cumulative updates to their apps in the cloud of increasingly standardized solutions. They have hinted that cloud rules and on-premise rules will gradually converge leaving one set of rules. In short, the good old days of only manually testing your Dynamics NAV/BC solution is coming to an end. If you have not done so already, you likely need to start reconsidering your QA strategy.

Microsoft will push for more use of test code units of AL code for new development – this is clear. However, back tracking to write automated test for 10-year-old object embedded code, this is the time that you should be spending on new development instead. Still, with a faster development cycle, you likely also need to automate your testing of this code.

At the same time, as they push test code units, Microsoft is refactoring the NAV/BC code and exploring tier one and tier two apps anchored on CDS. This means that in the future, a lot of ERP functionality may not even be written in AL. Maybe you need more than one approach – this at least is our conclusion, and this is what this initial blog in a series is about.

The rules have changed – the attitude has not yet caught up

In the past, solutions were individualized and updates where infrequent – you could always roll back a modification. As the Dynamics NAV/BC solution has evolved, it has become more standardized with frequent updates. This means that the cost of catching a bug downstream has risen as you need to fix it for more users and more often. As a result, improving quality by finding the bug upstream is starting to help increase profits.

Many developers have not yet accepted this and even if they had, they do not ‘enjoy writing the same code twice’. Adding in a move from C/side to VS Code with a code review, continued integration and deployment does not make the journey easier. Still, this is the ‘brave new world’ that is now completely normal for our .net core, Angular and Azure developers that are building our UI Builder for Business Central.

New Dynamics NAV/BC customers expect testing to be part of their ‘subscription’. Older (typically larger) non-subscription customers on the other hand, are not too keen on having an even larger up-front investment for their solution. It is only when they get a clear view on the time wasted by their staff on testing that automation starts to make sense.

We are covering three different scenarios

Many of our ISV customers have moved to the new development platform and are pushing test code units to make ‘test automation’ for their standardized solutions. As Luc van Vugt fans, we encourage that, the more coded automation the better.

However, at Global Mediator we build a lot of JavaScript add-ins that are not written in AL and our own UI Builder for Business Central has very few lines of AL code. Surely, we will see more such hybrid development that need to be tested and maybe updated with the same frequency as Dynamics 365 BC. Here we cannot rely on test code units and having to do manual test with every update is not time-efficient.

In addition, we see a fair amount of repeated manual testing for development in larger legacy solutions that are gradually being moved forward to new coding standards. Ongoing development and many smaller modifications suggest some form of automated testing is needed. However, writing test code units for the legacy code can seem a wasted effort and repeated manual testing is costly.

Improving quality by finding the bug upstream is starting to help increase profits

Coded, automated or manual?

Ignoring the established QA terminology and saving it for a later blog, a very high level of our current QA concept at Global Mediator is as follows.

1. We encourage writing ‘test automation’ for all new code and scenarios where it makes sense, but we carefully consider if this is sufficient to validate the user experience for the entire process, or only help to verify the functionality.
2. We push to ‘automate tests’ when we experience a need for repeated manual test, but do not see the benefit in writing it in test code units.
3. We perform manual testing for one-off development tasks where there is a limited economic benefit to automation. For automated tests, we record the process from a user experience for simple testing and then modify this in JavaScript for a more in-depth testing as needed. We have chosen Rapise from our partner Inflectra for this; one of nearly 30 testing tools we evaluated in the process of choosing our preference.

To guide our decision on which of the above methods to use, we use the (still inaccurate) norm that ‘automating the test’ takes about three times what it would take to do it manually and that writing ‘test automation’ for older code takes 5 times longer than ‘automating the test’.

At Global Mediator we are pretty sure that with all the changes going on in Dynamics 365 BC, optimal testing will be an ongoing topic. But, what is your current concept?