Blog: Achieving Better Coverage with VHDL – Part 1

When we deal with small designs or small blocks of larger designs, we can create testbenches that will produce correct simulation results and exercise all the behaviour of the design
—

In the case of modern large designs with complex functionality however, even the most knowledgeable designer may miss some design behaviour while creating the testbench. For verification, we are looking for the confirmation that the testbench tested all design behaviour and that the design behaviour is the intended behaviour. But with complex designs, how can the designer or verification engineer ensure that all the design behaviour is covered? Well the clue is in the last word of the question; they can employ the use of ‘Coverage’.

In the first of a three part journey, we will introduce code coverage, code coverage types and the HDL aspects of code coverage. In part two, we will look at property coverage (a close relative of assertions) and introduce the basic terms along with some sequence and property examples. Finally, in part three we will examine smart functional coverage and how this is implemented with OSVVM (Open Source VHDL Verification Methodology).

By the end of this journey we hope to give you enough of an understanding of the types of coverage available for VHDL to improve the integrity of your verification methodology.

What is Coverage?

Coverage is an extremely ambiguous term when used on its own. We can say that with respect to electronic design, coverage is akin to software design metrics that tells us if certain aspects of the design were properly exercised during the testing procedure.

We usually qualify the coverage term by asking what is the aspect that was tested i.e.

• Code quality testing yields code coverage
• Property checking yields property coverage
• Collection of design variable values yields functional coverage

When we simulate the HDL code of our design with code coverage enabled, we can monitor if all elements of the code structure were exercised. If they were, we assume that design quality is high. Fully tested HDL code may still not meet the design specification i.e. a lack of errors means that actual, not intended functionality of the code was tested. Functionality testing (represented by property coverage and functional coverage) gives us better information than structure testing.

When we consider structure and functionality tests, we may ask ourselves if they are mutually exclusive or do they overlap? In order to answer this, we need to understand more about the verification challenge and what each form of coverage represents.

Regarding the design challenge, we agree that we have to have good design code. Bad code can sometimes surprise us with good functionality, but it is not manageable, this is where we need code coverage. We also know that good code can represent bad functionality and we can simulate a lot to verify that it really represents the desired design behaviour. We will see in part two and three that property and functional coverage can carry out this task faster and better. But for now let us see three different types of coverage and their advantages and disadvantages.

Code Coverage Types

Before we look at the different types of code coverage it’s worth just understanding the origin of code coverage. Code Coverage was invented to improve the quality of programs written in “regular” programming languages such as C. By its nature, code coverage analyses the structural correctness of the code, not the functional correctness. Structurally incorrect code usually malfunctions, so correcting bad code usually corrects functional errors.

Code coverage developed into a wide variety of types to address obvious and well-hidden problems that can creep into the code. The most basic types of code coverage are:

• Statement Coverage – checking if all statements in the code were executed at least once
• Branch Coverage – checking if all branches of conditional statements (if, case, etc.) were executed

Statement Coverage is closely related to Line Coverage, results of both differ only if the coder squeezes more than one statement in one line (not a good practice). The results of Branch Coverage could be extracted from the statement coverage results, but in large designs it would be a very tedious task.

Statement and branch coverage can yield false positive results when they deal with non-trivial expressions:

• Trig := SensA or SensB;
• if Load=’1′ and Enable=’1′ then …

There are 4 different combinations of values of SensA and SensB, but the statement coverage will be OK after just one execution of the assignment. Therefore we need Expression Coverage here …

Branch coverage will just tell us that the branch was not executed without specifying if Load, Enable or both are to blame. Therefore we need Condition Coverage here …

Finally, If we run a simulation of the code presented below, then two executions are needed to achieve statement and branch coverage they are marked with ① and ②.

But there are two additional execution paths that were not exercised yet – marked with ③ and ④.

We need two more executions and Path Coverage enabled to fully test this code.

HDL Aspects of Code Coverage

Original code coverage was created for sequential code and ignores the concurrency and modularity of VHDL. Modularity issues can be addressed by per instance data collection in addition to per unit data collection, for example; If you have a D-flipflop component instantiated 4 times and per unit data collection is enabled, execution of one instance signifies complete coverage.

If you enable per instance data collection in the same design, all 4 instances must be executed to signify complete coverage. There isn’t a way of verifying (with code coverage) concurrent execution of two processes residing in one unit.

If we consider code coverage in simulation tools it is important to understand that code coverage is not a language feature – it is a tool feature. As such, it usually requires a more advanced version of the simulator or additional license features. Therefore you should always check which types of code coverage are supported by the tool. All tools should generate textual coverage reports; look for additional coverage viewers that make analysis of code coverage results easier. You should not enable all coverage types at the same time as it will slow down your simulation and will usually not yield significantly better results.

Part 1 Conclusion

We have introduced code coverage and how it checks structural integrity of the code directly and only indirectly suggests functional correctness. We have introduced coverage types and seen that the use of code coverage can improve the quality of your verification procedures. It is important however, to understand that 100% code coverage never gives a guarantee that the design functionality is correct and less than 100% code coverage gives a strong indication that there are also functional problems in the design. We also noted that code coverage will not check concurrency (or timing).

In part two, we will continue the code coverage journey and look at property coverage (a close relative of assertions). We will introduce the basic terms along with some sequence and property examples. In the meantime, if you are looking to improve your verification solution then you can read about our FPGA methodology solutions. Alternatively, you can request a demo with one of our Applications Specialists who will be happy to discuss your interests and answer your questions.