FirstEDA Blog – Raising the Bar on FPGA Verification

David Clift

David Clift

FirstEDA

One of the many things that has changed with FPGA design over the years is the evolving mix of commercial and open-source software options that our customers have at their disposal to realise their designs. We have also seen the rise of what I would call ‘commercialised’ open-source companies, that supply and provide support for open-source software (e.g. RedHat). But in general, when I think of open-source software, I’m thinking of continuously developed applications, being maintained and supported by their user communities. In this bracket I would include: Jenkins, GitLab, OSVVM, VS Code and Eclipse, to name but a few. By the way, yes, Microsoft does do open-source!

 

Customers mix open-source and commercial software to build the tool flows and environments they need to tackle today’s complex FPGA designs. Reproducibility is a crucial issue, and too the associated generation of the necessary documentation on the design and its validation.

 

In this blog I will talk about OSVVM, not so much an application but a library and a methodology that our customers have been using to boost their design verification to the next level. We will look at the recent changes that have occurred and how you could benefit from them.

 

OSVVM Reporting

 

OSVVM has for many years provided an open API, to allow simulation vendors to take the coverage data from OSVVM and merge this with their internal coverage database. This enables you to visualise the functional coverage and statement coverage in a single report. When we build functional coverage models with OSVVM, coverage results can be dumped into the log file; it’s essential but not that pretty. You can view the Aldec webinar on this here: https://www.aldec.com/en/support/resources/multimedia/webinars/1893

 

There are however, some disadvantages to these reporting methods. For example, it’s not always easy to identify which tests fail due to faulty functionality detected in the simulation, or a build failure when compiling the DUT and the test environment. To address these, OSVVM has been beefing up their reporting, including HTML reports telling you exactly what happened when you ran the test suite to Junit reports for your CI system. Using the reports, you can generate documentation on:

 

  • Completion status of the build
  • Completion status for each test suite in the build
  • Completion status for each test case in each test suite
  • Functional coverage, including details of the coverage settings and the Coverage Bins

 

You can read further about these features in the following OSVVM blog posts:

 

In the latest release, OSVVM can also generate HTML reports for:

  • Log (Transcript) files
  • Link to HTML transcript file now included in detailed test reports to quickly find the simulation output of each test
  • Scoreboard reports are now included in the test detailed reports
  • Scripts optionally activate code coverage and simplify code coverage collection, merging and reporting (HTML)
  • Code coverage reports, when generated, are linked into the OSVVM HTML build reports

 

You can read more about this in the following OSVVM Blog post: https://osvvm.org/archives/1936.

 

At FirstEDA, we see many customer projects that are safety-critical in nature and are requirements-driven. By producing this level of documentation on completed tests, the dates and times can form a vital part of the supporting documentation for a project. Regardless, even if this doesn’t apply to you, good documentation is a vital resource for any project.

 

On that topic, did you know that OSVVM does requirements tracking as well? See blog: https://osvvm.org/archives/1700

 

OSVVM Model-Independent Transactions

 

When it comes to developing verification stimulus, one of the biggest efficiencies you can make is to move away from thinking at the pin-level signal behaviour and abstracting it to the transaction level. Now, if you think about the stimulus at a transaction level, you will soon realise that most interfaces have very similar transactions; read, write, send, and get, with the types of transactions depending on the interface type, i.e. address-based interfaces or streaming.

 

So, with the OSVVM model-independent transaction, we have a standard API. This makes it quicker to adopt for the verification engineer, but also, as the API is standard across OSVVM verification components, test sequences can easily be reused and provide another productivity boost.

 

Read more about OSVVM model-independent transactions here: https://osvvm.org/archives/1755 and here: https://osvvm.org/archives/1705

 

OSVVM Verification Components

 

In the previous section, we spoke about OSVVM model-independent transaction, which forms a standard API used by the OSVVM VCs.

 

Read more about the OSVVM Verification Components here: https://osvvm.org/archives/1745 and https://osvvm.org/archives/1688

 

In writing this blog, and reviewing the evolution of just one such community driven open-source initiative, I’m reminded of the significant contributions made by so many fellow engineers for the benefit of our profession. Indeed, together we’re better.