Art of verification

Accessing class attributes bottom up !!!

2012-02-18T19:53:00.007+05:30

Accessing a property of a class top down is done through hierarchical access through dot operator which is very common in the HVL world. Accessing class properties bottom up through multiple levels is also a requirement we come across in verification which is little bit tricky to implement.

Following is an example on how we could do a bottom up access of a class property.

class top;
int flag;
level_1 obj;

function new();
obj=new(this);
endfunction

endclass

class level_1;
top parent_of_the_class;

function new ( top parent_of_the_class);
this.parent_of_the_class=parent_of_the_class;
endfunction

task run();
if(parent_of_the_class.flag==1)
begin
$display(" Flag set \n");
end
else
begin
$display(" Flag not set \n");
end
endtask

endclass

Constraint solver performance !!!

2012-01-28T09:29:00.003+05:30

Debugging constraint solver performance issues is a real challenge; even a small mistake in a constraint can impact constraint solver performance . Common mistake people do which results in constrain solver performance degradation is to constraint the data payload to a predictable pattern to enable designers to debug RTL faster. When predictable data pattern is constrained for every transaction the number of bidirectional constraints solved increases exponentially resulting in constraint solver performance degradation. The problem would be magnified further when a scenario generator is used to randomize sequence of transactions.

The solution to this is if you would require a predictable data pattern to be used in the data payload override the random value with predictable pattern in post_randomize() method instead of going for a constraint, this would improve the runtime performance of your simulation.

Using vmm_opts to configure the test bench !!!

2011-12-03T14:56:00.002+05:30

Traditionally plusargs switch is used in verification environments to configure attributes of the verification environment from the command line, like configuring timeout value of the test environment, configuring no of packets to be generated by the generator and setting error limits for the test to exit simulation. More compact and robust implementation of configuration mechanism in VMM is through vmm_opts. vmm_opts class has methods like get_object_bit(), get_object_int() to receive the runtime values in the environment. We can set different values to different instances from the command line. Example you can use the same configuration attribute for configuring no of packets for both TX as well as RX transactions and hierarchically set different values for TX and RX from the command line. Configuring global values like the test bench timeout does not require hierarchical access. The values can also be overridden from the test case using set_int () and set_bit() methods in place of a command line override.

Configurability of test bench from command line is a must have feature especially to abstract your test bench complexity from the consumers in this case a RTL designer , test case development team or release management team.

Register Automation using VMM/UVM RAL

2011-11-12T15:12:00.004+05:30

VMM RAL had been around for a long time is a very powerful feature to verify your hardware registers it provides the user with features like name based register access, mirroring registers, back door access, functional coverage and the predefined tests .The same set of VMM RAL features are available in UVM as well. One of the features which i was impressed with was auto mirror update feature which updates the RAL mirror on register changes through backdoor, this feature is handy when you do your register read/write through an embedded processor instead of regular front door access and you want to synchronize your test bench based on the value of register mirrors.

Accellera has come up with standards like IPXACT and system RDL to define your registers, this enables vendor independence to the users and most of generator tools are converging on these standards. The ideal way of automating your register is to define the registers using the standards and use generators from vendors to auto generate the RTL, firmware code, documentation and system Verilog RAL classes.

Quick survey on the tools supporting the standards indicated that there are many players in this space. One aspect which did not sink into my head was why the entire register solution can’t be packaged with the simulator itself, so that the user does not have to make additional investment on another tool. May be it might be on the product road map for the simulators.

With VMM RAL being adopted in UVM, definitely the user base for RAL is going to increase.

Downside of being the “guinea pig” for adopting the latest buzz words in verification !!!

2011-11-05T07:58:00.020+05:30

Adopting the latest’s verification methods and trends is my hobby right from the year 2002 and I have been continuing this till date. Adoption includes adopting new feature in the tool/methodology or construct in a language to adopting a new method of verification which could potentially improve the productivity, results in finding more bugs and improve the maintenance of the test bench. The upside of this hobby is you keep yourself updated with new trends in the market and become the early bird in adopting a trend. Evaluating the down side of this hobby did not come to my mind till one of my friends put an insight into my mind, adopting all the marketing buzz words can become fatal.

His words of wisdom made me rewind my thoughts on all the possible trends I tried to adopt over years. Few common trends i observed in each of the adoption are as follows

1) New code development is only 10% rest of the effort is making enhancements to the current environment. Tool vendors do not take this equation and end up facing gaps in the feature offering.

2) Being the first one to adopt a new verification trend, be prepared to face tool bugs, and sometimes even showstopper bugs.

3) Not all people have the practice of updating themselves to the latest trend and are more than happy to implement things with outdated technology. These people find it difficult to adapt to changes, we have to carry this type of people with us.

There have been feature or tools i have adopted early with became highly popular over years and some features i adopted early don’t even exist now.

Gate level simulation !!!

2011-10-08T08:18:00.008+05:30

One of time-consuming task in functional verification life cycle is gate level simulation. Gate level simulation can be broadly broken down into

1) Gate level simulation after synthesis to check equivalence with the RTL.

2) Gate level simulation with SDF on the post routed netlist.

A common question that is asked by many fokes is why step 2 is required. The netlist before P & R is compared with the netlist after P & R using formal tools for logic equivalence. Timing after P & R is checked with STA for all possible corners and configuration. It looks like an SDF simulation on post routed netlist is redundant.

Assume now a designer by mistake has place timing exception like false path and multi cycle path, the above condition goes undetected in the STA. Dynamic functional gate level simulation with SDF on post routed netlist is counter check for STA and catches if the designer has made mistakes on placing an timing exception.

Code coverage !!!

2011-10-05T12:54:00.002+05:30

Coverage closure be it functional coverage or code coverage is one of the time consuming task in functional verification life cycle. With respect to functional coverage automatic coverage closure tools are available in the market. When it comes to code coverage it is very tedious and manual process to measure and close coverage, Code coverage number (Line, condition, fsm , toggle) of 95% is unacceptable in many places. The remaining 5 % coverage holes need to be completely analyzed and closed. More perfection requires more time and resource . Generally code coverage analysis is done by verification engineers who have relatively less knowledge in design this impacts the overall efficiency of the task execution. Why is code coverage important especially when we have functional coverage ? missing test cases are captured easily by code coverage which seems to have slipped through the functional coverage plan.

It is good idea to not include the register package test for code coverage as this test would give a false pass on the register coverage. We generally expect the register to be covered using functional tests, by including register package test we will end up covering registers by just doing a read and write to all the bits which is not our intent.

What make a verification team great !!!

2011-09-14T15:28:00.003+05:30

Move with the industry trend

Verification keeps moving at a very fast pace, great verification teams move with the advancements very fast and implement those in the projects to reap the benefits. Ordinary teams stay with the old ways of doing things.

Knowledge sharing

The most important aspect for a verification team to become a great team is to share the knowledge they gained with the entire team improving the overall efficiency of the team.

Take the entire team with you

Most important aspect in managing and staging advance verification techniques in the project is the ability of the members to take the entire team with them. Ordinary teams follow the decision of the most powerful person in the team; great teams take a collaborative decision.

Risk taking

To implement or adopt new verification trends the team should be able to take calculated risk. If the risk taking ability is not there then it points to an ordinary team.

Expect the unexpected

One of the attributes of the great verification team is to expect the unexpected result during an adoption to a new trend and successfully overcome it. Ordinary verification teams back out when the unexpected behavior happens.

VMM Channel methods grab, ungrab, lock, unlock !!!

2011-08-20T21:38:00.005+05:30

Assume two different threads feeding a single channel with a sequence of transactions. If the transactions are just put in to the channel without using channel grab() the result would be that the sequence between two threads will be mixed producing unexpected results. Grab() method is used to request for exclusive access to the channel once the grab is activated no other thread can put an object in to the channel. Once the channel is loaded with the sequence of transaction object ungrab() should be used to release the channel for other threads. Is_grabbed () function can be used to know if the channel is grabbed.

Lock () method can be used to lock the channel producer (put ) or channel consumer (get ). Unlock () can be initiated to remove the lock(). Status of the lock can be got using is_locked() method. These methods are useful to control channel from a different location say different block.

Using a verification methodology to solve a verification problem !!!

2011-07-11T03:02:00.001+05:30

Few problems that i always come across with verification methodology users is limited knowledge on what the methodology can offer and lack of effort on their side to update them self on the new features of the methodology. Once they are presented a verification problem to be solved using a verification methodology they come up with a solution based on their limited knowledge of the methodology. The solution they come up is exactly not the best solution that the methodology can offer. User manual / reference guide of a methodology can only educate you on the features the methodology can offer. Ultimately taking a decision on what features to use under what circumstance is totally left to the users of the methodology. This is where I find many people making mistakes starting from a simple noncompliance to a major architectural mistake. Once such types of mistakes are done the code eventually becomes an excess baggage that needs to be carried and maintained for the rest of the project life span.

vmm_broadcast and vmm_scheduler !!!

2011-06-18T23:13:00.010+05:30

Channels are point-to-point data transfer mechanisms. If multiple consumers had to extract same transaction descriptors from a channel then vmm_broadcast should be used. vmm_broadcast broadcasts transaction from one source channel to multiple output channels. Copy of the transaction from the source channel is forwarded to the output channels. Assume if you have multiple interfaces having different signal level protocol transmitting the same transaction at the same time a VMM broadcast can be used in this scenario. Unified generator is connected to the source channel of the vmm_broadcast and output channels of the broadcast are connected to the drivers of the interfaces.

Vmm_scheduler directs the transaction from different input channels to single output channel based on a scheduling algorithm. The default scheduling algorithm is round robin mode, by adjusting the constraint you can also have a random scheduling implemented. If you need custom scheduling you can implement the same in the vmm_scheduler_election class. Scheduler can be used if you need to schedule transactions based on some timing information.

Verification environment architecture !!!

2011-05-08T03:56:00.010+05:30

Architecting a verification environment with a dynamically changing requirement is a real challenge, when the entire requirement is known upfront you can fit your problem statement perfectly in to methodology. But if you get requirements in bit and pieces then you make an architectural decision based on the know requirements, when more requirements trickle in you find that decision you did earlier was not right. Option you have on hand is to re-write the code and correct your mistake or patch up the code and deviate it from the methodology recommendations resulting in less reusability. This is a typical problem that happens when overall picture of the problem statement is not understood by the architect. Another problem is due to the attitude that let us get the basic stuff working first, then incrementally fit the requirements in to the basic architecture.Typical project flow when requirements are not know upfront will be like this ( Just for humor !!! ).

Channel record & playback in VMM !!!

2011-04-02T18:06:00.003+05:30

Channel record and playback is useful feature which can be used to reproduce an issue hit at top level environment in a different block level environment. As you know the randomization changes with the change in system Verilog files for the same seed, the same scenario cannot be reproduced with the same seed in a different environment due to the change in the system Verilog files. We need to spend time running random regression with different seeds at block level to reproduce the issue that happened in the top level with a particular sequence. The alternative way to reproduce the issue at block level is to record the transaction at top level which is done using channel record. Then playback the transaction through channel playback from the block level to reproduce the issue.

// Record transaction in top level

gen.out_chan.record("Record_transaction");

// Play back at Block level

gen.out_chan.playback(status,"Record_transaction",tr);

If (!status)

`vmm_error(log,”play back failed \n”);

First look at UVM Methodology !!!

2011-03-20T07:23:00.002+05:30

I have been a RVM/VMM user for many years following advancements that come in VMM in every release; I try to use the new features whenever i get a chance. Recently i had a chance to take a first look at the UVM methodology. I was fully aware that UVM is based on the OVM methodology, with register package derived from VMM (RAL). Now how easy or difficult is it for a person with VMM background to pick up UVM with the fact that he does not have an OVM knowledge. From my experience I felt that this can be done very quickly in a matter of few days. As i started reading the UVM methodology and exploring the features of the methodology, I did see a lot of similarities between UVM and VMM. People who had a chance to use VMM 1.2 can make this switch even faster. I went through all the basic features UVM had to offer, looks very interesting. Then i decided i should explore UVM features in detail.

Error injection in VMM environments !!!

2011-02-20T05:30:00.007+05:30

There are different approaches in VMM to inject error , each one selects a way which is comfortable for them. But when finalizing on an approach it is good to know the advantages and disadvantages in terms of test bench reuse and code organization.

Error injection in transaction class attributes

Design a transaction class with virtual fields specifying the error types and control the physical fields based on the virtual error types.

This error injection code can be placed with basic transaction class or the error injection code can be a separate class extending the basic transaction class. I would prefer the error injection class to be a separate class extending the basic transaction class as the code become better organized and you don’t end up having one big monolithic transaction class.

Most of the transaction based error injection should be done using this approach. Sequence involving error injection can be easily generated using this approach.

Error injection through call back registered through driver.

Error injection on the signal level protocol controlling the driver attributes should be done through call back registers to the driver. Transaction class error injection should not be handled through driver call backs.

Formal verification !!!

2011-01-03T17:20:00.008+05:30

Recently i had a chance to know what formal verification is all about. My first impression on formal verification, it uses white box techniques to verify design against the black box approach used in constraint random verification. If your assertions or property definitions is accurate, formal tool can hit bugs faster than regular simulation effort. Ramp up on the formal tool takes little bit of time for people who are new to the formal verification world, debugging failures requires little bit of ramp up time as well. we need to debug failures without timing information by tracing schematics. When you get through the initial hiccups, you will definitely enjoy doing formal verification. Formal tool takes system Verilog or PSL assertions and tries to prove that your assertion can be violated by generating all possible stimuli.

uni-directional, bi-directional constraint & functions in constraints !!!

2010-12-04T21:25:00.005+05:30

Consider a simple constraint example a == b + c;

Now if you constraint any of the 2 values of a,b,c the third value gets generated automatically due to the bi-directional nature of the constraint.

Now modify the constraint void (a) == b+ C;

Value of a gets generated first based on which b and c are generated. Generating a based on b and c is not possible. The same functionality can be achieved by using a solve before constraint as follows.

a== b+c;
solve a before b; solve a before c;

Functions can be used in constraints for reusability between constraints, but important point to note is the constraints will become uni-directional when used in a function.

function integer sum (integer b, integer c);
sum=b+c;
end function

Constraint sum_constraint { a==sum(b,c); }

Randomization of floating point or Real variable !!!

2010-11-26T01:57:00.002+05:30

What are the application areas of floating point numbers ?

1) Floating point numbers are used in PLL configuration were fractional values are required.
2) Processor,image processing & graphics applications mostly work on floating point numbers.

Vera/ NTB does not even have a real or a floating point type; forget about randomizing a floating point number. The workaround for this is to write your own floating point class, randomize the class and use it. System verilog has a real type which is used to represents a floating point number, but a real type cannot be randomized. The LRM does not support the randomization of real data types. Some time back when i was discussing about this with one of my friend he was telling me that system verilog committee was working on this, not sure how true this information is.

Test end condition using vmm_consensus !!!

2010-11-07T23:21:00.005+05:30

VMM provides the vmm_consensus class as a voting mechanism, which is used to determine when the test could be terminated. Earlier before the introduction of vmm_consensus class end of test condition was determined by some condition like score board empty condition or a timeout. The test end condition was tied to a specific environment and was not reusable across environments.

With the introduction of vmm_consensus class various elements such as channels, notification, and transactors play the role of voter and all voters have to agree for a consensus. Even if one voter opposes there will be no consensus. The vmm_consensus has a wait_for_consensus ( ) method which will be called in vmm_env’s wait_for_end ( ) method. The wait_for_consensus ( ) method will block till all voters consent.

How to use vmm_consensus to determine when the test should end ?

1) Add vmm_consensus::wait_for_consensus ( ) method to vmm_env::wait_for_end () method

2) Register voters in the vmm_env::build ( ) using end_vote.register_* () method, where end_vote is the instance of vmm_consensus defined in vmm.sv

3) Each of the sub environment instance has a single vote, it can consent or oppose.

4) We have options such as consensus_force_thru() which can be used by a particular subenv or a VMM components to force consensus through even though other components oppose the decision.

5) As usual we have methods that can be used to monitor the status of consensus, which components oppose and which components consent.

Scoreboard for checking interrupts !!!

2010-10-09T18:48:00.004+05:30

Architecting a good interrupt monitor and an interrupt scoreboard is very essential to hit bugs and close coverage on the interrupt logic of the design.The architecture is very simple, you need to have a shadow register for your interrupt status register and predict the interrupt by writing in to this register, the predication should be done in the transactor based on the transaction class attributes for a regular transaction, error condition ect .. The interrupt scoreboard should reside on the passive interrupt monitor which reads the shadow register as well as the original hardware register and the mask and makes a comparison on the expected interrupts and flags an error if the interrupt is mismatching. The monitor should have options to clear the interrupt when it is available or accumulate interrupt and clear it when it is required. At end of the simulation a comparison of shadow register and real interrupt status register needs to be done to check if all the expected interrupts have arrived and flag appropriate error messages.

Interfacing CRV environment with procedural environments !!!

2010-09-11T23:49:00.005+05:30

One of interesting challenges in verification is building constraint random verification environment on top of an existing code which is a procedural code.The challenges are unique to each of the environments; simplest solution is wrapping the procedural code in a transactor which is connected to a channel which interfaces with the constraint random generators. Transaction object attributes are mapped to different functionality in the procedural code.

Tips to improve constraint solver performance !!!

2010-08-22T20:32:00.006+05:30

Performance issues are very hard to debug, especially to find the root cause of the issue. If the constraint solving time is at unacceptable limits, the first area to debug is the bidirectional nature of the constraint implemented. quickly review the constrains that does not require bi-directional functionality and convert constraints from bidirectional to unidirectional using void termination or solve before, also reduce the number of constraints solved at a single point of time to improve performance.

Structures and union in system verilog !!!

2010-07-03T23:38:00.002+05:30

VERA/NTB users migrating to system verilog have a tendency of not using structures and union construct in system verilog when architecting or implementing a system verilog verification environment. The reason being, this constructs are not available in VERA/NTB. Struct is pretty useful construct to group fields which are logically related to each other. It helps to organize you code. The union construct is very similar to a structure, but only one of the fields will be valid at a given point of time. Using packed struct and packed unions helps to organize fields in the memory without gaps, which in turn results in faster access to memory and results in faster simulation time.

Factory replacement in the scenario generator scenario using ‘.using’ gotcha !!!

2010-05-20T01:50:00.004+05:30

When you want to do a factory replacement of a transaction class in your scenario,we assign the factory to .using.

Scenario.using=transaction_factory;

The gotcha in this factory replacement is we need to implement the allocate() and copy() methods in the extended class for the factory replacement to work.

Many times i have seen people including myself spending time debugging their code when using “.using” for factory replacement as they are not aware of the gotcha.Maybe the RVM/VMM documentation should have this requirement highlighted so that the user can easily understand this requirement.

Atomic generator using allocate() !!!

2010-05-08T22:45:00.006+05:30

RVM/VMM atomic and scenario generator randomizes a blueprint pattern which is assigned with the extended class (factory) , the copy of the randomized blueprint is pushed into the channel. Most of the RVM/VMM users use this approch in there custom generators. Is there a different way of implementing your atomic generator without using a copy() method ?. The answer is yes, following method can be used to generate atomic transaction without using copy().

For the above code to work you need to implement the allocate method in your extended class. The extended class is assigned to the factory before the start_xactor() method is called.

ASIC verification tasks verification engineers dislike the most !!!

2010-05-02T01:41:00.004+05:30

I was having a friendly discussion with few of my fellow verification engineers, we were discussing about the verification tasks they dislike the most. The conclusion from the discussion was they disliked verification tasks which were laborious, which requires manual effort and had very little learning opportunity. The verification task they disliked are as follows.

1)Language migration without the help of a commercial language conversion tool
2)Code coverage analysis and identification of code coverage exclusions.
3) “X” tracing while debugging gate level simulation.
4)Release management ( Tagging the release code ,Triggering regression and debug of regression results ,becomes messy when more that 25 people are involved and release management is handled by a single person)
5)Working on outdated technology which is away from the verification market movement.

Top 6 verification misconceptions!!!

2010-04-25T00:53:00.005+05:30

1)This is a silicon proven IP no need to do rigorous testing and coverage – Hold on are you sure if all the feature crosses are validated by the IP vendor in the silicon.

2)RTL is the one that is taped out and converted to a product no need to review verification environment, test plan review is sufficient --- 70 % of time is spend on verification out of which considerable amount of time is spend on developing test bench, re-usable and well architected test bench helps you to achieve your verification objectives in less time and in maintenance of your code in longer run.

3)This is a working legacy code keep it --- Legacy code with defect in the architecture is difficult to maintain and doing an enhancement on the code is a nightmare. Consider re-coding sections of the legacy code when you have to do an enhancement in the code to support a new feature.

4)Start the implementation and study the protocol as you go. We want you to be productive from day1 --- A big no!! You cannot architect an environment without understanding the protocol completely and you will always end up with a situation were you were not informed about a requirement.

5)Good verification engineer is the one who hit lot of bugs – Hold on you can have a bad designer who can make many mistakes, making ordinary verification engineer look exceptional.

6)No need to have an architecture document for a verification environment --- Reverse engineering a code to find out the verification environment architecture is not an easy job. It is a pain of understanding the architecture of an environment after reverse engineering a code. Always document the architecture of your verification environment if possible document a class diagram.

Checker for complex constraints spanning multiple transaction !!!

2010-04-17T22:12:00.003+05:30

It is hard to debug the constraint failures spanning multiple transactions say sequence of ten to fifteen transaction were the constraints on each atomic transaction is dependent on the other. Manual debug on such failures are time consuming.We can use a procedural checker to check the randomization of the scenario which is an array of transaction objects. In RVM/VMM the procedural checker can be placed in the post_scenario_gen callback of the RVM/VMM scenario generator. The post_scenario_gen has drop bit which can be set from the callback to stop the transactions from being pushed in to the channel.The checker is active at all times and flags an error if there is an error in the randomization across transaction.

System verilog 2009 new features !!!

2010-04-14T02:55:00.016+05:30

Following are some of the new features of system verilog 2009 standard that caught my attention.

1)`begin_keyword `end_keyword

The above defines can be used between any section of code to maintain backward compatibility with verilog or system verilog 2005, very useful if you are migrating your testbench or design to system verilog, you might come across system verilog keywords used in your verilog design which can cause compile to fail. With system verilog 2009 just wrap the code with `begin_keyword and `end_keyword to get past the error without modifying your code.

2) Let construct substitute to macros ?

Package example_package;
Let expand_operation (a,b) = assert ( !a & b)
end package;

module test ( …);
import example_package::*;

always @ ( …) begin
expand_operation (read,write); // expands to assert(!read & write)
end

end module

3) Pure constraints

virtual class example;
pure constraint valid;
endclass

Allows you to declare pure constraint in the abstract class,just the declaration alone without implementation. The implementation of this constraint is provided in the extended class with the same constraint name.

Closing “quality gap” in functional verification !!!

2010-03-25T20:46:00.003+05:30

Recently i came across a white paper on a new EDA tool certitude which is used to close the ”quality gap” in functional verification using mutation techniques. The tool introduces mutations in the RTL code and then subjects the mutated RTL to the verification team's test bench and checks if the verification environment has been able to detect, activate and propagate the mutations. If the verification environment is able to catch the mutations, then the verification is probably complete and there is a reasonable certainty that no bugs are left. If not, there could be serious weaknesses in the test bench and it needs to be reworked. Put simply, if the verification testbench cannot detect the mutations (or bugs) introduced, chances are there that other bugs are also left out.

Another way to see the situation is that after introducing mutations, we have two versions of the RTL code, one original and one mutated (say, bugs) and the verification environment is passing both. Clearly then there is a weakness in the verification environment which needs to be fixed.

The tool works with all the major simulators available in the market and works with all languages C, System C, System Verilog, Specman e, Vera.

VMM 1.2 tutorial !!!

2010-03-21T04:11:00.001+05:30

I was looking out for a nice tutorial on the new features of VMM 1.2. I came across this 2 hr video tutorial on VMM 1.2 features on the VMM central web site (http://www.vmmcentral.org ). Nicely composed video covers all aspects of VMM 1.2 release from implicit phasing, analysis ports, transport & factory. Definitely useful for VMM 1.1 users migrating to VMM 1.2.

Another verification methodology UVM !!!

2010-03-17T03:16:00.008+05:30

Recently through one of my friends i came to know about the development of a new verification methodology UVM.The UVM ( Universal verification methodology ) is being standardized by Accellera Technical Subcommittee (TSC) and claims to solve the system verilog cross methodology interoperability problem. This methodology is being suppored by all the 3 major EDA vendors synopsys, cadence and mentor. The advantage of switching to this methodology (when it is available) is portability of the methodology across different vendors. The base code for this methodology will be from OVM version 2.0.3.The base classes for UVM methodology is expected by Q1 2010.

It is still not clear about the backward compatability of this methodology with OVM. To me backward compatibility with VMM is out of question as the base code for this methodology is from OVM.

Functional coverage in RVM /VMM !!!

2010-02-20T23:54:00.004+05:30

The conventional way of collecting functional coverage for a transaction class in RVM/VMM is through RVM/VMM callback and it generally reports the functional coverage on the random transaction generated by the generator, The functional coverage point hit or miss is purely dependent on the generated random values there is no real check if that scenario was hit in the RTL or not, meaning if you disconnect the RTL completely from the verification environment and run the test the functional converge point will still be hit.

Ideal way is to have a passive monitor hooked up to the RTL which will collect functional coverage, the idea is like sampling the values from the RTL through the passive monitor and unpacking the bytes to a transaction class object and from the transaction class object we can easily collect functional coverage. By having such an arrangement we make sure that the functional coverage point is only hit when the scenario is hit in the RTL. I know this approach requires more effort but we can be absolutely sure that the functional coverage hit will happen only if the scenario happens in the RTL. More over your functional coverage code will be isolated from rest of the verification environment code. In case of VIP development, this is the right approach to group you functional coverage model with a passive monitor.

Some of the drawbacks to this approach are the latest coverage convergence technology might not work well with this approach. You might not be able to tune or bias your constraint solver to achieve your functional coverage convergence automatically. Especially when you collect the functional coverage from RTL nodes you need to update your test bench whenever there is a change in the RTL code which affects the way functional coverage is collected.

one more reason for choosing this approach is if you have more than one way of verifying the design and we want the functional coverage monitor to collect functional coverage irrespective of the way of generating the stimulus this approach will work perfectly well.

RVM/VMM Scenario generator !!!

2010-01-01T12:17:00.012+05:30

RVM/VMM ships with some pretty useful built-in components and applications. RVM/VMM's Atomic Generator is probably one of the most powerful ones, yet it's pretty basic. It can definitely help you generate a flow of random items but it was not intended for generation of sequences. A sequence (scenario) is a set of items that have some sort of correlation between them. For example - consider a set of 6 transactions where the transaction 4 depends on the previous transactions say transaction 2. Atomic generator can not generate this kind of sequence, RVM/VMM addresses the need for smart scenarios with the "RVM/VMM Scenario Generator"

Since we can not anticipate the future enhancement in the verification environment it is better that we provide the flexibility in the verification environment for generating scenarios. Selecting scenario generator will be the right step.

The deployment of scenario generator in the verification environment is slightly complex that deploying an atomic generator, but we can have the complete controllability on the micro transactions in a sequence using scenario generators. Deploying a scenario generator in the verification environment will help you in the longer run, also you can randomize between scenarios.

Infact coverage convergence tool !!!

2009-12-16T17:57:00.005+05:30

To start with infact tools features; infact is a graph based tool which generates the random values and traverse through the various predefined paths through graph algorithm. It uses a complimentary technology to a constraint solver (i.e) a graph algorithm. The tool has to be inputted with graph rules which will be processed by the tool to generate all possible combination of the rules. There are no redundant values or combinations generated. The unique value is generated across multiple parallel runs on server farm. We need to provide the tool with all possible combinations. The tool can work with any language say verilog, vhdl, vera, ntb, specmen, system verilog & C++ . Infact tool communicates with simulator through a PLI call. Existing constraint random code needs to be represented as a graph rule and the infact tool will generate the random values.

Joy of doing ASIC verification!!!

2009-10-07T18:47:00.000+05:30

Verification is often treated as the step-child of design. Decade back verification was considered less critical task than design by some companies and fresher’s where often pushed in to verification It’s not surprising, then, that most of the verification engineers want to be designers. But now verification is more lucrative career option than design and many experience people now hold on to verification without moving to design. It is generally estimated that 70% of ASIC design cycle is spend on functional verification. The ratio of verification engineers to design engineers is approximately 3:1. Job switching for verification engineers is easy when compared to the designers provided they have the right skill set. The advancements happen in verification at a very fast rate that design.

Earlier verification job was looked down, as the design is what is getting taped out and moves in to mass production stage not the test bench. But verification requires lot more effort and skills, example, to test a 100 line state machine we need to develop testbench which will have atleast 500 lines of code and draft a test plan which covers all the possible scenarios. VIP development companies get their revenues from their testbench which is licensed and shipped as a product.

Reasonably experienced person will know that building a re-useable system level verification environment and verifying the design without any post silicon bugs is more difficult than adding a glue logic in the design.

Do you still believe verification is a less critical task and requires lesser expertise that design?

Score board architecture !!!

2009-09-27T12:59:00.005+05:30

How would you implement the following requirement of designing a scoreboard ?

1) Scoreboard should predict a DUT transformation.
2) Scoreboard should be able to handle packet drops.

Just extend the VMM data stream scoreboard and implementing few virtual methods like transform, quick_compare & compare. use expect_with_losses() method for requirement (2). The requirement (1) can be implemented easily with transform method ().

VMM has much more robust features to offer, it is a good idea to check out the features available in VMM score board before coding your own scoreboard, i believe it would save lot of time.

How do you identify a good functional verification engineer?

2009-09-20T15:48:00.001+05:30

Evaluation based on product success:

The answer looks straight forward, at the end of an emulation effort, chip tape out & chip production. If there are no functional bugs and design works as expected then obviously the person who has verified the design is a good verification engineer.

The above statement has a rider; the above result can be produced in three circumstances

1) Good designer, bad verification engineer & very low bug rate.
2) Bad designer, good verification engineer & very high bug rate.
3) Re-used design which is silicon proven & no bugs.

If your chip taped out successfully without any functional issues with scenario 2 , then you have identified a good verification engineer.

Evaluation based on process success:

You wrote a verification environment, found lot of design bugs, now you need to verify a design enhancement which requires changes in your earlier verification environment. Now effort required to do the changes in your verification environment depends on reusability of code you have written earlier. If you are able add the enhancement within a short span of time with little code changes you are on track to be identified as good verification engineer.

Evaluation when the design success is not immediately visible:

This type of scenario is seen in VIP development where the development is verified internally and product release is done for the customers use.

The only way to identify a good verification engineer under this scenario is “customer bug rate over a fixed time say 12 months” Vs “internal bug rate during development”. If there are no customer bugs for a long period of time on the feature, you have identified a good verification engineer.

Evaluation based on knowledge of language /methodology /protocol:

This is the method most of the people use for identifying a verification engineer during hiring in a new organization. If the person has good coding experience in projects, he will be having a very good command on verification languages and will be well versed with the intricacies of the language. Asking a person to write a code for a given scenario will test his knowledge of the language and test his problem solving skills. Testing a persons knowledge of protocol will help us to know how well he has understood the protocol and used the knowledge in verifying the design.

Evaluation based on moving with technology:

This is very important aspect in today’s industry. The verification methodology, tools improvements happens at a very fast rate from the EDA vendors helping the verification engineers to reduce the time on verification. A good verification engineer will definitely keep himself updated on the new aspects of functional verification which is good sign in identifying a good verification engineer.

I have also in my past come across some sense less interviewers evaluating verification hires on his knowledge of digital electronics and CMOS. Does a verification engineer use digital design or CMOS for architecting or writing his test bench?

Verification effectiveness!!!

2009-09-12T14:33:00.001+05:30

“Functional verification takes 70% of the chip design cycle”. Writing test plans, Writing reusable verification environment, writing assertions for the design, debugging RTL failures, attaining code coverage and functional coverage goals & gate level simulation and debug are some of the common activities a functional verification engineer goes through in project life cycle before tape out. The work of verification engineer exponentially increases if the design under test has more number of bugs, which involves lot of RTL debug effort. Metrics on which a verification engineer is evaluated is on “How many bugs where hit during functional verification” Vs “bugs hit during emulation/post silicon validation” Even a single post silicon functional bug indicates the ineffectiveness in the functional verification.

If you are verification engineer and you feel that you have hard time in meeting your schedules and you work for more hours in office (say more than 8 hours) to meet the dead lines. Following are some effective ways i used to meet the dead lines without compromising on work quality or the compromise on working hours.

1) Micro schedule your tasks with effort estimates and get an approval on the time line from your manager.

2) Whenever scope of the work is increased / decreased re-schedule the effort estimates and keep your manager updated about this.

3) Prioritize your tasks and complete one after the other.

4) Whenever you are writing a test bench make sure your test bench is reusable. This can help you to minimize your work at a later point of time.

5) Always try to use module level verification environment at the system level. Maximum effort you need to put is the integration effort from module level to system level.

6) Always write random verification environment to test your design, most of the bugs are easily captured by random verification environment. Write a directed test case only if it is absolutely required to hit a functional coverage / code coverage hole.

7) Always move with the market, try learning and using new technology which will overall reduce the verification effort. Example adoption to tested methodology like VMM or OVM might be initially tough but on a longer run it will reduce your time to verification.

8) Always file a bug when you hit design issue, this is very important because this is the only metrics on which a verification engineer gets evaluated. Also top level management will know the effectiveness of the verification and schedule slips due to design issues.

9) Always keep your manager updated with status of your task so that he will be in position to evaluate your bandwidth for future tasks.

10) Never compromise on testing a DUT feature due to lack of test bench support, this may lead to emulation/post-silicon bugs.

11) Always keep the code coverage analysis towards the end of the project after your function coverage goals are met.

12) When you are stuck with a problem, do not work continuously to fix the issue this will increase the stress level and you will end up spending more time hitting on the areas around the issue. Take a break from the issue and come back with a fresh mind.

13) Try to understand the code base relevant for the enhancement or the modification, spending time on understanding overall code base has diminishing returns.

Does the functional verification engineer get rewarded for his verification efforts is really a question mark and largely dependent on the company you work for.

six reasons why you should use system verilog for verification !!!

2009-08-09T08:35:00.001+05:30

1) System verilog is an IEEE standard supported by multiple vendors, your code is portable across simulators.You are not tied to a single vendor, which is the case if you are using HVL like VERA/NTB/SPECMAN.

2) Free open source standard verification methodologys like VMM & OVM are available which can be used with system verilog.

3) Simulation speed will improve,if you are an HVL user using VERA/SPECMAN for verification.

4) Most of the VIP vendors support system verilog, building verification environment for SOC will not be an issue.

5) System verilog interoperability layers are available, you can re-use you VERA/NTB code from system verilog,similar arrangement is available for specman user too.

6) System verilog supports most of the constructs supported by HVL ( VERA/NTB/SPECMAN) , migration to sytem verilog for HVL users will not be an issue.

Learning curve for a verification engineer !!!

2009-07-19T15:24:00.001+05:30

For a verification engineer, which of following work environment gives him maximum learning opportunity ?

1) IP verification
2) SOC verification
3) Verification IP developement
4) Verification consultancy

I will try to evaluate each of these work environment.

According to me,the skill you accuire on doing your day to day activities at your work place should match the requirements of the industry and should be portable across companies. (i.e) Assume you are doing an assembly level verification for a processor design using internal tools of that company, the methodology and tool knowledge is limited to that particular company and skill is not portable between companies, then the skill you accuired is not marketable, hence the learing curve is minimum in this case.

We can find our self in above scenario in verification consultancy work environment where we have very little control of job nature , implementation flexibily is also minimal in this case. One good thing about this work enviroment is you will get your hands dirty on different type of projects and you will rarely find your self struck with the same project. In comparitive scale verification consultancy work enviroment gives verification engineer moderate learning curve.

Verification IP developement requirement and process are different from RTL developement and verification. In this kind of work environment we have very good learning curve on the new verification methodology , we can improve our knowledge on different languages as VIP are developed in single language but controlled through different languages like VERA/NTB/VERILOG/SYSTEM VERILOG /C. you can gain good protocol knowledge by developing the VIP and stay updated on the developements in the protocol. One of the draw backs in VIP developement work environment is your initial learning curve will be steep, after few years most of you work will be just bug fixes and some times VIP enhancements. In comparitive scale VIP developement work enviroment gives verification engineer moderate learning curve.

SOC verification work environment is different, as the verification is done on proven design IP,finer points in protocol are generally ignored. One good thing in SOC verification is we end up working on different type of protocol and interfaces. In comparitive scale SOC verification work enviroment gives verification engineer good learning curve provided he works on different interfaces every project.

In case of IP developement work environment , your learning curve on the protocol will be good.The test plan and implementation will touch the finer points in protocol verification. In comparitive scale IP verification work enviroment gives verification engineer good learning curve provided his company has migrated to system verilog or HVL based verification.

The best scenario is to have at least a few years of experience in all the type work environments, so that you can get good experience in verification methodology, verification tools,languages & different protocols.

VMM Planner

2009-04-30T13:00:00.002+05:30

So what is VMM planner ? VMM planner is a tool which can atomatically annotate the functional coverage and code coverage from regression runs and present the data in a XLS or XML format. It associates the test plan to the test result automatically. The planner can be used for managing verification effort for any project. The basic requirement for using VMM planner is we need to have complete test plan mapped to a functional coverage model. Once we have the plan as an XML or HVP document we can automaticlly annotate the test result using HVP commands. Some of the user provided metrics like bug count , test pass/fail count can be provided to planner tool using the userdata command .We can use VMM planner to report verification status to top level management.

Coverage convergence technology (CCT)

2009-03-20T17:49:00.002+05:30

CCT automates the process of going between coverage goals and determining what constraints to modify, then modifying the constraint to achive the functional coverage goal. CCT also has a provision to automatically generate functional coverage groups from the VERA/NTB/System verilog code based on the constraints specified.

The automatically generated functional coverage code can be used as the starting point for writing the functional coverage model and can be integrated with the DV enviroment. CCT also allows parallel test runs with each test run targetting different coverage points without having any overlap between them. parallel test runs without overlapping random values is achived by providing the tool with a bias file which is generated by the tool based on the functional coverage data base.

Streaming operators --- System verilog

2009-03-01T11:07:00.000+05:30

In VERA/NTB we have vera_pack & vera_unpack methods to pack a class object to bit stream or unpack a bit stream to a class object. System verilog does not have a pack or unpack methods the replacement for this methods are the Streaming operators << , >>. These operaters have the same functionality as the pack/unpack methods. We frequently see the usage of pack and unpack methods while extending the RVM/VMM classes like rvm_data/vmm_data.

Following is an example on the usage of Streaming operators

byte stream[$]; // byte stream

class Packet
rand int header;
rand int len;
rand byte payload[];
int crc;
constraint G { len > 1; payload.size == len ; }
function void post_randomize; crc = payload.sum; endfunction
endclass
...

send: begin // Create random packer and transmit
byte q[$];
Packet p = new;
void’(p.randomize());
q = {<< byte{p.header, p.len, p.payload, p.crc}}; // pack
stream = {stream, q}; // append to stream
end
...

receive: begin // Receive packet, unpack, and remove
byte q[$];
Packet p = new;
{<< byte{ p.header, p.len, p.payload with [0 +: p.len], p.crc }} = stream;
stream = stream[ $bits(p) / 8 : $ ]; // remove packet
end

Automated coverage closure

2009-02-19T17:48:00.001+05:30

Nusym tool does automated coverage closure. It automatically directs the random constraints so as to target coverage points.It automates the process of going between coverage goals and determining what constraints to modify, then modifying the constraint to achive the coverage goal. When we hit bug in random scenario we generally re-play the sequence to fix the bug , but with Nusym's verification tool the same bug is reproduced just at the point of bug , the long sequence of random transaction need not be reproduced to hit the bug. The tool supports both VERA and Systemverilog languages for automatic coverage closure.

Randomization of scalar variables -- System Verilog

2009-02-15T10:34:00.000+05:30

In VERA/NTB to randomize a set of variables we need to have variables in the class add constraints/ in-line constraints to randomize the variables. More over the variables have to be of type rand or randc Assume we have a requirement to randomize a set of variables outside a class with a set of constraints. We have the option of using random(),urandom() or urandom_range() and randomize the variables separately. When we use the above listed random methods we can not randomize a variable based on another variable.

System verilog has an option of randomizing scalar variables out side a class with constraints.

Example

integer a,b,c;
void'( std::randomize(a,b,c) with { a==b;b>0;c==10; } );

The above construct generates random values for a,b,c the constraint is provided in line. The variables a,b,c are out side the class scope.

Coverage grading

2009-02-14T11:43:00.002+05:30

What is coverage grading ?

Coverage grading is an option used to rank the test cases based on the number of functional coverage points hit by the individual test case. Grading option can be used to analyze and remove redundant test cases which is targeting the same functionality. This will help to optimize the regression run and save simulation time.

In random test run scenario it also helps in identifying the random seeds which provides maximum coverage.It will be a good idea to go for functional coverage grading when the verification environment and test cases are frozen and suitable functional coverage numbers are achieved. if your test bench is constantly in development and changing ,using the same seed that gave you good coverage before may not do so again since the randomization may have been affected by the changes in the source code.

Command to generate functional coverage grading in VCS is

urg -dir ( *.vdb ) -grade -metric group

functional coverage

2009-02-08T10:51:00.001+05:30

Code coverage metrics such as line coverage, fsm coverage, expression coverage, block coverage, toggle coverage and branch coverage is extracted automatically by the code coverage tool it gives us the picture of which sections of the RTL have been executed. Root cause analysis can be done on the code coverage holes and suitable test case can be added to cover the RTL functionality.Code coverage has a draw back of not identifying missing features in the RTL. There is no automatic way of getting the correlation between the functionality to be tested and the implementation of the functionality.Lot of manual effort has to put in to get this correlation.

Functional coverage is the determination of how much functionality of the design has been exercised by the verification environment. Functional coverage is a user defined coverage which maps every functionality to be tested (defined in the test plan) to a coverage point. When ever the functionality to be tested is hit in the simulation the functional coverage point is automatically updated. Functional coverage report can be generated which gives us the summary of how many coverage points where hit. Functional coverage metrics can be used as a feedback path to measure the progress of a verification effort.

Adding functional coverage to a verification environment involves three steps.

1. Identifying the functional coverage and cross coverage points.
( directly maps to your test plan )
2. Implementing the functional coverage monitors.
3. Running simulation to collect the functional coverage and functional coverage analysis.

Methodology of identifying functional coverage points and cross coverage points can be explained with a simple USB2.0 Bulk transfer example.

12 diffrent axis have been identified for a simple USB 2.0 bulk transfer. Total no of basic coverage points for this functional coverage group is 34 coverage points. Now we need to get the cross coverage points. So what is a cross coverage point ? Cross coverage is the valid combinations of the identified axis for example one cross coverage point can be

HS(speed) --> IN(direction) -->ACK(response) -->SMALL(length) --> No (data toggle error)

--> No (crc error) -->No ( pid error) --> SMALL (No of tokens per uframe)

--> No ( token error) --> ODD (payload) --> OFF (ping token)

We need to identify all the cross coverage points for this functional coverage group. Each of the cross coverage point is an test scenario. The way to find all the cross coverage point is to simply cross all the axis which will give you all the possible combinations, now eliminate the invalid combinations. Eliminating the invalid cross can be done using ignore construct or bad state construct in VERA/NTB.

example :: finding the cross coverage points for Axis1 and Axis2 alone. ( In the actual scenario all the axis should be taken in to account). possible cross coverage points

HS --> IN, HS -->OUT , FS --> IN , FS-->OUT

4 cross coverage points have been identifyed by crossing Axis 1 and Axis 2.

Now the identified functional coverage / cross coverage points need to be implemented in VERA/NTB/SV as a coverage group and integrated to the verification environment.

Fine-grain process control -- System Verilog

2009-02-07T16:23:00.000+05:30

In Vera/NTB the user has very limited control to the threads/process spawned by fork ..join construct . We have constructs like wait_child() to wait for all the threads to complete. terminate() to kill all threads spawned. What was missing in VERA/NTB was the fine-grain process control which will allow the user to selectively suspend,resume,wait and kill the spawned threads.

System Verilog has a build in process class which can be used for fine-grain process control.This process class is a good addition in system verilog and provides fine-grain process control which was not available in VERA/NTB. The prototype of the process class is as follows.

class process;
enum state { FINISHED, RUNNING, WAITING, SUSPENDED, KILLED };
static function process self();
function state status();
task kill();
task await();
task suspend();
task resume();
endclass

Objects of type process are created internally when processes are spawned. Users cannot create objects of type process; attempts to call new shall not create a new process, and instead result in an error. The process class cannot be extended. Attempts to extend it shall result in a compilation error.

The self() function returns a handle to the current process, that is, a handle to the process making the call. The status() function returns the process status, as defined by the state enumeration:

FINISHED Process terminated normally.

RUNNING Process is currently running (not in a blocking statement).

WAITING Process is waiting in a blocking statement.

SUSPENDED Process is stopped awaiting a resume.

KILLED Process was forcibly killed (via kill or disable).

The await() task allows one process to wait for the completion of another process. It shall be an error to call this task on the current process, i.e., a process cannot wait for its own completion.

The suspend() task allows a process to suspend either its own execution or that of another process. If the process to be suspended is not blocked waiting on some other condition, such as an event, wait expression, or a delay then the process shall be suspended at some unspecified time in the current time step.

The resume() task restarts a previously suspended process.

The kill() task terminates the given process and all its sub-processes, that is, processes spawned using fork statements by the process being killed.

Usage example for the process class

task do_n_way( int N );
process job[1:N];
for ( int j = 1; j <= N; j++ )
fork
automatic int k = j;
begin job[j] = process::self(); ... ; end
join_none
for( int j = 1; j <= N; j++ ) // wait for all processes to start
wait( job[j] != null );
job[1].await(); // wait for first process to finish
for ( int k = 1; k <= N; k++ ) begin
if ( job[k].status != process::FINISHED )
job[k].kill();
end
endtask

System verilog !!!

2009-01-26T11:35:00.004+05:30

System verilog is now being used widely across the industry for any new code development. With the introduction of standard verification methodology like VMM from Synopsys, AVM from mentor, URM from cadence and OVM from cadence and mentor. There is a wide range of verification methodologies to choose from. Companies now are thinking in terms of verification reuse .All the verification methodology tell us how to write a reusable code ,but the real bottle neck is the legacy code written during time where no standard verification methodology like RVM,OVM & VMM existed. The solution to this issue is to re-write the code in more reusable way.

To use system verilog for the newer development in an SOC is also challenging. Luckily in some simulators have an interop mode where we can access HVL code from system verilog and vice versa. This solves the issue of integrating and using the legacy HVL code with system verilog. interop solves integration issues with system verilog but we still have to use two languages ( one for doing enhancements in legacy HVL code and system verilog for new development ). The Solution to this issue is to migrate the VERA/NTB code to system verilog. Most of the features used in VERA/NTB is available in system verilog. Syntax migration from one language to other is the fastest way to port your code and it takes only fraction of your development time. Industry standard conversion tools are available to covert your VERA/NTB code to System verilog or you can develop your own Perl script for migration. Open source perl scripts are also available on the internet to convert code from VERA/NTB to system verilog.