Evaluation based on product success:
The answer looks straight forward, at the end of an emulation effort, chip tape out & chip production. If there are no functional bugs and design works as expected then obviously the person who has verified the design is a good verification engineer.
The above statement has a rider; the above result can be produced in three circumstances
1) Good designer, bad verification engineer & very low bug rate.
2) Bad designer, good verification engineer & very high bug rate.
3) Re-used design which is silicon proven & no bugs.
If your chip taped out successfully without any functional issues with scenario 2 , then you have identified a good verification engineer.
Evaluation based on process success:
You wrote a verification environment, found lot of design bugs, now you need to verify a design enhancement which requires changes in your earlier verification environment. Now effort required to do the changes in your verification environment depends on reusability of code you have written earlier. If you are able add the enhancement within a short span of time with little code changes you are on track to be identified as good verification engineer.
Evaluation when the design success is not immediately visible:
This type of scenario is seen in VIP development where the development is verified internally and product release is done for the customers use.
The only way to identify a good verification engineer under this scenario is “customer bug rate over a fixed time say 12 months” Vs “internal bug rate during development”. If there are no customer bugs for a long period of time on the feature, you have identified a good verification engineer.
Evaluation based on knowledge of language /methodology /protocol:
This is the method most of the people use for identifying a verification engineer during hiring in a new organization. If the person has good coding experience in projects, he will be having a very good command on verification languages and will be well versed with the intricacies of the language. Asking a person to write a code for a given scenario will test his knowledge of the language and test his problem solving skills. Testing a persons knowledge of protocol will help us to know how well he has understood the protocol and used the knowledge in verifying the design.
Evaluation based on moving with technology:
This is very important aspect in today’s industry. The verification methodology, tools improvements happens at a very fast rate from the EDA vendors helping the verification engineers to reduce the time on verification. A good verification engineer will definitely keep himself updated on the new aspects of functional verification which is good sign in identifying a good verification engineer.
I have also in my past come across some sense less interviewers evaluating verification hires on his knowledge of digital electronics and CMOS. Does a verification engineer use digital design or CMOS for architecting or writing his test bench?