Import external defects with kwdefectimport
Import external defects with kwdefectimportYou can use Kwdefectimport to import a list of external defects into Klocwork. These are defects that were discovered by an external tool or process, independent of Klocwork. You run Kwdefectimport on the command line at build time after Kwbuildproject and before kwadmin -load. When run, it uses the data in the artifacts (tables and files) that are created by kwbuildproject, then creates new problem data for kwadmin to load. ![]() Importing external defects into Klocwork involves the following steps:
Working with source defect filesWhile the exact format of your defect source can vary, the data must be valid and it must contain at least the following four elements per defect:
The following listing shows a typical defect source file. <ErrorId Id="ERR_TYP_TIMER_NOT_ACTIVE" NumberOfEntries="4"> <Msg F="czagenmx.sdl" L="2786" S="C">timer t_wf_czg not activated in any path</Msg> <Msg F="czahanmx.sdl" L="5051" S="C">timer t_wf_czg_sup_ack not activated in any path</Msg> <Msg F="czahanmx.sdl" L="5074" S="C">timer t_wf_czg_sup_ack not activated in any path</Msg> <Msg F="czahanmx.sdl" L="5092" S="C">timer t_wf_czg_sup_ack not activated in any path</Msg> </ErrorId> Note: The sample also contains the S attribute which represents the severity level of each defect. Since the severity level of each defect is defined in the taxonomy, this attribute can be ignored by the parser.
Create a taxonomyBefore you can import defects, you must define the defect taxonomies, categories, and defect types with which the external defects will be associated. This is done by creating a .pconf file, which is an XML file that defines the categories and severity types of the defect, and contains the error message that is displayed to the user. The following example shows a snippet from a .pconf file.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <checkers version="1.4"> <severitytable max="10" warningStart="3" locale="en"> <severity name="Critical" number="1"/> ... </severitytable> <categories name="TN SDL SDT" locale="en"> <category name="Err.Lex.Tn"> <error id="ERR_LEX_COM_IN_COM"/> <error id="ERR_LEX_LONG_ID"/> <error id="ERR_LEX_RESERVED"/> </category> ... </categories> <checkergroup api="none" id="TN.DEFAULT.CHECKERGROUP" language="SDL/SDT"> <checker id="ERR.LEX.TN"> <error autogroupby="0" enabled="true" id="ERR_LEX_LONG_ID" severity="3"> <title locale="en" text="ID longer than 31 characters"/> <title locale="ja" text="ID longer than 31 characters"/> <message locale="en" text="{0}"/> <message locale="ja" text="{0}"/> </error> <error autogroupby="0" enabled="true" id="ERR_LEX_RESERVED" severity="3"> <title locale="en" text="ID is reserved in ANSI C"/> <title locale="ja" text="ID is reserved in ANSI C"/> <message locale="en" text="{0}"/> <message locale="ja" text="{0}"/> </error> <error autogroupby="0" enabled="true" id="ERR_LEX_COM_IN_COM" severity="3"> <title locale="en" text="Missing close comment"/> <title locale="ja" text="Missing close comment"/> <message locale="en" text="{0}"/> <message locale="ja" text="{0}"/> </error> </checker> ... </checkergroup> </checkers> The .pconf file contains three child elements that you must define and configure:
Create your .pconf fileFollow the walk-through below to create and import your .pconf file.
Create a defect parserTo import defects into Klocwork, you must write a Java class that implements the interface com.klocwork.defectimport.IDefectParser. In addition, this implementation must have a public single argument constructor that accepts a String. This String represents the resource that contains the listing of external defects to be imported into Klocwork. This resource can be a file, a stream, a port or any other user-defined source. In the example below, it represents the full path to the file that contains the defects. Note also that there is no public method that can be explicitly called from within Kwdefectimport that will create an Iterator to satisfy the following interface method:
Iterator<SourceDefect> getDefects();Therefore, the parser must prepare this Iterator from within its constructor. In the example below, see the call to the implementation-specific method, parse(), from within the constructor. Sample parserIn this complete example of a parser, the defects are exported as XML. The implementation of IDefectParser extends SAX's DefaultHandler: package com.thirdparty.defectexport.parser; import com.klocwork.defectimport.IDefectParser; import com.klocwork.defectimport.SourceDefect; import org.jetbrains.annotations.NonNls; import org.xml.sax.Attributes; import org.xml.sax.SAXException; import org.xml.sax.XMLReader; import org.xml.sax.helpers.DefaultHandler; import javax.xml.parsers.ParserConfigurationException; import javax.xml.parsers.SAXParser; import javax.xml.parsers.SAXParserFactory; import java.io.File; import java.io.IOException; import java.util.*; public class ThirdPartyKlocworkDefectParser extends DefaultHandler implements IDefectParser { //where the listing of the source defects resides (file, stream?, port?, etc) private final String defectsSource; //list of items discovered at the defectsSource, each of which having been converted to a SourceDefect private final List<SourceDefect> sourceDefects = new ArrayList<SourceDefect>(); private SourceDefect sourceDefect; private boolean inByErrorId; private boolean inMsg; private String currentErrorId = ""; StringBuilder buffer = new StringBuilder(1024); public ThirdPartyKlocworkDefectParser (final String defectsSource) throws ParserConfigurationException, SAXException, IOException { this.defectsSource = defectsSource; parse(); } @Override public void startElement(final String uri, final String localName, final String qName, final Attributes atts) throws SAXException { buffer.setLength(0); if(localName.equals(ELEM_BY_ERROR_ID)){ inByErrorId = true; } if(inByErrorId && localName.equals(ELEM_ERROR_ID)){ final String attributeId = atts.getValue(ATTR_ID); if(!attributeId.equals(currentErrorId)){ currentErrorId = attributeId; } } if(inByErrorId && localName.equals(ELEM_MSG)){ inMsg = true; sourceDefect = new SourceDefect(); sourceDefect.setErrorId(currentErrorId); final String fileName = atts.getValue(ATTR_FILE); sourceDefect.setFileName(fileName); sourceDefect.setLineNumber(Integer.valueOf(atts.getValue(ATTR_LINE))); sourceDefect.setMessage(qName); } } @Override public void endElement (final String uri, final String localName, final String qName) throws SAXException { if(inByErrorId && localName.equals(ELEM_MSG)){ sourceDefect.setMessage(buffer.toString()); sourceDefects.add(sourceDefect); inMsg = false; } } @Override public void characters(final char[] ch, final int start, final int length) throws SAXException { if(inByErrorId && inMsg){ buffer.append(ch, start, length); } } @Override public Iterator<SourceDefect> getDefects() { return Collections.unmodifiableCollection(sourceDefects).iterator(); } private void parse() throws ParserConfigurationException, SAXException, IOException { final SAXParserFactory spf = SAXParserFactory.newInstance(); spf.setNamespaceAware(true); final SAXParser parser = spf.newSAXParser(); final XMLReader reader = parser.getXMLReader(); reader.setContentHandler(this); reader.parse(new File(defectsSource).toURI().toURL().toString()); } @Override public void close() { //not required for this implementation } @SuppressWarnings("HardCodedStringLiteral") @NonNls private static final String ELEM_BY_ERROR_ID = "ByErrorId"; @SuppressWarnings("HardCodedStringLiteral") @NonNls private static final String ELEM_ERROR_ID = "ErrorId"; @SuppressWarnings("HardCodedStringLiteral") @NonNls private static final String ATTR_ID = "Id"; @SuppressWarnings("HardCodedStringLiteral") @NonNls private static final String ELEM_MSG = "Msg"; @SuppressWarnings("HardCodedStringLiteral") @NonNls private static final String ATTR_FILE = "F"; @SuppressWarnings("HardCodedStringLiteral") @NonNls private static final String ATTR_LINE = "L"; } Import the defectsOnce you've defined your external defects file, you've mapped those defects to a checker configuration, and you've written your parser, you can run kwdefectimport to import the external defects into Klocwork. You run Kwdefectimport from the command line, after running Kwbuildproject, but before running kwadmin -load: At runtime, kwdefectimport opens the source defects listing file, reads the defect types and maps them to the appropriate source file and line of code. In the tables directory the files file.dat, and entity.dat are modified. Defects are written to a new file called problem_ext.pbf which is then loaded by kwadmin -load. Run kwdefectimport on the command lineThe following example shows kwdefectimport run from the command line: kwdefectimport --tables-directory "C:/Klocwork/Server XY.Z/projects_root/projects/abc_main" --working-directory "C:/_workspaces/ABC_Main/src" --parser-class com.klocwork.defectimport.parser.TNSDLDefectParser --rejected-defects C:/tables/rejecteddefects.json --ext-libs "C:/Klocwork/lib/kwdefectimport.jar" --ext-libs "C:/Klocwork/lib/another_required_library.jar" "C:\already_analyzed_defects_in_proprietary_format\defects_01.xml" "C:\already_analyzed_defects_in_proprietary_format\defects_02.xml For more information about running kwdefectimport and for a complete description of the options, see Kwdefectimport. Running kwdefectimport as part of an automated buildIf you include kwdefectimport as a regular part of your automated Klocwork tool chain (between kwbuildproject and kwadmin -load), you will require kwdefectimport's exit code to determine whether to continue. See Kwdefectimport for a complete list of error codes. The following example uses a DOS batch script to demonstrate how to use the exit code to determine the action to take based on the exit code. In this example, there are 2 batch files: caller.bat and run_kwdefectimport.bat The file caller.bat is responsible for executing kwdefectimport and returning its exit code to the caller. The file run_kwdefectimport.bat calls caller.bat and then makes execution decisions based on thecaller.bat's exit code. run_kwdefectimport.bat @echo off set errorlevel= "C:\Klocwork\Server 12.0\bin\kwdefectimport.exe" --tables-directory C:\tables\20131024 --working-directory "C:/_workspaces/Main/REP/sources/kwdefectimport/test_resources/source_dir" --parser-class com.klocwork.defectimport.parser.TNSDLDefectParser --rejected-defects C:/tables/rejecteddefects.json "C:/_workspaces/Main/REP/sources/kwdefectimport/test_resources/sdl_defects/tncheck.xml" exit /b %errorlevel% caller.bat caller.bat @echo off call run_kwdefectimport.bat echo Exit Code is %errorlevel% set atemp=%errorlevel% echo atemp is %atemp% set BUILD_FAILED_EXIT_CODE=1 set DEFECT_TYPES_NOT_REGISTERED_EXIT_CODE=16 set DUPLICATE_DEFECTS_FOUND_EXIT_CODE=32 set SOURCE_FILES_NOT_FOUND_EXIT_CODE=64 set SOURCE_FILES_NOT_REFERENCED_EXIT_CODE=128 set /A critical="%atemp%&%BUILD_FAILED_EXIT_CODE%" set /A dnr="%atemp%&%DEFECT_TYPES_NOT_REGISTERED_EXIT_CODE%" set /A dup="%atemp%&%DUPLICATE_DEFECTS_FOUND_EXIT_CODE%" set /A snf="%atemp%&%SOURCE_FILES_NOT_FOUND_EXIT_CODE%" set /A snr="%atemp%&%SOURCE_FILES_NOT_REFERENCED_EXIT_CODE%" echo dnr %dnr% echo dup %dup% echo snf %snf% echo snr %snr% if "%dnr%" == "%DEFECT_TYPES_NOT_REGISTERED_EXIT_CODE%" echo Defect type(s) not registered rem if "%dnr%" == 16 goto whatever_you_want_to_do_with_this_case if "%dup%" == "%DUPLICATE_DEFECTS_FOUND_EXIT_CODE%" echo Duplicate defect(s) found rem as above if "%snf%" == "%SOURCE_FILES_NOT_FOUND_EXIT_CODE%" echo Source file(s) not found rem as above if "%snr%" == "%SOURCE_FILES_NOT_REFERENCED_EXIT_CODE%" echo Source file(s) not referenced rem as above if "%critical%" == "1" echo BUILD FAILED rem You probably don't want to continue REM place actions to take here pause rem Pause would _not_ be used in a production environment |