Using a build trace to troubleshoot build specification problemsUsing a build trace to troubleshoot build specification problemsYou can use kwinject to generate a build trace, a file that contains the sequence of process calls that took place during your C/C++ build. A build trace can be useful for:
Build tracea file that contains the sequence of process calls that took place during your C/C++ build. Can be generated by kwinject as an intermediate troubleshooting step in generating a C/C++ build specification. See Using a build trace to troubleshoot build specification problems for how-to information. See also Build trace file format. Generating a build tracekwinject [options] --trace-out <trace_file> <build_command> where
Example build traceHere's an excerpt from an example build trace file:
{"version":100}
{"creator":"kwinject, version 2018.1.0"}
{"env":{"LIBGL_DRIVERS":"/usr/lib/fglrx/dri:/usr/lib32/dri", "MAKEFLAGS": " --jobserver-fds=3"}}
{"id":26008,"parent_id":-1,"work_dir":"/space/zlib-1.2.8","executable":"/space/gcc-4.9.0/bin/gcc","args":["gcc","-D_LARGEFILE64_SOURCE=1","-DHAVE_HIDDEN","-c","crc32.c"],"env-diff":{"ANT_HOME":"/opt/apache-ant-1.7","COLORTERM":"gnome-terminal","JAVA_HOME":"/opt/sun-jdk-1.6","LANG":"en_US.UTF-8"}}
{"id":26008,"parent_id":-1,"work_dir":"/space/zlib-1.2.8","executable":"/space/gcc-4.9.0/bin/gcc","args":["gcc","-D_LARGEFILE64_SOURCE=1","-DHAVE_HIDDEN","-c","foo.c"],"env-diff":{"ANT_HOME":"/opt/apache-ant-1.7","COLORTERM":"gnome-terminal","JAVA_HOME":"/opt/sun-jdk-1.6","LANG":"en_US.UTF-8"}}
...
{"id":26008,"parent_id":-1,"work_dir":"/space/zlib-1.2.8","executable":"/space/gcc-4.9.0/bin/gcc","args":["gcc","-D_LARGEFILE64_SOURCE=1","-DHAVE_HIDDEN","-c","bar32.c"],"env-diff":{"ANT_HOME":"/opt/apache-ant-1.7","COLORTERM":"gnome-terminal","JAVA_HOME":"/opt/sun-jdk-1.6","LANG":"en_US.UTF-8"}}
For the format of the build trace file and additional examples, see Build trace file format. Reviewing the build trace for your compiler and linker namesTypically, compiler and linker names are mentioned several times in a trace file. You need to identify the name of your root compiler/linker, and not your compiler/linker tool chain. To locate the compiler name, search for the name of a file that you know is being built (something with a .c or .cpp extension). In this example, the compiler name is g++: { "id": 5396, "parentid": 6024, "workdir": "C:\\tutorial\\npp\\PowerEditor\\src", "executable": "C:\\Perl64\\site\\bin\\g++.exe", "args": [ "g++", "foo.cpp" ], To see a more complete example of a build trace, see Build trace file format. Checking whether your compiler is recognized by kwinjectOnce you've identified your compiler/linker name, check the name against the list of C/C++ compilers supported for build integration. If your compiler is:
Converting the build trace to a build specificationTo convert the build trace to a build specification: kwinject [options] --trace-in <trace_file> where<trace_file> is the input trace file What's next?Now that you've created a build specification, you're ready to run the C/C++integration build analysis. Or, if you're running distributed builds, go to Running a distributed Klocwork C/C++ analysis. Build trace file formatThe build trace file is a log of your C/C++ build, an unfiltered execution trace, which can be processed by kwinject to generate a build specification. The build trace file is output in the JSON file format. The file contains:
Root level fieldsThe top level of the JSON file contains basic information about the trace file, as well as the env object and one or more id objects (or event objects). Each top level object (version, creator, top-level env or id) must go on a single separate line. Version, creator, and env must be listed before the event(s). Below is an example of the root level of a typical JSON build trace file: {"version":101} {"creator":"kwinject, version 10.1.1"} {"env":{"LIBGL_DRIVERS":"/usr/lib/fglrx/dri:/usr/lib32/dri", "MAKEFLAGS": " --jobserver-fds=3"}} {"id":26008,"parent_id":-1,"work_dir":"/space/zlib-1.2.8","executable":"/space/gcc-4.9.0/bin/gcc","args":["gcc","-D_LARGEFILE64_SOURCE=1","-DHAVE_HIDDEN","-c","crc32.c"],"env-diff":{"ANT_HOME":"/opt/apache-ant-1.7","COLORTERM":"gnome-terminal","JAVA_HOME":"/opt/sun-jdk-1.6","LANG":"en_US.UTF-8"}} {"id":26008,"parent_id":-1,"work_dir":"/space/zlib-1.2.8","executable":"/space/gcc-4.9.0/bin/gcc","args":["gcc","-D_LARGEFILE64_SOURCE=1","-DHAVE_HIDDEN","-c","foo.c"],"env-diff":{"ANT_HOME":"/opt/apache-ant-1.7","COLORTERM":"gnome-terminal","JAVA_HOME":"/opt/sun-jdk-1.6","LANG":"en_US.UTF-8"}} ... {"id":26008,"parent_id":-1,"work_dir":"/space/zlib-1.2.8","executable":"/space/gcc-4.9.0/bin/gcc","args":["gcc","-D_LARGEFILE64_SOURCE=1","-DHAVE_HIDDEN","-c","bar32.c"],"env-diff":{"ANT_HOME":"/opt/apache-ant-1.7","COLORTERM":"gnome-terminal","JAVA_HOME":"/opt/sun-jdk-1.6","LANG":"en_US.UTF-8"}} The fields are:
The id (event) objectThe id object contains information about actual system execution events that occur as part of your build. A new id object is created for each process and sub process that occur as part of your build. The fields are:
ExampleThe following example shows a complete build trace file that was created by kwinject. It captures the output of the g++ compiler compiling a simple C++ file called foo.cpp. { "version": 100, "creator": "kwinject, version 2018.1.0", "env": { "ALLUSERSPROFILE": "C:\\ProgramData", "ANT_HOME": "C:\\apache-ant-1.8.4", "APPDATA": "C:\\Users\\jdoe\\AppData\\Roaming", "ASL.LOG": "Destination=file", "COMMONPROGRAMFILES": "C:\\Program Files (x86)\\Common Files", "COMMONPROGRAMFILES(X86)": "C:\\Program Files (x86)\\Common Files", "COMMONPROGRAMW6432": "C:\\Program Files\\Common Files", "COMPUTERNAME": "JDOE2", "COMSPEC": "C:\\Windows\\system32\\cmd.exe", "FP_NO_HOST_CHECK": "NO", "HOMEDRIVE": "C:", "HOMEPATH": "\\Users\\jdoe", "JAVA_HOME": "C:\\Program Files\\Java\\jdk1.7.0_07", "LOCALAPPDATA": "C:\\Users\\jdoe\\AppData\\Local", "LOGONSERVER": "\\\\SERVER01", "NUMBER_OF_PROCESSORS": "4", "OS": "Windows_NT", "PATH": "C:\\Klocwork\\Server 2018.1\\bin;C:\\Program Files\\CollabNet\\Subversion Client;C:\\Perl64\\site\\bin;C:\\Perl64\\bin;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\apache-ant-1.8.4\\bin;C:\\Program Files\\jEdit;C:\\Program Files (x86)\\BaseX\\bin;c:\\Program Files (x86)\\Microsoft SQL Server\\90\\Tools\\binn\\;C:\\Program Files (x86)\\Microsoft Visual Studio 2008 SDK\\VisualStudioIntegration\\Tools\\Sandcastle\\ProductionTools\\;C:\\Python26;C:\\Program Files (x86)\\Microsoft Visual Studio 2018.1\\VC\\bin;C:\\bin\\curl.exe", "PATHEXT": ".COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC", "PROCESSOR_ARCHITECTURE": "x86", "PROCESSOR_ARCHITEW6432": "AMD64", "PROCESSOR_IDENTIFIER": "Intel64 Family 6 Model 58 Stepping 9, GenuineIntel", "PROCESSOR_LEVEL": "6", "PROCESSOR_REVISION": "3a09", "PROGRAMDATA": "C:\\ProgramData", "PROGRAMFILES": "C:\\Program Files (x86)", "PROGRAMFILES(X86)": "C:\\Program Files (x86)", "PROGRAMW6432": "C:\\Program Files", "PROMPT": "$P$G", "PSMODULEPATH": "C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Modules\\", "PUBLIC": "C:\\Users\\Public", "SESSIONNAME": "Console", "SYSTEMDRIVE": "C:", "SYSTEMROOT": "C:\\Windows", "TEAMCITY_DATA_PATH": "C:\\ProgramData\\JetBrains\\TeamCity", "TEMP": "C:\\Users\\jdoe\\AppData\\Local\\Temp", "TFS_DIR": "C:\\Program Files\\ThinkVantage Fingerprint Software\\", "TMP": "C:\\Users\\jdoe\\AppData\\Local\\Temp", "USERDNSDOMAIN": "COMPANY.COM", "USERDOMAIN": "WORKGROUP", "USERNAME": "jdoe", "USERPROFILE": "C:\\Users\\jdoe", "VS100COMNTOOLS": "c:\\Program Files (x86)\\Microsoft Visual Studio 2018.1\\Common7\\Tools\\", "VS90COMNTOOLS": "c:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\Common7\\Tools\\", "VSSDK90INSTALL": "C:\\Program Files (x86)\\Microsoft Visual Studio 2008 SDK\\", "WINDIR": "C:\\Windows", "WINDOWS_TRACING_FLAGS": "3", "WINDOWS_TRACING_LOGFILE": "C:\\BVTBin\\Tests\\installpackage\\csilogfile.log" }, "events": [ { "id": 5396, "parentid": 6024, "workdir": "C:\\tutorial\\npp\\PowerEditor\\src", "executable": "C:\\Perl64\\site\\bin\\g++.exe", "args": [ "g++", "foo.cpp" ], "env-diff": { "=::": "::\\", "=C:": "C:\\tutorial\\npp\\PowerEditor\\src", "=EXITCODE": "00000001", "COMMONPROGRAMFILES": "C:\\Program Files\\Common Files", "KW_INST_DIR": "C:\\Klocwork\\Server 2018.1\\", "MSBUILDDISABLENODEREUSE": "1", "PROCESSOR_ARCHITECTURE": "AMD64", "PROCESSOR_ARCHITEW6432": null, "PROGRAMFILES": "C:\\Program Files" } }, { "id": 2520, "parentid": 5396, "workdir": "C:\\tutorial\\npp\\PowerEditor\\src", "executable": "C:\\Perl64\\site\\lib\\auto\\MinGW\\bin\\g++.exe", "args": [ "C:\\Perl64\\site\\lib\\auto\\MinGW\\bin\\g++.exe", "foo.cpp" ], "env-diff": { "COMMONPROGRAMFILES": "C:\\Program Files (x86)\\Common Files", "PROCESSOR_ARCHITECTURE": "x86", "PROCESSOR_ARCHITEW6432": "AMD64", "PROGRAMFILES": "C:\\Program Files (x86)" } }, { "id": 6704, "parentid": 2520, "workdir": "C:\\tutorial\\npp\\PowerEditor\\src", "executable": "c:\\perl64\\site\\lib\\auto\\mingw\\bin\\..\\libexec\\gcc\\x86_64-w64-mingw32\\4.5.4\\cc1plus.exe", "args": [ "c:/perl64/site/lib/auto/mingw/bin/../libexec/gcc/x86_64-w64-mingw32/4.5.4/cc1plus.exe", "-quiet", "-iprefix", "c:\\perl64\\site\\lib\\auto\\mingw\\bin\\../lib/gcc/x86_64-w64-mingw32/4.5.4/", "-isysroot", "c:\\perl64\\site\\lib\\auto\\mingw\\bin\\../../mingw64", "foo.cpp", "-quiet", "-dumpbase", "foo.cpp", "-mtune=generic", "-march=x86-64", "-auxbase", "foo", "-o", "C:\\Users\\jdoe\\AppData\\Local\\Temp\\ccUkGgFd.s" ], "env-diff": { "COLLECT_GCC": "C:\\Perl64\\site\\lib\\auto\\MinGW\\bin\\g++.exe", "COLLECT_GCC_OPTIONS": "'-shared-libgcc' '-mtune=generic' '-march=x86-64'", "COLLECT_LTO_WRAPPER": "c:/perl64/site/lib/auto/mingw/bin/../libexec/gcc/x86_64-w64-mingw32/4.5.4/lto-wrapper.exe", "COMMONPROGRAMFILES": "C:\\Program Files\\Common Files", "GCC_EXEC_PREFIX": "c:\\perl64\\site\\lib\\auto\\mingw\\bin\\../lib/gcc/", "PROCESSOR_ARCHITECTURE": "AMD64", "PROCESSOR_ARCHITEW6432": null, "PROGRAMFILES": "C:\\Program Files" } }, { "id": 8924, "parentid": 2520, "workdir": "C:\\tutorial\\npp\\PowerEditor\\src", "executable": "c:\\perl64\\site\\lib\\auto\\mingw\\bin\\..\\lib\\gcc\\x86_64-w64-mingw32\\4.5.4\\..\\..\\..\\..\\x86_64-w64-mingw32\\bin\\as.exe", "args": [ "c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/bin/as.exe", "-o", "C:\\Users\\jdoe\\AppData\\Local\\Temp\\cc3VdMAp.o", "C:\\Users\\jdoe\\AppData\\Local\\Temp\\ccUkGgFd.s" ], "env-diff": { "COLLECT_GCC": "C:\\Perl64\\site\\lib\\auto\\MinGW\\bin\\g++.exe", "COLLECT_GCC_OPTIONS": "'-shared-libgcc' '-mtune=generic' '-march=x86-64'", "COLLECT_LTO_WRAPPER": "c:/perl64/site/lib/auto/mingw/bin/../libexec/gcc/x86_64-w64-mingw32/4.5.4/lto-wrapper.exe", "COMMONPROGRAMFILES": "C:\\Program Files\\Common Files", "GCC_EXEC_PREFIX": "c:\\perl64\\site\\lib\\auto\\mingw\\bin\\../lib/gcc/", "PROCESSOR_ARCHITECTURE": "AMD64", "PROCESSOR_ARCHITEW6432": null, "PROGRAMFILES": "C:\\Program Files" } }, { "id": 7824, "parentid": 2520, "workdir": "C:\\tutorial\\npp\\PowerEditor\\src", "executable": "c:\\perl64\\site\\lib\\auto\\mingw\\bin\\..\\libexec\\gcc\\x86_64-w64-mingw32\\4.5.4\\collect2.exe", "args": [ "c:/perl64/site/lib/auto/mingw/bin/../libexec/gcc/x86_64-w64-mingw32/4.5.4/collect2.exe", "--sysroot=c:\\perl64\\site\\lib\\auto\\mingw\\bin\\../../mingw64", "-m", "i386pep", "-Bdynamic", "c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/lib/crt2.o", "c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/lib/crtbegin.o", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../lib64", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/lib", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../..", "C:\\Users\\jdoe\\AppData\\Local\\Temp\\cc3VdMAp.o", "-lstdc++", "-lmingw32", "-lgcc_s", "-lgcc", "-lmoldname", "-lmingwex", "-lmsvcrt", "-luser32", "-lkernel32", "-ladvapi32", "-lshell32", "-lmingw32", "-lgcc_s", "-lgcc", "-lmoldname", "-lmingwex", "-lmsvcrt", "c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/lib/crtend.o" ], "env-diff": { "COLLECT_GCC": "C:\\Perl64\\site\\lib\\auto\\MinGW\\bin\\g++.exe", "COLLECT_GCC_OPTIONS": "'-shared-libgcc' '-mtune=generic' '-march=x86-64'", "COLLECT_LTO_WRAPPER": "c:/perl64/site/lib/auto/mingw/bin/../libexec/gcc/x86_64-w64-mingw32/4.5.4/lto-wrapper.exe", "COMMONPROGRAMFILES": "C:\\Program Files\\Common Files", "COMPILER_PATH": "c:/perl64/site/lib/auto/mingw/bin/../libexec/gcc/x86_64-w64-mingw32/4.5.4/;c:/perl64/site/lib/auto/mingw/bin/../libexec/gcc/;c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/bin/", "GCC_EXEC_PREFIX": "c:\\perl64\\site\\lib\\auto\\mingw\\bin\\../lib/gcc/", "LIBRARY_PATH": "c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/;c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/;c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../lib64/;c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/lib/;c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../", "PROCESSOR_ARCHITECTURE": "AMD64", "PROCESSOR_ARCHITEW6432": null, "PROGRAMFILES": "C:\\Program Files" } }, { "id": 8440, "parent_id": 7824, "work_dir": "C:\\tutorial\\npp\\PowerEditor\\src", "executable": "c:\\perl64\\site\\lib\\auto\\mingw\\bin\\..\\lib\\gcc\\x86_64-w64-mingw32\\4.5.4\\..\\..\\..\\..\\x86_64-w64-mingw32\\bin\\ld.exe", "args": [ "c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/bin/ld.exe", "--sysroot=c:\\perl64\\site\\lib\\auto\\mingw\\bin\\../../mingw64", "-m", "i386pep", "-Bdynamic", "c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/lib/crt2.o", "c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/lib/crtbegin.o", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../lib64", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/lib", "-Lc:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../..", "C:\\Users\\jdoe\\AppData\\Local\\Temp\\cc3VdMAp.o", "-lstdc++", "-lmingw32", "-lgcc_s", "-lgcc", "-lmoldname", "-lmingwex", "-lmsvcrt", "-luser32", "-lkernel32", "-ladvapi32", "-lshell32", "-lmingw32", "-lgcc_s", "-lgcc", "-lmoldname", "-lmingwex", "-lmsvcrt", "c:/perl64/site/lib/auto/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.5.4/../../../../x86_64-w64-mingw32/lib/crtend.o" ] } ] } Note: The build trace file format changed in Klocwork version 2018.1. Note that previous versions of the build trace file format are still supported by kwinject.
Compiler mapping file format (kwfilter.conf)To obtain a list of all known filters and to determine which filter to use for an intercepted command (since several different commands can use the same option filter), kwinject and kwwrap use a master compiler filter file, located at <klocwork_install>/config/kwfilter.conf . This master compiler filter file is a tab- and comma-separated text file. Empty strings and comments (lines starting with # symbol) are ignored. You may need to edit this file if:
Filter binding linesThe filter bindings specify which filter should be used by specific commands. The binding lines have the following syntax: filter <compiler-type> <prog-name>[, <prog-name> ...] where
For example: # Default GNU C/C++ compiler filter gnu gcc, g++ This specifies that the gcc and g++ compiler executables belong to the GNU compiler type. This filter is described in a file named gnu_filter.py.
Use a wildcard to specify a set of related compiler executablesYou can use a wildcard(*) to bind a group of similarly named executables. For example: #More GNU C/C++ compilers filter gnu *ccpentium, *-xt-xcc This will match all executables that end in the pattern ccpentium and -xt-xcc. Wildcards can be used anywhere in the compiler name. For example: filter gnu *-gcc, *-cc, gcc-*, gcc1.*.mips Note that the period character is always matched to the period (.) Note: Because entries in kwfilter.conf are matched in order of appearance, it's a good idea to use wildcards only at the end of the file.
Attach and detach keywords (Windows only)Occasionally, kwinject's process interception/monitoring mechanism can interfere with the process itself. On Windows, you can use the attach and detach keywords to specify a list of commands which kwinject should not intercept when monitoring your build. kwinject will detach from the specified processes as soon as they are started. For instructions, see Local build processes failing in kwinject-enabled builds. Deploying kwfilter.confBy default, the kwfilter.conf file is located in the <klocwork_install>/config directory for both the server and the desktop analysis tools. To deploy a custom kwfilter.conf file to clients or desktop users, you'll need to copy your custom kwfilter.conf file into the projects_root/compiler_config directory, then run the kwdeploy command to deploy it to the server. To deploy your compiler settings to a server project that has connected desktops:
When the kwfilter.conf file is synced, a version of it is stored in the <klocwork_install>/synced/config directory. When you run kwinject, it searches for the kwfilter.conf file in the following locations, in order:
The system merges kwfilter.conf entries from each kwfilter.conf file. For example, when run, kwinject will pick up entries from both <klocwork_install>/synced/config/kwfilter.conf and <klocwork_install>/config/kwfilter.conf. kwinject will use the first entry it encounters that matches the compiler name. If a user connects to a different server that maintains different compiler settings, the kwfilter.conf file is synced to the desktop and any local compiler config changes are synced using changes from the kwfilter.conf file on the server. |