Oovaide Builder Design

Created: Jan 10, 2014
Modified: July 17, 2015

  1. Oovaide Builder Design
    1. Build Design Goals
    2. Tasks performed by Oovaide (GUI)
    3. Tasks performed by OovBuilder
      1. Analyze
        1. Check build configurations
        2. Scan external root directories
        3. Scan external package directories
        4. Scan project directories
        5. Pre-Build Cleanup
        6. Generate Analysis Model Data
      2. Build
        1. Determine Component and Package Dependencies
        2. Build object files
        3. Build library files
        4. Build executables
    4. Tasks performed by OovCppParser
      1. Scan project source
    5. Other optimizations

Build Design Goals

The design of the build system is to be as easy as possible for the user.  The general idea is:

Tasks performed by Oovaide (GUI)

This document describes a sequence starting without any preexisting oovaide project files or any type of make files. The Oovaide program performs the following tasks.
Output:

File Content
oovaide.txt
  • Project source root directory
  • List of directories to exclude from scanning
  • Tools for each build configuration
  • Arguments specific to each build configuration
oovaide-pkg.txt Defines every external package (-EP) used by the project. Each package defines:
  • Package root directory - wildcards must be resolved for analysis.
  • Include directories - can be a wildcard (*)
  • Library directories
  • Library names - can be a wildcard (*)
oovaide-comptypes.txt See the section "Scan project directories" for the details. Oovaide is typically used to set up the component types after the project directory has been scanned.

Tasks performed by OovBuilder

The OovBuilder uses the output from the Oovaide program as inputs to define the build.  These input files are fairly simple text files, and could potentially be generated by another tool.  The OovBuilder command line arguments are simply

OovBuilder calculates CRC's for sets of build arguments so that unique configurations of build data are regenerated whenever the arguments are changed. This allows switching between build configurations quickly while retaining data.

Analyze

The Analyze functionality is performed whenever the mode switch is "-mode-analyze", or "-mode-build".

Check build configurations

CRC's are calculated for the project arguments, so that when the arguments are changed, stale data can be discarded.

See the Build CRC's section for the types of CRC's that are created and compared.

If the oovaide-tmp-buildpkg.txt file is older than the oovaide-pkg.txt file, then the output file is deleted.

[WARNING: At this time, all external project directories are considered non-volatile, user has to erase project analysis contents (including analysis directory, and oovaide-buildpkg.txt) manually if the contents change in a way that will change include or library dependencies.]

[WARNING: If source files are deleted, the analysis information needs to be deleted. (include dependencies and source lists must be updated)]

[WARNING - The analysis directories are not kept unique for the clang and GCC compilers. Only arguments are used to make unique analysis directories. If they are using the same directory and the include dependencies must be created differently, set an argument that is different.]

[OPTIMIZE: The project could specify volatile or non-volatile external directories.]

Scan external root directories

The directories that were specified using -ER in the project are scanned for include files and executables.Scanning the external directories is only done if the "External" CRC config is different for the current build configuration.

Scan external package directories

Makes a copy of the oovaide-pkg.txt file named oovaide-tmp-buildpkg.txt, and updates it with actual include paths and library names found during scanning. If the library names is a wildcard (*), search for libraries below the package root directory and add them as library names. If the include directories is a wildcard, search for header files below the package root directory, and add the include paths to the include directories.

Note that the GNU or CLang compiler will automatically search some include paths. These external paths do not have to be specified to Oovaide. See the Oovaide user guide for more information.

Output:

File Content
oovaide-tmp-buildpkg.txt Contains a copy of the project packages where all of the wildcarded include paths and library names are listed.  The library names are not in sorted dependency order yet.

Scan project directories

Search the project from the project root directory for source and include files. This recusively create include and library paths that are saved in the compsources.txt file.

The component names are simply the names of the directories in the project.  Each component name can be assigned a component type in the Oovaide program.  Examples of component types are static library, executable, etc.

Output:
File
Content
oovaide-comptypes.txt
  • Component names found during scanning.
  • Component types - the component type starts as the default "Unknown". (Oovaide is used to set the component types)
oovaide-tmp-compsources.txt
  • Source files defined in each component (used by oovaide to display component tree)


[OPTIMIZE: unused components remain in the oovaide-buildconfig.txt file, but they do not cause any harm.]

Pre-Build Cleanup

Check the CRC's of arguments to detect changes.

Generate Analysis Model Data

If any analysis (XMI extension) files are out of date compared to the inputs, they are recreated. Analysis of each file is performed by a different thread and process, up to the number of cores or processors in the system.
If the analysis content does not exist, create an analysis-<crc> directory for the XMI files, oovaide-incdeps.txt and  oovaide-comps.txt file.Each time a build is performed, multiple types of CRC's are saved in the oovaide-tmp-buildconfig.txt file.

Output:
File Content
oovaide-tmp-buildconfig.txt Each build configuration has the following CRC's along with the text used to produce the CRC's.
  • Analysis - This combined from the "External" and the "Project"
  • External (-ER)
  • Project - All include directories in the project
  • Link Arguments (From cppArgs)(-lnk, -L, -l, also from -EP link args)
  • Other Arguments - All other arguments not covered by above. This includes -EP compile arguments.
analysis-<crc>/oovaide-extdirs.txt Defines directories that contain include files in the project directories. Contents define:
  • The initial include paths. These are used during analysis. During compilation, the actual required include paths are used.
analysis-<crc>/*.xmi OovBuilder invokes oovCppParser to build these files. See the oovCppParser section.
analysis-<crc>/*err.txt Lists the errors during the analysis from the OovCppParser.
analysis-<crc>/oovaide-incdeps OovBuilder invokes oovCppParser to build these files. See oovCppParser section.
dups/*.hsh OovBuilder invokes oovCppParser to build these files. See oovCppParser section.


[WARNING: At this time, all external directories are considered non-volatile]

Build

The Build functionality is performed whenever the mode switch is "-mode-build".

Determine Component and Package Dependencies

The include dependencies are used to determine which components are dependent on which packages.  This is done before compilation, because some packages have compiler flags that are used to create the object files.

Build object files

Build object files for all components not marked as Unknown. Determine the include paths and order by examining the include dependencies found during the analysis. Only build files if they are older than the files they are dependent on.

File Content
bld-<config>/*.o Object files

Build library files

Build libraries for all components marked as a static library. Only build files if they are older than the files they are dependent on. If any project libraries were built, make symbols for them. Use the symbols to determine the project sort library order and which libraries should be used to link.

File Content
out-<config>/*.lib Library files
bld-<config>/sym/<libname.txt Symbols that are in the library files. These symbols are found using the library name tool, which is typically "nm".
LibSym-<clump>-Def.txt List the symbols that are defined in a clump.  A clump is all of the symbols of every library in either a package, or the project.
LibSym-<clump>-Undef.txt Lists the symbols that are required for the clump.
LibSym-<clump>-Deps.txt Lists the dependency library order for each library in the clump so that the libraries can be linked in order.

Build executables

Builds executables for all components marked as executables.

For each component, there may be multiple packages that it is dependent on. For each package, libraries are either listed in order, or they were scanned using a directory search. The scanned libraries within each package are ordered in the same manner as building project library files.

File Content
oovaide-tmp-buildpkg.txt The scanned library names are sorted in dependency order.

Only build files if they are older than the files they are dependent on.

File Content
out-<config>/*.exe (or no extension in linux) Executable files
out-<config>/*.dll (or .s in linux)

Tasks performed by OovCppParser

Scan project source

Scan source for includes and drawing info. Compares age of XMI file to source files.

Output file: analysis-<crc>/*.xmi
Output file: analysis-<crc>/*err.txt
Output file: analysis-<crc>/oovaide-incdeps (OovBuilder specifies the directory)

The incdeps file contains each header file, and the header files in includes. It also contains the times that the files were scanned or changed.

Other optimizations