diff options
Diffstat (limited to 'Lessons_uncategorized/Lesson_Build_tools/README.md')
| -rw-r--r-- | Lessons_uncategorized/Lesson_Build_tools/README.md | 365 |
1 files changed, 365 insertions, 0 deletions
diff --git a/Lessons_uncategorized/Lesson_Build_tools/README.md b/Lessons_uncategorized/Lesson_Build_tools/README.md new file mode 100644 index 0000000..5d974a3 --- /dev/null +++ b/Lessons_uncategorized/Lesson_Build_tools/README.md @@ -0,0 +1,365 @@ +In this lesson we are going to look how EDKII configures its build tools. + +As you might remember when you source `edksetup.sh` script for the first time, this script creates `Conf` folder from the contents inside the [https://github.com/tianocore/edk2/tree/master/BaseTools/Conf](https://github.com/tianocore/edk2/tree/master/BaseTools/Conf) directory. + +In this lesson we are going to look at the content of the `Conf/tools_def.txt` file which is created from the [https://github.com/tianocore/edk2/blob/master/BaseTools/Conf/tools_def.template](https://github.com/tianocore/edk2/blob/master/BaseTools/Conf/tools_def.template). + +This file defines configurations for the edk2 build tools. + +Every configuration is representent in a format `TARGET_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE`, for example: +``` +RELEASE_VS2013x86_X64_CC_FLAGS = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw +``` +In this example: +``` +TARGET = RELEASE +TOOLCHAIN = VS2013x86 +ARCH = X64 +COMMANDTYPE = CC +ATTRIBUTE = FLAGS +``` +This means, when the project is build in `RELEASE` mode with a `VS2013x86` toolchain for `X64` architecture use the `FLAGS` flags for the compiler `CC`. + +If we use `*` in place of some part of the format string, it means `for every`. For example: +``` +*_GCC5_X64_NASM_FLAGS = -f elf64 +``` +This means, when the project is build with a `GCC5` toolchain for the `X64` architecture use the `FLAGS` flags for the nasm assembler `NASM`. And this configuration would be active for every build mode (RELEASE/DEBUG/NOOPT). + +There can be many `*` in the configuration string. For example: +``` +*_*_*_BROTLI_PATH = BrotliCompress +``` +This means that the `PATH` attribute for the `BROTLI` command is `BrotliCompress`. And this is true for builds for any arch, toolchain or build mode. + +Another example: +``` +*_CLANG38_*_MAKE_PATH = make +``` +I guess no explanation is needed at this point. I just wanted to point out, that the `*` could be in any part of the format string. + +# `DEF` and `ENV` + +The `tools_def.txt` file has several syntactic constructions to help format strings definition. + +`DEFINE` statement is used to create a named data that can be later expanded via `DEF` statement. This is similar to the C language define statemnt. For example: +``` +DEFINE IASL_OUTFLAGS = -p +... +*_GCC49_*_ASL_OUTFLAGS = DEF(IASL_OUTFLAGS) +``` +There can be chains of defines: +``` +DEFINE GCC_ASM_FLAGS = -c -x assembler -imacros AutoGen.h +DEFINE GCC48_ASM_FLAGS = DEF(GCC_ASM_FLAGS) +DEFINE GCC49_ASM_FLAGS = DEF(GCC48_ASM_FLAGS) +DEFINE GCC5_ASM_FLAGS = DEF(GCC49_ASM_FLAGS) +*_GCC5_X64_ASM_FLAGS = DEF(GCC5_ASM_FLAGS) -m64 +``` + +Another syntactic construction is `ENV`. It is used when the value should be obtained from the environment: +``` +*_GCC5_ARM_CC_PATH = ENV(GCC5_ARM_PREFIX)gcc +``` +This way if we cross compile EDKII for the ARM architecture we can set gcc prefix: +``` +export GCC5_ARM_PREFIX=<...> +``` + +# Precedence rules + +The more defined statements override the more common ones. The `tools_def.txt` has a comment about the override rules: + +``` +TARGET_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE (Highest) +******_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE +TARGET_*********_ARCH_COMMANDTYPE_ATTRIBUTE +******_*********_ARCH_COMMANDTYPE_ATTRIBUTE +TARGET_TOOLCHAIN_****_COMMANDTYPE_ATTRIBUTE +******_TOOLCHAIN_****_COMMANDTYPE_ATTRIBUTE +TARGET_*********_****_COMMANDTYPE_ATTRIBUTE +******_*********_****_COMMANDTYPE_ATTRIBUTE +TARGET_TOOLCHAIN_ARCH_***********_ATTRIBUTE +******_TOOLCHAIN_ARCH_***********_ATTRIBUTE +TARGET_*********_ARCH_***********_ATTRIBUTE +******_*********_ARCH_***********_ATTRIBUTE +TARGET_TOOLCHAIN_****_***********_ATTRIBUTE +******_TOOLCHAIN_****_***********_ATTRIBUTE +TARGET_*********_****_***********_ATTRIBUTE +******_*********_****_***********_ATTRIBUTE (Lowest) +``` + +# Tools and flags review + +Here is some peak on the parameters that currently can be defined for the commands: + +``` +TARGET = + RELEASE | DEBUG | NOOPT +``` + +``` +TOOLCHAIN = + VS2008 | VS2008x86 | VS2010 | VS2010x86 | VS2012 | VS2012x86 | VS2013 | VS2013x86 | VS2015 | VS2015x86 | VS2017 | VS2019 + GCC48 | GCC49 | GCC5 + CLANG35 | CLANG38 | CLANGPDB | CLANGDWARF + XCODE5 + RVCT | RVCTLINUX | RVCTCYGWIN # ARM RealView Tools Windows | ARM RealView Tools Linux | ARM RealView Tools - Cygwin +``` + +``` +ARCH = + IA32 | X64 | EBC | AARCH64 | ARM | RISCV64 +``` + +``` +ATTRIBUTE = + PATH | FLAGS | OUTFLAGS | XIPFLAGS | GUID | DLL | FAMILY | BUILDRULEFAMILY +``` + +And here are descriptions for some of the common commands: +``` +COMMANDTYPE = + APP # C compiler for applications + ASL # ACPI Compiler for generating ACPI tables + ASLCC # ACPI Table C compiler + ASLDLINK # ACPI Table C Dynamic linker + ASLPP # ASL C pre-processor + ASM # A Macro Assembler for assembly code in some libraries + ASMLINK # The Linker to use for assembly code generated by the ASM tool + CC # C compiler for PE32/PE32+/Coff images + DLINK # The C dynamic linker + MAKE # Required for tool chains. This identifies the utility used to process the Makefiles generated by the first phase of the build + PCH # The compiler for generating pre-compiled headers + PP # The C pre-processor command + SLINK # The C static linker + TIANO # This special keyword identifies a compression tool used to generate compression sections as well as the library needed to uncompress an image in the firmware volume + VFR # The VFR file compiler which creates IFR code + VFRPP # The C pre-processor used to process VFR files +``` + +# Setting `TARGET`/`TOOLCHAIN`/`ARCH` for the project + +Configuration strings are represented in the form `TARGET_TOOLCHAIN_ARCH_COMMANDTYPE_ATTRIBUTE`. + +The first three parameters `TARGET`/`TOOLCHAIN`/`ARCH` are usually constant for the build environment, so we've even fixed them in our `Conf/target.txt` file: +``` +... +TARGET = RELEASE +TARGET_ARCH = X64 +TOOL_CHAIN_TAG = GCC5 +... +``` +But as you remember we can always override them from command prompt to build different configurations: +``` +build <...> --arch=X64 --buildtarget=RELEASE --tagname=GCC5 +``` + +# Modify build tools attributes via INF files + +It is possible to modify build tools options in the INF files. + +Let's create new app: +``` +./createNewApp.sh BuildOptionsApp +``` +And add it to our package DSC file `UefiLessonsPkg/UefiLessonsPkg.dsc`: +``` +[Components] + ... + UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp.inf +``` + +Add this code to the `UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp.c`: +```cpp +#include <Library/UefiBootServicesTableLib.h> +#include <Library/UefiLib.h> + +EFI_STATUS +EFIAPI +UefiMain ( + IN EFI_HANDLE ImageHandle, + IN EFI_SYSTEM_TABLE *SystemTable + ) +{ + #ifdef MY_DEBUG + Print(L"MY_DEBUG is definedi\n"); + #endif + #ifdef MY_RELEASE + Print(L"MY_RELEASE is defined\n"); + #endif + #ifdef MY_ALL_TARGETS + Print(L"MY_ALL_TARGETS is defined\n"); + #endif + + return EFI_SUCCESS; +} +``` + +Build our application: +``` +$ build --buildtarget=RELEASE +``` +And copy result to the QEMU shared folder +``` +$ cp Build/UefiLessonsPkg/RELEASE_GCC5/X64/BuildOptionsApp.efi ~/UEFI_disk/ +``` +If you execute our app under UEFI shell now, it wouldn't produce any output as we didn't define any of the defines. + +Now let's add this `[BuildOptions]` section to the application INF file: +``` +[BuildOptions] + RELEASE_GCC5_X64_CC_FLAGS = "-DMY_RELEASE" + DEBUG_GCC5_X64_CC_FLAGS = "-DMY_DEBUG" + *_GCC5_CC_X64_FLAGS = "-DMY_ALL_TARGETS" +``` + +Now build and test our application in the RELEASE mode: +``` +$ build --buildtarget=RELEASE +$ cp Build/UefiLessonsPkg/RELEASE_GCC5/X64/BuildOptionsApp.efi ~/UEFI_disk/ +``` +This version would give you this output: +``` +FS0:\> BuildOptionsApp.efi +MY_RELEASE is defined +MY_ALL_TARGETS is defined +``` +And if you build with the DEBUG mode: +``` +$ build --buildtarget=DEBUG +$ cp Build/UefiLessonsPkg/DEBUG_GCC5/X64/BuildOptionsApp.efi ~/UEFI_disk/ +``` +You would get: +``` +FS0:\> BuildOptionsApp.efi +MY_DEBUG is defined +MY_ALL_TARGETS is defined +``` + +If you look closely to the build log you could even see our defines in the EDK2 build output. For example this is a sample from the RELEASE build: +``` +... +"gcc" -MMD -MF /<...>/Build/UefiLessonsPkg/RELEASE_GCC5/X64/UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp/OUTPUT/AutoGen.obj.deps -g -Os -fshort-wchar -fno-builtin -fno-strict-aliasing -Wall -Werror -Wno-array-bounds -include AutoGen.h -fno-common -ffunction-sections -fdata-sections -DSTRING_ARRAY_NAME=BuildOptionsAppStrings -m64 -fno-stack-protector "-DEFIAPI=__attribute__((ms_abi))" -maccumulate-outgoing-args -mno-red-zone -Wno-address -mcmodel=small -fpie -fno-asynchronous-unwind-tables -Wno-address -flto -DUSING_LTO -Os -Wno-unused-but-set-variable -Wno-unused-const-variable "-DMY_RELEASE" "-DMY_ALL_TARGETS" -c -o /<...>/Build/UefiLessonsPkg/RELEASE_GCC5/X64/UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp/OUTPUT/./AutoGen.obj -I/<...>/UefiLessonsPkg/BuildOptionsApp -I/<...>/Build/UefiLessonsPkg/RELEASE_GCC5/X64/UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp/DEBUG -I/<...>/MdePkg -I/<...>/MdePkg/Include -I/<...>/MdePkg/Test/UnitTest/Include -I/<...>/MdePkg/Include/X64 /<...>/Build/UefiLessonsPkg/RELEASE_GCC5/X64/UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp/DEBUG/AutoGen.c +... +``` + +The `=` sign in the our statements actually means "append". This is why we have both defines in the end. +If you want to completely override some attribute, you should use "==" syntax. EDK2 has many necessary options for the `CC_FLAGS`, so you probably shouldn't override this variable, but you might need this functionality for some other attributes. + +# `FAMILY` attribute + +With the help of a `FAMILY` attribute, it is possible to define rules for several toolchains. For example if we have these strings in the `tools_def.txt` file: +``` +*_GCC48_*_*_FAMILY = GCC +*_GCC49_*_*_FAMILY = GCC +*_GCC5_*_*_FAMILY = GCC +``` + +Then we can add append like this: +``` +[BuildOptions] + GCC:*_*_*_CC_FLAGS = "-DMY_FAMILY" +``` + +And it would be active if our toolchain one of the GCC48/GCC49/GCC5 + +# Command substitution in the tools options + +As we were adding defines to the gcc in our example, I want to point out one interesting usage of this feature. + +Look at the [https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/VirtualRealTimeClockLib/VirtualRealTimeClockLib.inf](https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/VirtualRealTimeClockLib/VirtualRealTimeClockLib.inf) `BuildOptions` section: +``` +[BuildOptions] + GCC:*_*_*_CC_FLAGS = -DBUILD_EPOCH=`date +%s` +``` +Here you can see that it is even possible to use command substitution in the `BuildOptions`. Fun, right? + +In case you don't know this command gives current date as Unix time (which is the number of (non-leap) seconds since 1970-01-01). Try to execute it in you Linux shell: +``` +$ date +%s +1657034765 +``` + +In the code [https://github.com/tianocore/edk2/blob/master/EmbeddedPkg/Library/VirtualRealTimeClockLib/VirtualRealTimeClockLib.c] this define is used like this: +```cpp +EFI_STATUS +EFIAPI +LibGetTime ( + OUT EFI_TIME *Time, + OUT EFI_TIME_CAPABILITIES *Capabilities + ) +{ + UINTN EpochSeconds; + ... + EpochSeconds = BUILD_EPOCH; + ... +} +``` + +# Modify build tools attributes via DSC files + +Besides the INF files it is also possible to modify build tool attributes via the DSC file. + +You can do it globally for all the package modules via the `[BuildOptions]` section like we did in the INF file: +``` +[BuildOptions] + RELEASE_GCC5_X64_CC_FLAGS = "-DMY_RELEASE_DSC" +``` + +Or only for individual modules: +``` +[Components] + ... + + UefiLessonsPkg/BuildOptionsApp/BuildOptionsApp.inf { + <BuildOptions> + RELEASE_GCC5_X64_CC_FLAGS = "-DMY_RELEASE_DSC" + } +``` + +In the first case this would lead to recompilation of all the modules listed in the DSC. And in the second case build system will recompile only one module. + +# Section tag + +In the examples above we've always used `[BuildOptions]` name for our section. But this is just a short form. The most full form for the section is `[BuildOptions.$(arch).CodeBase.ModuleType]`. +Here are: +- `$(arch)` - target architecture. You can use `common` for any architecture or define particular rules just for particalar one like `X64`, `AARCH64`, ... +- `CodeBase` - this is equal to the `EDKII` +- `ModuleType` - with this option we can define options for different classes of modules, e.g. `DXE_RUNTIME_DRIVER`, `SMM_CORE`. This value corresponds to the `MODULE_TYPE` value in the INF files + +So the section `[BuildOptions.X64.EDKII.DXE_RUNTIME_DRIVER]` would change build options for the modules of type `DXE_RUNTIME_DRIVER` if they are compiled for the `X64` arch. + +The `[BuildOptions]` is the shortest form, but it is possible to have all intermidiate forms. For example `[BuildOptions.X64]` section name is allowed. + +Also it is possible to list several names as a section name. Look at this as an example: +``` +[BuildOptions.common.EDKII.DXE_RUNTIME_DRIVER, BuildOptions.common.EDKII.COMBINED_SMM_DXE, BuildOptions.common.EDKII.DXE_SMM_DRIVER, BuildOptions.common.EDKII.SMM_CORE, BuildOptions.Common.EDK.DXE_RUNTIME_DRIVER] +``` +With this you could define the same modifiers for different situations. + +# Override priority + +Generally if we use override `==` syntax, the DSC override would take the precedence over the INF override. But if we use `[BuildOptions.$(arch).CodeBase.ModuleType]` section names, things get a little bit more complicated. + +Here is complete override sequence from the EDKII docs: +``` +HIGHEST PRIORITY +- DSC file's component scoped <BuildOptions> for individual INF files +- [BuildOptions.$(arch).CodeBase.ModuleType] +- [BuildOptions.$(arch).CodeBase] +- [BuildOptions.common.CodeBase] +- [BuildOptions.$(arch)] +- [BuildOptions.common] +- [BuildOptions] +- INF file's [BuildOptions] section +- tools_def.txt +LOWEST PRIORITY +``` + +# Links + +Here are links to the EDKII docs about `BuildOptions` section: +- in the INF file: [https://edk2-docs.gitbook.io/edk-ii-inf-specification/3_edk_ii_inf_file_format/35_-buildoptions-_sections](https://edk2-docs.gitbook.io/edk-ii-inf-specification/3_edk_ii_inf_file_format/35_-buildoptions-_sections) and [https://github.com/tianocore-docs/edk2-InfSpecification/blob/master/2_inf_overview/26_%5Bbuildoptions%5D_section.md](https://github.com/tianocore-docs/edk2-InfSpecification/blob/master/2_inf_overview/26_%5Bbuildoptions%5D_section.md) +- in the DSC file: [https://edk2-docs.gitbook.io/edk-ii-dsc-specification/3_edk_ii_dsc_file_format/36_-buildoptions-_sections](https://edk2-docs.gitbook.io/edk-ii-dsc-specification/3_edk_ii_dsc_file_format/36_-buildoptions-_sections) and [https://github.com/tianocore-docs/edk2-DscSpecification/blob/master/2_dsc_overview/24_%5Bbuildoptions%5D_section.md](https://github.com/tianocore-docs/edk2-DscSpecification/blob/master/2_dsc_overview/24_%5Bbuildoptions%5D_section.md) + |