From 338cbcb0f43d3704ab16e4517bd1c6643efe226e Mon Sep 17 00:00:00 2001
From: Leah Rowe <leah@libreboot.org>
Date: Fri, 20 Oct 2023 15:13:44 +0100
Subject: [PATCH] libreboot build system audit 3 announcement
Signed-off-by: Leah Rowe <leah@libreboot.org>
---
site/news/MANIFEST | 1 +
site/news/audit3.md | 825 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 826 insertions(+)
create mode 100644 site/news/audit3.md
diff --git a/site/news/MANIFEST b/site/news/MANIFEST
index b4f2542..1ae4a15 100644
--- a/site/news/MANIFEST
+++ b/site/news/MANIFEST
@@ -1,3 +1,4 @@
+audit3.md
audit2.md
argon2.md
hp8470p_and_dell_t1650.md
diff --git a/site/news/audit3.md b/site/news/audit3.md
new file mode 100644
index 0000000..8f50a81
--- /dev/null
+++ b/site/news/audit3.md
@@ -0,0 +1,825 @@
+% Libreboot Build System Audit 3
+% Leah Rowe
+% 20 October 2023
+
+Introduction
+============
+
+This article pertains to the current Libreboot (lbmk) revision at this time
+of writing, which is `c7e764a3f088e2dbcacb507995476ce3082451ef` from 20
+October 2023. A *release* is also being compiled today, 20 October 2023, based
+upon that revision, and the announcement for that release
+is: [Libreboot 20231020 release announcement](libreboot20231020.md)
+
+A lot of work has gone into Libreboot since [Libreboot Build System
+Audit 2](audit2.md), the main focus of which has been yet another
+audit; audit2 and the [audit before that](audit.md) was mild by comparison.
+
+It was previously planned that audit2 would come just before a release, but I
+decided that I would perform *another* audit immediately, so there have been
+two audits (this audit, dubbed audit 3, and the previous audit 2) between now
+and the Libreboot 20230625 release.
+
+Massive code size reduction
+---------------------------
+
+The focus has been improved error handling, general bug fixing and improving
+the efficiency of lbmk (in terms of speed). Reduced complexity. The focus has
+therefore been on the *build system*, lbmk, which you can read about in
+the [lbmk maintenance manual](../docs/maintain/).
+
+*This* audit, today, has resulted in a substantial code size decrease in the
+Libreboot build system, now totalling *1744 source lines* when counting shell
+scripts; after audit2, the sloccount was 2644 sloc and in Libreboot 20230625 it
+was 3388.
+
+That's a *34 percent* reduction versus audit2, and 48.5 percent versus 20230625.
+That is *without* reducing any functionality, in any way. Many *new* features
+have been implemented.
+
+Summarised changes since audit2
+-------------------------------
+
+NOTE: In this summarised list, only those changes that pertain to the *audit*
+have been listed, because I'm writing a release announcement simultaneously.
+Therefore, *all* changes (including audit-related) will be in the release
+announcement, but only audit changes will be listed below. *Not all* changes
+are listed, in the summarised list below, but a full list is written later on
+in this article.
+
+To re-iterate: only audit-related changes will be listed, and even then, not all
+changes are listed below; a full list of actual commits is printed, further down.
+
+Overall changes (summary):
+
+* Better integrity checking when downloading vendor files
+* Safety check: don't allow running as root when not needed, on most commands.
+* Multi-threading now used more universally, when compiling source code, and
+ when creating tar archives from the Libreboot build system.
+* Safer GRUB configuration file handling; less likely to brick during
+ configuration changes that are subject to re-flashing.
+* *Much* stricter, more verbose error handling in general, but some overly
+ verbose or redundant error handling was deferred to using `x_` instead (more
+ on this later).
+* Vastly reduced number of modules in GRUB, keeping only what is required.
+ Many redundant modules have been removed, resulting in lower payload size
+ and higher boot speed when using the GRUB payload.
+* Much safer handling of git name/email credentials, at build time.
+* Greatly improved speed during build, on certain parts of the build process.
+* Improved `/tmp` handling; tmpfiles are now robustly cleaned, on exit from lbmk.
+* All main project revisions have been updated to recent code; coreboot, SeaBIOS,
+ U-Boot and GRUB. This, in addition to several other repositories.
+* GitHub is no longer used on main Git repository links, instead only as backup
+* Backup repositories now defined, for all main repos under `config/git/`
+* GRUB LUKS2 support now available, with argon2 key derivation; previously, only
+ PBKDF2 worked so most LUKS2 setups were unbootable in Libreboot. This is fixed.
+* 12 shell scripts, not 20, in lbmk. This, in addition to sloccount reduction.
+ NO FUCTIONALITY HAS BEEN LOST!
+* Most logic has been greatly unified, into single scripts that perform the
+ same style of task, rather than the same style of task being handled by
+ multiple scripts. This is done without creating *spaghetti*.
+* Many bugs fixed, and sloccount reduced. Too many to summarise, so see below.
+* Dependencies configs (package lists for distros) have been updated, and new
+ distros are supported e.g. popos and linuxmint.
+* Generally improved help text, in lbmk commands that provide help text.
+* The main Makefile has been removed, because it was redundant.
+
+Here's a more full list of changes, expanding on the above; some entries above
+are also repeated below but in more detail:
+
+* Scrubbing of vendor files *now* handled by the inject script, rather than
+ the release script. This enables more robust handling of configs pertaining
+ to vendor files, that tell lbmk where the files are and how to insert them; it
+ therefore follows that this same script should be used to delete them.
+* Use `--mtime` and option options in GNU Tar (if it is actually GNU Tar), when
+ creating Tar archives. This results in partially reproducible source archives,
+ and consistent hashes were seen in testing, but not between distros.
+* Always re-inialitise `.git` within lbmk, for the build system itself, if
+ Git history was removed as in releases. This work around some build systems
+ like coreboot that use Git extensively, and are error-prone without it.
+* More robust makefile handling in source trees; if one doesn't exist, error
+ out but also check other makefile name combinations, and only error out if
+ the command was to actually build.
+* ROMs build script: support the "all" argument, even when getopt options are
+ used e.g. `-k`
+* Disabled the pager in `grub.cfg`, because it causes trouble in some
+ non-interactive setups where the user sees an errant message on the screen
+ and has to press enter. This fixes boot interruptions in some cases, allowing
+ normal use of the machine. The pager was initially enabled many years ago,
+ to make use of cat a bit easier in the GRUB shell, but the user can just
+ enable the pager themselves if they really want to.
+* U-Boot can now be compiled standalone, without using the ROMs build script,
+ because crossgcc handling is provided for U-Boot now in addition to coreboot.
+* Unified handling of git/vendor config files, containing URLs, revisions,
+ checksums and so on. This is handled by a single function
+ under `include/option.sh`
+* All helper scripts are now under `include/`, and main scripts in `script/`,
+ called by the main `build` script
+* Intel ME extraction is now provided in one function, instead of two, when
+ downloading vendor files per mainboard, before running it
+ through `me_cleaner`
+* Unified checking of the destination file, when downloading vendor updates.
+ This results in more reliable checking of whether a vendor file has already
+ been downloaded or not, where it is only handled if missing.
+* Generally purge unused variables in shell scripts
+* Simplified initialisation of variables in shell scripts, using the `setvars`
+ function defined under `include/err.sh`
+* Vendor scripts: archive extraction is now unified, the same method used for
+ each archive. This enables more robust checking of hashes and so on.
+* Support patch subdirectories, when applying patches. This is done recursively,
+ making it possible to split up patch files into smaller sets inside sub
+ directories, per each source tree (or target of each source tree, where a
+ project is multi-tree within lbmk)
+* SPDX license headers now used, almost universally, in all parts of lbmk.
+* Files such as those under `config/git` or `config/vendor` are now
+ concatenated, traversing recursively through the target directory; files first,
+ then directories in order, and for each directory, follow the same pattern
+ until all files are concatenated. This same logic is also used for patches.
+ This now enables use of subdirectories, in some config/patch directories.
+* General code cleanup on `util/nvmutil`
+* Git histories are more thoroughly deleted, in third party source trees during
+ release time.
+* Symlinks in release archives are no longer hard copies; the symlinks are
+ re-created by the release script, because it clones the current lbmk work
+ directory via Git (local git clone), rather than just using `cp` to copy links.
+* More deeply integrated the Intel MRC download script (from coreboot) into
+ Libreboot's vendor scripts, removing its download logic and re-using that
+ from Libreboot's scripts instead; now, the MRC script only contains extraction
+ logic, and it is an *include* file, rather than a standalone script.
+* Properly output to stderr, on printf commands in scripts where it is either
+ a warning prior to calling `err`, or just something that belongs on the error
+ output (instead of standard output).
+* Don't use the `-B` option in make commands.
+* Where no-microcode ROM images are provided, ensure that the ROM hashes still
+ match when running the vendor inject script. This is only useful on the
+ Dell Latitude E6400, which is otherwise FSDG-compatible but (in Libreboot)
+ comes with or without microcode updates, and with or without the Nvidia VGA
+ ROM (handled by vendor inject/download scripts) for dGPU variants. Verification
+ previously failed, under certain conditions, when inserting that VGA ROM.
+* SECURITY: Use sha512sum (not sha1sum) when verifying certain downloads. This
+ reduces the chance for collisions, during checksum verification.
+* Set GRUB timout to 5s by default, but allow override and set to 10s or 15s
+ on some mainboards.
+* Vendor scripts: don't use `/tmp` for ROM images when inserting vendor files.
+ In case `/tmp` is a tmpfs and not much RAM is available, it is paramount that
+ the user's file system is used instead, where there is likely greater capacity;
+ it is done under `tmp/` in lbmk (not to be confused with `/tmp`).
+* Support both curl and wget, where files are downloaded outside of Git; defer
+ to Wget when Curl fails, and try each program three times before failing. This
+ results in more resilient downloading, on wobbly internet connections.
+* Don't clone Git repositories into `/tmp`, because it might be a tmpfs with
+ little memory available; clone into `tmp/gitclone` instead, within lbmk,
+ and `mv` it to avoid unnecessary additional writes (`mv` is much more efficient
+ than `cp`, for this purpose).
+* Removed unused `target.cfg` handling in vendor scripts, because they use
+ the concatenated config format instead (they always have).
+* move `me7_updater_parser.py` to `util/` (not under `script/`)
+* The directory containing vendor files no longer exists in lbmk, because it
+ is instead created when needed; the ifd/gbe files were moved to `config/ifd`
+ so the vendorfile directory became redundant.
+* Coreboot builds: automatically run make-oldconfig, to mitigate use of raw
+ coreboot config where a revision was updated but the config was untouched.
+ This may still result in a confirmation dialog, and it's still recommended
+ that the configs be updated per revision (or switch them to defconfigs).
+* Vastly simplified directory structure; `resources/scripts/` is now `script/`,
+ and `resources/` was renamed to `config/`; ifd and gbe files were also moved
+ to `config/ifd/`. Commands are now 1-argument instead of 2, for example
+ the `./build boot roms` command is now `./build roms`.
+* memtest86plus: only build it on 64-bit hosts, for now (32-bit building is
+ broken on a lot of distros nowadays, and lbmk doesn't properly handle cross
+ compilation except on coreboot or U-Boot)
+* (courtesy of Riku Viitanen) don't use cat on loops that handle lines of text.
+ Instead, use the `read` command that is built into `sh`, reading each line.
+ This is more efficient, and provides more robust handling on lines with
+ spaces in them.
+* Don't support removal of microcode (during release time) on untested targets.
+ Set `microcode_required="y"` on most boards, but leave it set to `"n"` on
+ platfroms such as GM45 (ThinkPad X200/T400, Dell E6400, etc); anything FSDG
+ compatible, in other words.
+* Improved Dell Latitude E6400 support; the same image now provides iGPU and
+ dGPU support, since it's SeaBIOS-only anyway, so a VGA ROM is inserted into
+ the same ROM that also enables libgfxinit, enabling the Intel or Nvidia GPU
+ to be used (if the VGA ROM is missing, only the Intel GPU will work).
+* *ALL* projects now have submodules downloaded at build time, not just multi
+ tree projects such as coreboot.
+* Improved cbutils handling; it's now even less likely to needlessly re-build
+ if it was already built.
+* The release build script no longer archives what was already built, but
+ instead builds from scratch, creating an archive from source downloads
+ first before building the ROM archives. This saves time because it enables
+ a single build test per release, whereas at was previously necessary to test
+ the Git repository and then the release archive. Testing both is still desired,
+ but this behaviour also means that whatever is built at release time is
+ guaranteed to be the same as what the user would build (from archives).
+* Improved handling of `target.cfg` files in multi-tree projects coreboot,
+ SeaBIOS and U-Boot. Unified to all such projects, under one script, and
+ with improved error handling.
+* Only remove microcode (where that behaviour is enabled per board) in release
+ ROMs, but not during build time. This results in reduced disk usage during
+ development, but release archives still contain the no-microcode option if
+ you want to use that; manual removal is also still possible, during development.
+* GRUB payload: all ROM images now contain the same ELF, with all keymaps
+ inserted. This speeds up the build process, and enables easier configuration
+ when changing the keyboard layout because less re-flashing is needed.
+* Simplified IFD handling on ICH9M platforms (e.g. X200/T400 thinkpads); the
+ ich9gen utility wasn't needed anymore so ich9utils has been removed, and now
+ the IFD/GbE files are included pre-assembled (generated by ich9gen). Ich9gen
+ can still be used, or you can re-generate with coreboot's bincfg; the ifdtool
+ util can be used to edit IFD and nvmutil (part of Libreboot) can change MAC
+ addresses. The ich9utils code was always redundant for the last few years,
+ especially since 2022 when nvmutil was first written.
+* Running as root is now forbidden, for most commands; lbmk will exit with
+ non-zero status if you try. The `./build dependencies x` commands still work
+ as root (they're the only commands available as root).
+* Enabled memtest86plus on more boards, where it wasn't previously enabled.
+* Only enable SeaBIOS as first payload on desktops, but still enable GRUB as
+ second payload where GRUB is known to work (on each given host). The text
+ mode and coreboot framebuffer modes are provided in each case, where feasible.
+* The `list` command has been mostly unified, making it easier to tell (from
+ lbmk) what commands are available, without having to manually poke around
+ under `script/`.
+* The `-T0` flag is now used, universally, on xz commands. This makes `xz` run
+ on multiple threads, greatly speeding up the creation of large tar archives.
+* Universally use `-j` in make commands, for multi-threading, but it relies
+ on `nproc` to get thread count, so this only works if you have `nproc` (you
+ probably don't, if you run BSD; BSD porting is still on TODO for Libreboot)
+* File names as arguments now universally have quotes wrapped around them, and
+ similar auditing has been done to all variables used as arguments everywhere
+ in lbmk. There were cases where multiple arguments were wrongly quoted then
+ treated as a single argument, and vice versa. This is now fixed.
+* Re-wrote `.gitcheck`; now, a global git name/email config is always required.
+ The only behaviour (setting local config, and unsetting) was quite error-prone
+ under fault conditions, where cleanup may not have been provided, or when
+ execution was interrupted, resulting sometimes in accidentally committing
+ to `lbmk.git` as author named `lbmkplaceholder`.
+* The new BSD-like coding style is now used on *all* shell scripts in lbmk. A
+ few scripts still used the old lbmk coding style, as of audit 2.
+* Scripts no longer directly exit with non-zero status, under fault conditions;
+ instead, `x_` or `err` is used to provide such behaviour. This results in all
+ exits from lbmk being consolidated to `err`, under fault conditions. - zero
+ exits are also consolidated, going only through the main script, which has its
+ own exit function called `lbmk_exit` that provides `TMPDIR` cleanup.
+* *Copy* `dl_path`, don't move it, when downloading and extracting a vendor
+ file. This reduces the change of it being missing later when lbmk is run again.
+* BSD-style error handling implemented, with an `err` function (and functions
+ that use it) inside `include/err.sh`; there is also `x_` which can be used
+ to run a command and exit automatically with non-zero status, useful because
+ it provides more verbose output than if you just relied on `set -e`, and it
+ still works when a script *does not* use `set -e` - however, it is not used
+ on all functions, because it works by executing `$@` directly, which can break
+ depending on arguments. Therefore, some scripts just default to `|| err` for
+ providing breakage in scripts.
+* Memtest *6.2* now used (instead of *5.x* releases). This is essentially a
+ re-write, and it works on the coreboot framebuffer, whereas previous revisions
+ only worked on text mode setups.
+* NO MAKEFILE. The Makefile in lbmk has been removed. It was never meaningfully
+ used because all it did was run lbmk commands, without implementing any logic
+ itself. A Makefile may be added again in the future, but with a view to
+ installing *just the build system* onto the host system, to then build ROM
+ images under any number of directories. Lbmk's design is strictly no-Makefile,
+ but it uses Makefiles provided by third party source trees when building them.
+* Safer GRUB configuration file handling between GRUB memdisk and coreboot CBFS;
+ it is no longer possible to boot without a GRUB config, because the one in
+ GRUB memdisk is provided as a failsafe, overridden by *inserting* one in CBFS,
+ but there is no config in CBFS by default anymore.
+* VASTLY more efficient build process; all coreboot ROMs without payload are
+ now cached under `elf/`, as are payloads, then they are joined separately by
+ the usual ROMs build script, and these cached ROMs contain many changes in
+ them that were previously handled by `moverom` in the main ROM build script.
+ Under the new design, repetitive steps are avoided; payloads are inserted into
+ a copy of the cached ROMs under `TMPDIR`, *before* being copied for keymaps
+ and small files; this eliminates delays caused by slow compression (LZMA is
+ always used, when inserting payloads). After crossgcc and the payloads are
+ compiled, the ROM with coreboot builds in under a minute, whereas it would
+ have previously taken several minutes on most Libreboot-supported hardware.
+* VASTLY reduced GRUB payload size; modules that aren't needed have been removed
+ resulting in much smaller GRUB payloads, that also boot faster.
+* ALL defconfig creation, updating and modification are handled by the same
+ script that *also* handles compiling, as mentioned in the bullet-point below.
+* ALL main source trees are now compiled, downloaded, configured and cleaned
+ using the same script. The *download* (Git) logic is a separate file
+ under `include/` and its functions are called by the main build script, which
+ provides a stub for this.
+* Scripts are no longer executed directly, ever, except the main script. All
+ scripts are otherwise executed from `script/`, inheriting the `TMPDIR`
+ variable set (and exported) by lbmk.
+* Generally improved user feedback in scripts, especially the vendor scripts.
+* Coreboot, U-Boot and SeaBIOS are now downloaded, configured and compiled using
+ the exact same script. Although these codebases differ wildly, their build
+ systems use the same design, and they are compatible from a user-interface
+ perspective.
+* Vastly improved `/tmp` handling; a universal `TMPDIR` is set (environmental
+ variable) and exported to all child processes running lbmk scripts. On exit,
+ the main tmp directory is purged, cleaning all tmp directories under it.
+* Improved handling of vendor file hashes; previously, the backup would only
+ be tried if the first one failed to download, but if the first file succeeded
+ and yet had a bad hash, the backup would not be tried. Now the backup is tried
+ when either the first download fails OR it has a bad hash, making downloads
+ of vendor files more resilient to network failure.
+* When extracting ME files from vendors, more types of archives are supported
+ for decompression at build time.
+* Fixed bug where vendor files were always being downloaded from backup URLs
+ at build time.
+* Spoof the user agent string mimicking that of Tor Browser, when downloading
+ vendor files at build time. This circumvents restrictions based on user agent
+ string, when lbmk interacts with certain HTTP servers.
+* General simplification of coding style on all shell scripts.
+* Abort (with non-zero exit) if KBC1126 EC firmware fails to download at build
+ time.
+* Fixed some variable initialisations in the coreboot ROM image build script
+* Don't enable u-boot on QEMU x86 images (due to buggy builds, untested)
+* Haswell (libre MRC) coreboot tree: fixed acpica downloads, which no longer
+ work on the upstream URL. Old acpica binaries now hosted on Libreboot rsync.
+* Fixed coreboot-version file inserted into coreboot trees, when compiled
+ on Libreboot release archives.
+
+FULL list of changes (from lbmk git log)
+---------------------------------------
+
+The list above was written based on reading of the Git log, which was taken by
+running the command under lbmk:
+
+ git log --graph --pretty=format:'%Cred%h%Creset %s %Creset' --abbrev-commit 20230625..c7e764a3f088e2dbcacb507995476ce3082451ef
+
+The commits are, thus:
+
+```
+* c7e764a3 update/release: confirm vdir path on exit
+* 1c8b2114 update/release: copy crossgcc to archive
+* 54a05fc1 always re-generate .git in lbmk
+* 52c9416b update flashrom revision
+* af1c1e10 add backup git repo for flashrom
+* 18364822 Revert "config/git: don't download flashrom"
+* ac442808 config/git: add more backup repos
+* 75980052 git/config: don't use github on main repos
+* bf4ea810 config/git: don't download flashrom
+* da3044e7 git/config stm32-vserprog: don't fetch libopencm3
+* 782371a5 update/release: delete *all* .git and .gitmodules
+* 743a425c include/git: fix already-exists download message
+* 73145b79 Revert "Revert "include/git: don't re-download single-trees""
+* 31b35bb4 include/git: fix error caused by sh idiosyncrasy
+* baa3d4f2 Revert "include/git: don't re-download single-trees"
+* 8de7bc93 include/git: don't re-download single-trees
+* d1f23eca config/git: remove rpi-pico-tinyusb dependency
+* 97e5207e config/git: give pico-sdk its own file
+* 182ee8e4 update/trees: don't run make if mode=fetch
+* 54eb347a include/git: fetch submodules on one-tree projects
+* f855611c include/git: only download submodules if possible
+* 0c32c1d6 update/release .git/*: delete one more level up
+* 0375cfaf update/release: don't hardcode project names
+* d245e0b1 consistent naming for src/pico-serprog
+* fac62a8c config/git: name files per download name
+* 0e1602f5 do a nice thing
+* 7b206008 Merge pull request 'fix_distro_dependencies - part 2' (#139) from andreamtp/lbmk:fix_distro_dependencies into master
+|\
+| * a16cd1a3 Added python-unversioned-command for Fedora38
+| * 8a063f6b Fix Debian/Ubuntu dependencies
+* | 6af65ad4 error handling code cleanup and fixes
+* | 4e54a051 another code cleanup
+* | 8d9aeef3 lbmk: use 2-level directory structure in script/
+* | 0b98c9b0 minor code cleanup in shell scripts
+* | 8b6e44a1 Merge pull request 'Fix F38/Ubuntu 20.04 dependencies' (#137) from andreamtp/lbmk:fix_distro_dependencies into master
+|\|
+| * 6758b5c8 Fix F38/Ubuntu 20.04 dependencies
+* | 9fac3c12 Merge pull request 'Fix Void Dependencies for building Serprog' (#138) from neutrocyte/lbmk:fix_void_dependencies into master
+|\ \
+| |/
+|/|
+| * e63399cf Fixed Void Dependencies for building Serprog
+|/
+* 4cdf60e6 util/spkmodem-recv: detailed copyright history
+* fc2cab31 update/release: fix missing variable definition
+* c14461a5 delete include/vendor.sh and merge elsewhere
+* d8c2c245 vendor.sh: move some functions to vendor/download
+* 0f807762 update .gitignore for the dell-flash-unlock binary
+* 34b8687e coreboot/fam15h: remove redundant patch
+* 1a299f1b Merge pull request 'util/e6400-flash-unlock: Rename to dell-flash-unlock' (#135) from nic3-14159/lbmk:rename-e6400-flash-unlock into master
+|\
+| * 5d6946c4 util/e6400-flash-unlock: Rename to dell-flash-unlock
+* | 8583a05d Merge pull request 'Update U-Boot to v2023.10 and use default coreboot tree for gru chromebooks' (#136) from alpernebbi/lbmk:uboot-v2023.10 into master
+|\ \
+| * | 4d9567a7 coreboot: gru: Use default coreboot tree
+| * | 6e65595d u-boot: gru: Do not persist EFI variables
+| * | 4e7e4761 u-boot: gru: Enable more EFI commands
+| * | f08102a2 u-boot: gru: Enable more bootstd features
+| * | fea0cec2 u-boot: gru: Do not reset on panic
+| * | f9bad444 u-boot: gru: Enable poweroff command
+| * | f7db91c8 u-boot: gru: Disable VIDEO_COPY
+| * | 7afe2f39 u-boot: Set EFI variable buffer size to upstream value
+| * | 46e01c0e u-boot: Avoid building U-Boot-only binman images
+| * | 5b4ced33 u-boot: Add patch to avoid regulator errors
+| * | f459e05e u-boot: Update to v2023.10
+| * | b2d84213 update/project/trees: Add flags for more kconfig actions
+| * | 8b411963 u-boot: qemu_arm64_12mb: Remove misleading rev field
+| * | eb267733 build/fw/coreboot: Fix misuse of raw u-boot.bin as payload
+|/ /
+* | 65af756f x/xx: slightly more verbose error messages
+* | 19f1e008 vendor/inject: only build nvmutil if required
+* | 3f8636ff vendor/inject: simplified file handling
+* | 7b741dd0 update/release: remove unused variables
+* | e0feda63 update/release: fix/simplify mtime handling
+* | ec0b38af update/release: nuke roms using the inject script
+* | 2ebadb7f build/release: don't include tmp/ in src tarball
+* | 27aaae59 update/release: also set timestamp on srcdir
+* | ca78fc67 update/release: be more thorough updating times
+* | 7cd84aec update/release: use getops OPTARG correctly
+* | b5db0480 update/release: delete multi-tree upstream repos
+* | 6846c9f7 update/release: if *GNU* tar, use --mtime
+* | c401efdd build/release: support skipping rom builds
+* | 268fd6ce update/release: make src tarball first, then roms
+* | 653a8571 put space in the warning message about elf/
+* | c44a38ae only build cbutils if required
+|/
+* 42068f7c coreboot/default bump: rev d862695f5f, 12 Oct 2023
+* 09881212 use me_cleaner from coreboot instead of upstream
+* 1f331642 nvmutil: simplify endianness handling
+* 3162d60d nvmutil: don't reset errno before write
+* f989360e nvmutil: reset errno on successful write
+* 3ad171fd nvmutil: simplify prototype declarations
+* 96fd88c5 build: fix bad command in help text
+* 5b8b55f2 build/fw/coreboot: fix bad commands in help text
+* 067a358d fix warning about coreboot elf/ vs bin/
+* 13c58200 Merge pull request 'util/e6400-flash-unlock: Update to upstream version' (#134) from nic3-14159/lbmk:e6400-flash-unlock-updates into master
+|\
+| * 724cb39f util/e6400-flash-unlock: Update to upstream version
+* | 67ffb513 build/fw/coreboot: warning about bin/ versus elf/
+|/
+* 634aac0b config/dependencies: fix unifont on arch/parabola
+* 7e3a031a include/err.sh: don't run check_git
+* b61e3feb config/dependencies/ubuntu: symlink to debian
+* 4ea9b9fb config/dependencies: add popos config
+* f8528d12 config/dependencies/debian: add autopoint
+* 21db72b6 disable 32-bit memtest86plus, only build 64-bit
+* d1ba94ea update/release/*: merge to update/project/release
+* e7a77b50 build/fw/coreboot: reset grub background each time
+* 92abbb25 update/release/roms: copy license files to archive
+* 85bee1f8 bump grub revision
+* d58bc5ff bump seabios revision
+* 1e89264c update/project/*: merge to update/project/trees
+* a413c01a update/project/trees: handle seen in fetch_config
+* c8bace0d build/fw/grub: re-add end confirmation message
+* ba324d8c build/coreboot/grub: move to build/fw/grub
+* 4708da2c use quotes when checking empty strings in scripts
+* 0fad3497 build/fw/coreboot: fix error "unexpected operator"
+* ea27c928 update/project/build: move helpers to option.sh
+* 0ed2ec29 build/coreboot/util: merge to update/project/build
+* b6d9e6c1 build/fw/coreboot: don't support no-all all arg
+* 0962600c build/fw/coreboot: correctly check built targets
+* fa8e204f unified projectname/version/versiondate handling
+* 24584296 put include/export.sh in build script
+* 62cc895c rename blob/ to vendor/
+* 3c7e37b1 update/blobs: correct utils paths check
+* 5e81024e update/blobs: don't hardcode kbc1126 util check
+* 9f8f230b update/blobs: don't needlessly re-build uefitool
+* fe502da9 Rename blobs/ to blob/
+* 4e39d5a5 put all src downloads under src/
+* 965b6a7e rename build/firmware/ to build/fw/
+* 5494ffb3 build/firmware/coreboot: confirm compiled roms
+* ce10c1b3 build/firmware/coreboot: support "all" without all
+* 2d483d2f move build/release/* to update/release
+* 315d0c45 mv build/fw/serprog,build/boot/roms build/firmware
+* 863081c3 remove build symlink, rename lbmk to build
+* 2d16e1ee rename build/project/trees to update/project/build
+* 1c2de7f9 unify build/grub/* to build/coreboot/grub
+* 176722a8 unify handle/make/* into build/project/trees
+* 9d419e77 handle/make/*: unified main() function
+* 10684102 general code cleanup in shell scripts
+* cad7648a build/boot/*: merge all logic into one script
+* 923a96c1 check git/version: properly call err()
+* 1223bfae check_git: call fail() first (fallback to err)
+* 727dc7ff more verbosely print git config error
+* fbd464b4 include/err.sh: checkgit,checkversion
+* e638c3e4 update/project/trees: remove errant assignments
+* 68e1787c update/project/trees: split up main()
+* 5de8eda2 general code cleanup in shell scripts
+* 334aa1f7 handle/make/config: fix formatting on variables
+* 8097baa0 handle/make/file: check for all default makefiles
+* 0db6c0a4 update/blobs/download: remove errant comment
+* 3af63fb8 handle/make/file: exit 0 if no makefile
+* ad74b4c2 handle/make/file: run extra arg before, not after
+* 2e60e117 grub.cfg: disable the pager
+* d9719cae handle/make/file: do multiple project arguments
+* cb29c96c lbmk: simplify/correct exit commands / cleanup
+* 9dce8236 update/project/trees: fix error handling on mkdir
+* 0f86a393 update/project/trees: optimise error handling
+* 67ac799d update/project/trees: simplified error handling
+* d38b958d include/err x_(): more verbose error message
+* 8886f995 include/err: remove unused variable
+* cd2caecb update/project/trees: general code cleanup
+* bcbd3734 update/project/trees: rm yet another rm line
+* 0a63dce3 update/project/trees: remove one more rm line
+* 91c0f942 update/project/trees: remove redundant rm command
+* 7bead4f5 update/project/trees: remove unnecessary linebreak
+* 1dd97470 update/project/trees: rm "seen" in the right place
+* a3b3196d build/grub/payload: remove unnecessary linebreaks
+* 3fcad603 build/coreboot/utils: remove unnecessary check
+* 0a711ebc build/coreboot/utils: simplify argument handling
+* 7ce3f93e build/boot/*: unify more logic in main()
+* 7b02bb9a do not handle errors on mktemp in shell scripts
+* 8c03b886 Greatly simplify error handling in shell scripts
+* 5f914a4d build/boot/roms: optimise main() for code size
+* 92c6da7b build/boot/roms_helper: shorten variable names
+* 2a6fcf70 build/boot/roms: dont do init/displaymode argument
+* 42d4fa9b include/boot.sh: simplify variable initialisation
+* 9bc9dddf build/boot/roms_helper: simplify rom file handling
+* c477599c build/boot/roms_helper: general code cleanup
+* 26fc3f13 general code formatting cleanup in shell scripts
+* 0a0defd3 simplify initialising variables in shell scripts
+* 49b266eb build/boot/roms: only do 1 custom kbd/payload/mode
+* d268f5eb build/boot/roms: move usage() to include/boot.sh
+* 7922b6e0 build/boot/*: unified main() function
+* f3c4f208 build/boot/roms: split up handle_targets()
+* 4afa0aaa build/boot/roms: check all targets before building
+* 6125d341 build/boot/roms: merge handle_targets/build_target
+* 13f5a432 build/boot/roms: only run confirm_targets once
+* 5462bf1c build/boot/roms: rename buildrom to build_target
+* fc097b3e build/boot/roms: split up main()
+* 895073d7 build/boot/roms: simplify buildrom() handling
+* df7305a5 build/boot/roms: support "all" if argument passed
+* b3e69cd9 build/boot/roms: move help() to bottom of file
+* 385eb90c update/*/*: unified scanning of revisions/sources
+* 9f5a5450 blobs/download: move helpers to include/blobutil
+* 416704fb include/blobutil: try curl first, then wget
+* 6519cea9 include/blobutil: simplify check_defconfig()
+* ac05e5ff blobs/download: do IntelME extract in one function
+* 9b94df5d blobs/download: do final check of _dest in fetch()
+* 9a7bf4af blobs/download: don't pass dl_path as argument
+* 2b7ae8e2 blob scripts: unified handling of blob destination
+* 8ea62a16 remove unused variables in blob scripts
+* 32da4e31 merge include/fetch.sh, blobutil.sh, defconfig.sh
+* 710171f9 update/blobs/*: simplify mrc.bin handling
+* 0bb3c596 update/blobs/*: unified download/checksum logic
+* 5d934be7 blobs/download: remove unnecessary linebreaks
+* 3256ef3e blobs/download: remove unnecessary messages
+* 178b888a include/blobutil: properly set global variables
+* e9e1a3b4 blobs/download: simplify downloading of files
+* 781d0a80 blobs/download: remove unnecessary error pipes
+* 9aef57df blobs/download: unified archive extraction
+* 74c48a88 move build/command/options to include/option.sh
+* a00b4337 build/release/roms: simplify strip_rom_image()
+* 3b9442f7 blobs/download: unified blobdir handling
+* 373c84e4 blobs/download: unified archive extraction
+* 1e92abb1 blobs/download: remove errant debug line
+* e73306ba remove script/update/blobs/extract
+* 16235cb6 blobs/download: simplify fetch_update()
+* d023327f blobs/download: greatly simplify sources handling
+* 65a32698 include/blobutil: simplify setting empty strings
+* 6b17cda1 blobs/download: simplify defconfig handling
+* b5628131 handle/make/config: check project in main()
+* f052f61f handle/make/config: split up main()
+* 67f4919f simplify getopts loops in shell scripts
+* 36b7f01a only update git submodules in project/trees
+* 81d073d5 update/project/*: unified git reset handling
+* eae173ec split up grub patches into subdirectories
+* a823bab3 include/git: support applying patch subdirectories
+* 3738ec90 update/project/*: unified patch handling
+* cd3225d8 update/project/trees: remove extra.sh handling
+* 42c9d7d2 build/grub/*: move common strings to variables
+* e94ba1f7 build/grub/payload: split up main()
+* b727f966 util/: use SPDX license and copyright headers
+* 20862019 Update email address for Leah Rowe copyrights
+* cc164209 Use SPDX license headers on all scripts
+* a7b767a4 update/repos: concatenate multiple revision files
+* 7966f911 handle/make/config: run fail() on error, not err()
+* 2d0e978c update grub revision
+* 905f3d8e util/nvmutil: remove xorswap() macro
+* 231015ef util/nvmutil: make setWord a macro
+* d9bed115 util/nvmutil: further optimise swap command
+* 5e801360 util/nvmutil: use correct comparisons on pointers
+* 137a548b util/nvmutil: optimise swap command
+* 4d448201 util/nvmutil: don't use err_if on argc check
+* 0897a0be util/nvmutil: always print filename in err_if
+* 9a92524a util/nvmutil: remove SIZE_8KB define
+* 5a129cea util/nvmutil: remove xpread/xpwrite macros
+* ac0e4999 util/nvmutil: remove unnecessary xclose macro
+* 83e6cfb2 util/nvmutil: simplify pledge and unveil handling
+* 7bb92acd Merge pull request 'merge serprog scripts' (#131) from Riku_V/lbmk:master into master
+|\
+| * 3c30e1e3 merge serprog scripts
+|/
+* f8704c0a lbmk: more verbose error messages
+* a1db59a5 lbmk: reduce indentation in execute_command()
+* a9ea277e lbmk: fail if ./build command options fails
+* f1f5b91a lbmk: simplify execute_command()
+* 662b9266 lbmk: remove "./buildpath mode all"
+* 4c734308 lbmk: break up main()
+* 5f197023 lbmk: always use lbmk_exit for exits
+* 3400e5a1 rel/src: fix multi-line command
+* 4df3d09b remove ich9utils entries from .gitignore
+* 20bf3a19 Merge pull request 'make clean stm32-vserprog for release' (#130) from Riku_V/lbmk:makeclean into master
+|\
+| * c3ac62b1 serprog: list available boards
+| * 24185bca fix typo serprog -> vserprog
+| * ccb36aa6 make libopencm3 correctly
+| * 5737abf0 make clean libopencm3
+| * 0bed0c35 Download libopencm3 before building
+| * 3d77b8a0 download and copy serprog related src
+| * 7dc86325 clean up pico-serprog for release
+| * 34d3629e make clean stm32-vserprog for release
+* | c400916e coreboot/hp8200sff_4mb: fix bad ifd path in config
+|/
+* 087f0e06 make lbmk help text actually vaguely helpful
+* 093d40ee build/release/src: be more thorough deleting .git
+* 630a6546 build/release/src: delete elf/ in srcdir
+* 0543350d handle/make/file: run make-clean first
+* 12f9afe6 build/release/src: remove cbutils/ in srcdir
+* fe00ab4e build/release/src: remove errant code
+* be4ed540 handle/make/config: distclean once per tree
+* f227cc08 handle/make/config: fix distclean/crossgcc-clean
+* 669c9770 handle/make/config: fix whitespace and 80-line bug
+* d28ad6aa build/release/roms: use -T0 on serprog tarballs
+* 308c21dd build/boot/roms stragglers: properly handle errors
+* c16b28ef build/release/src: re-create symlinks, don't copy
+* 32dcf9e5 coreboot/qemu_x86_12mb: re-add this mainboard
+* 5aef8156 scripts: use printf, not echo, where appropriate
+* 76e12cd4 update/blobs printf statements: use double quotes
+* 84bf47b5 scripts: better handling of printf: stdout/stderr
+* b78009e2 checkgit: properly print output to stderr
+* f45f5e62 update/project/*: remove redundant checks
+* 3e76e70d blobs/download: don't use the -B option in make
+* 877c691e build/release/roms: remove errant line break
+* f03efbc2 blobs/inject: add error condition on rm command
+* 20be007f blobs/inject: fix checksum validation if no-ucode
+* f989d5b4 blobs/sources: fix backup links on some files
+* 878550d5 use sha512sum to check downloads, not sha1sum
+* 022e0200 Merge pull request 'Add stm32-vserprog' (#129) from Riku_V/lbmk:stm32 into master
+|\
+| * bed444ff Add stm32-vserprog
+* | e9e4ada5 build/boot/rom: only insert scan.cfg if needed
+* | 0e3f3efc build/boot/roms: delete tmpcfg when done
+* | a69e8548 set grub.cfg timeout to 5s (10 on some boards)
+* | 4a459b02 Merge pull request 'pico-serprog improvements' (#128) from Riku_V/lbmk:master into master
+|\|
+| * 7b6fb958 Build pico-serprog binary release archive
+| * c292e01b Build for all pico board, not just the "original"
+| * 1bde6bb3 Support multiple dependencies per project
+| * 4d3b16da Cleaner parent directory creation
+|/
+* 7e8465be grub: re-add module: play
+* e3b9dfc9 util/nvmutil: put code all in nvmutil.c
+* 8fc5f6ed update/blobs/inject: use tmp/romdir, not TMPDIR
+* da991262 build/release/roms: use tmp/romdir, not TMPDIR
+* 15081ed9 grub: make backgrounds configurable in target.cfg
+* 0d315c3a curl/wget downloads: set re-try count to 3
+* bdf171e3 don't use /tmp/ for git clones. use tmp/ instead.
+* 196f293a build/release/roms: fix ucode handling
+* c0c7f3ae build/release/roms: simplify defcongic handling
+* a56cad71 update/blobs: unify global variables
+* 2cbc7eea update/blobs/*: unify checking of defconfig files
+* 52677309 update/blobs/extract: replace errant target code
+* ea7fae97 build/boot/roms: don't create empty bin/ directory
+* c62a4239 update/blobs/inject: remove errant target handling
+* 950166da update/blobs/download: remove errant code
+* 0668d234 add checkversion to build/release/src
+* c92a596c grub: remove xnu module
+* e659ddd8 grub: remove legacy file system modules
+* cf535785 re-add grub modules cat, eval and pbkdf2
+* 33e6088a move script/misc/versioncheck to main directory
+* 2c769dc1 move me7_update_parser.py to util/
+* da3c9bb3 merge config/ and resources/
+* a0501050 blobs/download: don't handle ifd/gbe files
+* 03788d14 move ifd/gbe configs into config/ifd/
+* 6ddb0e09 run make oldconfig on coreboot/default mainboards
+* 19efdf9e ich9m mainboards: use pre-assembled ifd/gbe files
+* af8d8cda add ich9m ifd/gbe files
+* d554efae build/release/src: copy e6430 ifd/gbe
+* 09aae7be build/rpi-pico-serprog: better error handling
+* 1dc54608 fix rpi-pico builds when running it twice
+* c63052cf fix memtest86plus download/build
+* fb4e6834 Merge pull request 'Add Dell Latitude E6430' (#124) from nic3-14159/lbmk:e6430 into master
+|\
+| * ebc04e52 Add Dell Latitude E6430
+* | 71d361aa Merge pull request 'Less cat abuse' (#123) from Riku_V/lbmk:cat into master
+|\ \
+| * | ef3fb05d Less cat abuse
+* | | eebf7133 switch repo links for pico-serprog
+| |/
+|/|
+* | 9ef8a7ea Merge pull request 'Automate pico-serprog builds' (#122) from Riku_V/lbmk:mkserprog into master
+|\|
+| * e369e8fb automate rpi-pico serprog builds
+|/
+* 92b4db69 build/release/src: only clean kbc1126 if it exists
+* 7c6b35cf unify build/clean scripts: use handle/make instead
+* cec37747 build/release/*: use -T0 in xz, for multithreading
+* b4b63adb don't support ucode removal on untested targets
+* b30c7e33 coreboot/e6400: support nvidia models
+* 436b2ccb handle/make/config -m/-u: actually copy configs
+* 3c7b09ac handle/make/config: properly handle cbutils
+* a3bc7ccd handle/make/file: fix uefitool builds
+* 4885c796 handle TMPDIR from include/export.sh
+* 56f16bc8 don't do cmake on uefitool if the Makefile exists
+* 98d1ea5a build/release/src: bugfix: actually copy cb/ub/sb
+* 755f925a build/release/src: copy handle symlink
+* 3ad29d2d build/release/src: remove Makefile reference
+* d69c231e build/release/src: fix bad variable reference
+* 38440153 update build/release/src based on lbmk changes
+* 0e782e7e update the fetch scripts themselves
+* 98f30b6d build/coreboot/utils: exit 1 if target.cfg missing
+* b9662fbe handle project downloads in main lbmk script
+* 12b33eb8 lbmk script: always clean up /tmp files
+* 225e2609 only remove microcode in build/release/roms
+* bf774acf move build/boot/rom moverom to handle/make/config
+* e5546128 build/release/roms: fix syntax error
+* fbda0f04 re-add /dev/null redirect on . ${1}
+* b2bad5a0 build/release/src: copy the include/ directory
+* eb54e427 grub: all one grub.elf containing keymaps and cfg
+* c6fd4d2a lbmk: run ./build dependencies *before* root check
+* 6722624d build/boot/roms: fix bad variable assignment
+* 55be6dda dependencies/ubuntu2004: update based on debian
+* 0052f9d0 fix: don't require git config for dependencies
+* 6dbddf85 build/boot/roms: simplify ich9m ifd handling
+* f5787c9e build/boot/roms ich9m ifd: use fast dd command
+* d9292cec build/boot/roms: use the new coding style
+* 4623f3f2 Remove superfluous GRUB modules (save CBFS space)
+* 623c3389 fix typo in error message ("as not permitted")
+* 4a280c62 .gitcheck: re-write entirely. force global config.
+* 355eb765 move resources/scripts/ to script/
+* eed34d3e enable memtest86plus on various boards
+* bc0fb51d x86 desktops: only enable seabios_withgrub
+* 9457d6be unified list command for all scripts
+* 93d2dcad handle/make/config: add missing pipes for err
+* 0e6851c8 delete the Makefile
+* ebbefa60 handle/config/file: rename to handle/make/config
+* df6db1c6 handle/config: fix errant "handle src for" call
+* 6874bc39 "handle src for" - change to handle make file
+* 798ce03a handle/config: add missing error handle
+* 29a8193e build/src/for: rename to handle/make/file
+* 27c67295 handle/config/file: unified distclean handling
+* 197464bc build/src/for: use -j for multithreaded builds
+* 95f290d9 build/release/src: update based on recent changes
+* 5a47c01b scripts: put quotes around file/directory names
+* 1c8401be much, much stricter, more verbose error handling
+* 50c395df .gitcheck: continue if no .git (don't break)
+* be7a5b0c .gitcheck: must stricter error handling
+* 3a5ba57f .gitcheck: only redirect stdout to /dev/null
+* 8f4f0e00 use the new coding style in scripts
+* 4c6c7d10 scripts: never exit 1, always call err instead
+* 52f3fd35 blobs/download: copy dl_path, don't move it
+* 57adbc6e unify err functions across scripts
+* b3fbcdf6 .gitignore: ignore *all* seen files
+* 24f09335 Merge pull request 'hp8300usdt: enable mSATA' (#118) from Riku_V/lbmk:master into master
+|\
+| * df1e8913 hp8300usdt: enable mSATA
+* | dfb93166 Merge pull request 'memtest86+ v6.20' (#116) from Riku_V/lbmk:memtest into master
+|\|
+| * fa926632 memtest86+ v6.20
+|/
+* 1bd84209 Merge pull request 'osbmk->lbmk' (#117) from Riku_V/lbmk:osbmk-lbmk into master
+|\
+| * 7be203dd osbmk->lbmk
+* | 04ee2672 also clean up the main scripts
+* | 62f23123 general code cleanup on lbmk shell scripts
+|/
+* 7be47065 unify build/defconfig and modify/defconfig
+* 0faf2a0c main lbmk script: exit non-zero if argc is wrong
+* 6e92d9a3 fix "./build help"
+* 9031bb7b unify dependencies scripts
+* 023d6b69 unify build/clean into ./build release src
+* f893a29b unify most module build scripts
+* c83d1a8d unify grub scripts under one directory
+* 438bf2c9 grub/modules.list: add argon2
+* fd602532 grub: import phc argon2 implementation (for luks2)
+* 2c0c521e bump grub revision a bit
+* e076d893 unify update/modify coreboot/u-boot/seabios script
+* e25984d7 remove board: qemu_x86_12mb (not usable for now)
+* e5b898f6 consolidate u-boot/seabios/coreboot build scripts
+* 673b144a coreboot/fam15h: fix for gcc/gnat building
+* 63b0e99f don't call blobutil directly from lbmk
+* 08486227 remove download scripts, consolidate into script
+* 8459e33b improve user feedback in blobutil
+* 59dba6cf merge coreboot/u-boot download logic to one script
+* 2453c303 gitclone: always clean up /tmp
+* adeb065c fix permissions on arch dependencies script
+* 6075fed8 NEW BOARD: HP EliteBook 8470p (Intel GPU)
+* f9afeb6f NEW BOARD: Dell Precision T1650
+* f8f77cb2 NEW BOARD: HP EliteBook 2170p
+* c5c89467 Merge pull request 'Update 'README.md'' (#89) from ewpr5kwu/lbmk:master into master
+|\
+| * 5204f0a9 Update 'README.md'
+* cb8bf380 bump seabios revision to 30 May 2023
+* 27ee975e bump grub revision to 2.12-rc1
+* 705149a3 coreboot/default: bump revision to 2 August 2023
+* 22ee7f74 blobs/download: save ME file to correct location
+* cdd83ab1 blobs/download: try backup if bad hash on main
+* f18b1859 blobs/download: support more formats on ME extract
+* f0efaf79 add unar to dependencies scripts
+* e8ba0f87 blobs/download: declare full user agent
+* 4875eef1 blobs/download: properly handle backup/main url
+* cca93ca3 blobs/download: don't download backup on main
+* 3aeefaa7 blobs/download: set common user agent string
+* 5e83d2bc blobs/download: simplify for loop
+* 8f1d3ad1 scripts: fix indentation in switch/case blocks
+* 748e0972 blobutil/ec: abort if kbc1126 ec extraction fails
+* e594ac16 coreboot/fam15h: remove unused files
+* 44bd077a Revert "build/boot/roms mkUBootRoms: initialise variables"
+* 7c90a407 build/boot/roms mkUBootRoms: initialise variables
+* d918139f coreboot/fam15h: re-enable microcode updates
+* 8c777428 board/qemu_x86: don't enable u-boot
+* fb44c349 coreboot/haswell: fix acpica downloads
+* af084014 coreboot: re-add asus kgpe-d16/kcma-d8/kfsn4-dre
+* e6002b91 coreboot/cros: fix acpica downloads
+* f34e07ae build/boot/roms: fix coreboot-version in releases
+```
+
+This is 445 changes in total, since Libreboot 20230625.
--
GitLab