| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
Update Bazel to 8.2.1 and use bzlmod The update to bzlmod [1] changes how we handle third_party deps (dramatically) and how our toolchains work (minorly). To review and understand the changes, I suggest going language by language through the changes: C++ === We have Bazel check out git repositories listed in deps.bzl (like before), but this has been converted to a module_extension [2] which behaves basically the same. In MODULE.bazel, we load the extension and then specify all the dependencies that Skia directly depends on from there. If we can depend on versions from the Bazel registry [3], for example bazel_dep(name = "libpng", version = "1.6.47.bcr.1") we could remove the git repository versions and this might make things easier for external clients if there are version mismatches. A big change was required with how we customize our compilation of expat, FreeType, Harfbuzz, and ICU. Previously, we'd copied in (or cleverly referenced) files from the Skia repo in the third party copy of the code and this sort of worked. External clients had to use some helpers to load these configs and it was Fine (tm). This does not work with bzlmod's dependency graph. I tried several things to make this work cleanly and ran into road blocks with either RBE or external client usage (or both). See earlier patchsets for that, if interested. What *did* work (at least ok) was making a patch file per third party dep that needed it using tools/generate_patches.py to combine all those files and specify an output location. This has the downside of not being in sync, but I made the existing roll scripts for expat, FreeType, Harfbuzz shell out to the script to minimize that. Toolchains needed to be slightly updated to have a different path in the external/ output directory, which now includes some prefixing. To make this as robust as possible, I made the trampoline scripts use paths relative to where the script lives and use a glob to find the path no matter what the prefix is. Clang remains at 15.0.1 for Bazel builds. Rust ==== The rust toolchain has been updated to 1.86.0 (this was the primary impetus for this change). External crate dependences use crate_universe [4] and specify the versions we need directly in MODULE.bazel instead of using a toml file. I chose that approach because it seemed more direct when one wants to figure out what version our Bazel build uses. All crates except cxx worked "out of the box", meaning the rules_rust generated BUILD.bazel files for them in a way that just worked. I had to use crate.annotation to customize cxx to use bazel/external/cxx/BUILD.bazel.skia in addition to the generated rule so we can overwrite the include path. This was a bit fiddly and required me to spend a while looking through the bazel cache to experiment with what the output was. The only other major change is how we reference rust dependencies now. For example in src/ports/fontations/BUILD.bazel see that all rust dependencies are based out of @crates instead of all being top level dependencies. Again, poking into the bazel cache is helpful for figuring out what the generated target names are. Another upside of this update is the big long "splicing workspaces" seems to have gone away. Golang ====== The golang version is now 1.24.2. These were the most fiddly changes - particularly with some changes in the ecosystem related to protos. In MODULE.bazel, the go.mod file is read in to get both the go version and the direct golang dependencies. go_rules uses gazelle [5] to generate BUILD.bazel files for any deps if they don't have them. This includes generating BUILD.bazel files for proto files, which are a bit messy due to there being some conflicting ways to include these and thus resolve the deps. This [6] is why I ended up mothballing the upgrade of the Skia Infra repo in favor of a minimal [7] change. In this case, we only had to tell the go.chromium.org/luci repo not to autogenerate .go files for .proto files because there are pre-existing ones. While building task drivers, there was a problem where some go deps were not being brought in and thus our transitive dependencies were failing to build. This is why tools.go was added (inspired by [8] to force those to be included. Python3 ======= This was a straightforward upgrade to Python 3.13. MODULE.bazel still reads requirements.txt to figure out what to download. The BUILD.bazel file we build for Dawn uses these requirements, but we didn't have to change how that worked at all. Other Notes =========== Renames ------- There are many mechanical changes (e.g. io_bazel_rules_go->rules_go) throughout as a result of bzlmod having different (non prefixed) package names in some cases. bazel/buildrc ------------- Sandboxing seems much faster in this new Bazel version on Mac. Thus I deleted the no_sandbox config. This change also updates the version of emsdk we use to compile CanvasKit. I had to squelch some warnings that they have via our config because I didn't see another way to override their toolchain settings. buildifier ---------- Updated the version used by Bazel to 8.0.3 I suggest folks also update their local version too, so presubmits work [9] Not working (yet) / disabled ============================ - karma_tests (JS tests for CanvasKit). This was based on long-deprecated rules and I didn't have the time to fix this. We still have GN based tests for CanvasKit, so this was low enough priority to skip for now - codesize_test. git_common.WithGitFinder was not working. As this is a test for our task_driver and has been stable for a while, it seemed like a low priority to fix. - bazel run //:gofmt. This moved with new go_rules and I couldn't find an easy replacement. - bazel run //:errcheck. This needs to shell out to golang and doesn't work on systems that don't have go installed because of the hermeticity. Not sure what to do about that. - clang_ios and clang_windows_amd64 toolchains. We don't have any CI jobs that use these atm, and they are lower priority for now. I made some changes based on other toolchains, but didn't test these. [1] https://bazel.build/external/migration [2] https://bazel.build/external/extension [3] https://registry.bazel.build/modules/libpng [4] https://bazelbuild.github.io/rules_rust/crate_universe_bzlmod.html [5] https://github.com/bazel-contrib/bazel-gazelle/blob/master/extensions.md#go_deps [6] https://skia-review.googlesource.com/c/buildbot/+/988440/11/MODULE.bazel [7] http://review.skia.org/990801 [8] https://github.com/google/skia-buildbot/blob/b4a7f41e01ea288eadd197f9eee0b1a8a12c5aa4/tools.go [9] https://github.com/bazelbuild/buildtools/releases/tag/v8.0.3 Change-Id: I68be15c5c4a5d2b14359fa460075591dafab336a Bug: b/413044303 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/984976 Commit-Queue: Kaylee Lubick <kjlubick@google.com> Reviewed-by: Dominik Röttsches <drott@google.com> Reviewed-by: Eric Boren <borenet@google.com> | 1 年前 | |
Update Bazel to 8.2.1 and use bzlmod The update to bzlmod [1] changes how we handle third_party deps (dramatically) and how our toolchains work (minorly). To review and understand the changes, I suggest going language by language through the changes: C++ === We have Bazel check out git repositories listed in deps.bzl (like before), but this has been converted to a module_extension [2] which behaves basically the same. In MODULE.bazel, we load the extension and then specify all the dependencies that Skia directly depends on from there. If we can depend on versions from the Bazel registry [3], for example bazel_dep(name = "libpng", version = "1.6.47.bcr.1") we could remove the git repository versions and this might make things easier for external clients if there are version mismatches. A big change was required with how we customize our compilation of expat, FreeType, Harfbuzz, and ICU. Previously, we'd copied in (or cleverly referenced) files from the Skia repo in the third party copy of the code and this sort of worked. External clients had to use some helpers to load these configs and it was Fine (tm). This does not work with bzlmod's dependency graph. I tried several things to make this work cleanly and ran into road blocks with either RBE or external client usage (or both). See earlier patchsets for that, if interested. What *did* work (at least ok) was making a patch file per third party dep that needed it using tools/generate_patches.py to combine all those files and specify an output location. This has the downside of not being in sync, but I made the existing roll scripts for expat, FreeType, Harfbuzz shell out to the script to minimize that. Toolchains needed to be slightly updated to have a different path in the external/ output directory, which now includes some prefixing. To make this as robust as possible, I made the trampoline scripts use paths relative to where the script lives and use a glob to find the path no matter what the prefix is. Clang remains at 15.0.1 for Bazel builds. Rust ==== The rust toolchain has been updated to 1.86.0 (this was the primary impetus for this change). External crate dependences use crate_universe [4] and specify the versions we need directly in MODULE.bazel instead of using a toml file. I chose that approach because it seemed more direct when one wants to figure out what version our Bazel build uses. All crates except cxx worked "out of the box", meaning the rules_rust generated BUILD.bazel files for them in a way that just worked. I had to use crate.annotation to customize cxx to use bazel/external/cxx/BUILD.bazel.skia in addition to the generated rule so we can overwrite the include path. This was a bit fiddly and required me to spend a while looking through the bazel cache to experiment with what the output was. The only other major change is how we reference rust dependencies now. For example in src/ports/fontations/BUILD.bazel see that all rust dependencies are based out of @crates instead of all being top level dependencies. Again, poking into the bazel cache is helpful for figuring out what the generated target names are. Another upside of this update is the big long "splicing workspaces" seems to have gone away. Golang ====== The golang version is now 1.24.2. These were the most fiddly changes - particularly with some changes in the ecosystem related to protos. In MODULE.bazel, the go.mod file is read in to get both the go version and the direct golang dependencies. go_rules uses gazelle [5] to generate BUILD.bazel files for any deps if they don't have them. This includes generating BUILD.bazel files for proto files, which are a bit messy due to there being some conflicting ways to include these and thus resolve the deps. This [6] is why I ended up mothballing the upgrade of the Skia Infra repo in favor of a minimal [7] change. In this case, we only had to tell the go.chromium.org/luci repo not to autogenerate .go files for .proto files because there are pre-existing ones. While building task drivers, there was a problem where some go deps were not being brought in and thus our transitive dependencies were failing to build. This is why tools.go was added (inspired by [8] to force those to be included. Python3 ======= This was a straightforward upgrade to Python 3.13. MODULE.bazel still reads requirements.txt to figure out what to download. The BUILD.bazel file we build for Dawn uses these requirements, but we didn't have to change how that worked at all. Other Notes =========== Renames ------- There are many mechanical changes (e.g. io_bazel_rules_go->rules_go) throughout as a result of bzlmod having different (non prefixed) package names in some cases. bazel/buildrc ------------- Sandboxing seems much faster in this new Bazel version on Mac. Thus I deleted the no_sandbox config. This change also updates the version of emsdk we use to compile CanvasKit. I had to squelch some warnings that they have via our config because I didn't see another way to override their toolchain settings. buildifier ---------- Updated the version used by Bazel to 8.0.3 I suggest folks also update their local version too, so presubmits work [9] Not working (yet) / disabled ============================ - karma_tests (JS tests for CanvasKit). This was based on long-deprecated rules and I didn't have the time to fix this. We still have GN based tests for CanvasKit, so this was low enough priority to skip for now - codesize_test. git_common.WithGitFinder was not working. As this is a test for our task_driver and has been stable for a while, it seemed like a low priority to fix. - bazel run //:gofmt. This moved with new go_rules and I couldn't find an easy replacement. - bazel run //:errcheck. This needs to shell out to golang and doesn't work on systems that don't have go installed because of the hermeticity. Not sure what to do about that. - clang_ios and clang_windows_amd64 toolchains. We don't have any CI jobs that use these atm, and they are lower priority for now. I made some changes based on other toolchains, but didn't test these. [1] https://bazel.build/external/migration [2] https://bazel.build/external/extension [3] https://registry.bazel.build/modules/libpng [4] https://bazelbuild.github.io/rules_rust/crate_universe_bzlmod.html [5] https://github.com/bazel-contrib/bazel-gazelle/blob/master/extensions.md#go_deps [6] https://skia-review.googlesource.com/c/buildbot/+/988440/11/MODULE.bazel [7] http://review.skia.org/990801 [8] https://github.com/google/skia-buildbot/blob/b4a7f41e01ea288eadd197f9eee0b1a8a12c5aa4/tools.go [9] https://github.com/bazelbuild/buildtools/releases/tag/v8.0.3 Change-Id: I68be15c5c4a5d2b14359fa460075591dafab336a Bug: b/413044303 Reviewed-on: https://skia-review.googlesource.com/c/skia/+/984976 Commit-Queue: Kaylee Lubick <kjlubick@google.com> Reviewed-by: Dominik Röttsches <drott@google.com> Reviewed-by: Eric Boren <borenet@google.com> | 1 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 1 年前 |