| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat: MariaDB GTID support gh-ost hard-coded the go-mysql binlog syncer to MySQLFlavor and parsed every GTID set as MySQL, so --gtid could not be used against MariaDB. go-mysql already speaks the MariaDB GTID dialect; this wires it up. - Detect server flavor from the version string (IsMariaDB / FlavorFor) and set it on the BinlogSyncerConfig. - Make GTIDBinlogCoordinates hold the flavor-agnostic gomysql.GTIDSet interface; parse via ParseGTIDSet(flavor, ...). - Handle MariadbGTIDEvent alongside GTIDEvent in the streamer, and read the committed GTID set from XIDEvent.GSet without a MySQL-only cast. - Read MariaDB GTID positions from @@global.gtid_binlog_pos, Gtid_IO_Pos and @@global.gtid_slave_pos (MariaDB has no Executed_Gtid_Set column nor gtid_mode / enforce_gtid_consistency). - Skip the gtid_mode / enforce_gtid_consistency validation on MariaDB, where GTIDs are always recorded when binary logging is enabled. localtests: - Detect the server version once and reuse it; run gtid_mode=ON tests on MariaDB (normalize its current_gtid_mode to ON) and trim the GTID diagnostics that don't exist on MariaDB. - Enable gtid_strict_mode on the MariaDB test servers. Because --test-on-replica makes gh-ost write locally on the replica, give the replica its own GTID domain so its writes never collide with the primary's domain-0 stream under strict mode. - Add the gtid-resume case: interrupt a --gtid migration mid-copy via the interactive 'panic' command, then --resume, exercising the GTID checkpoint round-trip (WriteCheckpoint persists the GTID set, ReadLastCheckpoint parses it back via NewGTIDBinlogCoordinates(flavor)). Passes across the MySQL, MariaDB and Percona CI matrix. Verified: full localtests suite passes on MySQL 5.7/8.0/8.4 and on the MariaDB matrix (10.5, 10.6, 10.11, 11.4, 11.8) with gtid_strict_mode on. | 2 个月前 | |
test(localtests): multi-flavor docker replica-test harness (#1722) Rework the replica-test harness to run the localtests suite against MySQL 5.7, 8.0, 8.4 and Percona Server 8.0 from one parameterized setup. - Move docker-compose into script/ and split per-version server config into docker/<flavor>-<major.minor>/ dirs (common.cnf, create_user.sql, start_replication.sql), selected from TEST_MYSQL_IMAGE. Optional flavor.env per dir for knobs (Percona reads /etc/my.cnf.d). - Configure replication from the harness once both servers are up rather than at container init, so that replica connects on the first try. - Keep mysql_native_password for 5.7/8.0/percona, caching_sha2_password for 8.4. - Speed-ups: native mysql client with docker-exec fallback, detect server version once, shorter replication polls, configurable copy-throttle window (THROTTLE_SECONDS, default 2s to still exercise binlog DML replay). - toxiproxy split into its own compose file. | 3 个月前 | |
Fix local tests by making .gopath writable to avoid toolchain rm permission errors (#1644) | 6 个月前 | |
Add go mod (#935) * Add a go.mod file * run go mod vendor again * Move to a well-supported ini file reader * Remove GO111MODULE=off * Use go 1.16 * Rename github.com/outbrain/golib -> github.com/openark/golib * Remove *.go-e files * Fix for `strconv.ParseInt: parsing "": invalid syntax` error * Add test for '[osc]' section Co-authored-by: Nate Wernimont <nate.wernimont@workiva.com> | 5 年前 | |
Update to `go1.21` (#1351) * go1.21 + bullseye Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * go1.21 + bullseye pt 2 Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * checkout before setup-go Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * go fmt Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Use golangci-lint 1.54.2 to support go1.21 Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * stop using io/ioutil to make linter happy Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Fix typo Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Lint Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * revert replica-tests CI to ubuntu 20 due to linker errors Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Update ensure-go-installed * use `ubuntu-latest` for `ci` job Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * stretch -> bullseye Signed-off-by: Tim Vaillancourt <tvaillancourt@slack-corp.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Tim Vaillancourt <tvaillancourt@slack-corp.com> | 2 年前 | |
CI: adapting to internal builds system | 6 年前 | |
CI: adapting to internal builds system | 6 年前 | |
build: standardize release asset naming (#1729) * build: standardize release asset naming (#1728) Rework build.sh so released assets have stable, predictable names that are easy to automate against: - drop the generated timestamp from tarball names - drop the "-binary" infix from tarball names - name macOS tarballs with "darwin" (matches GOOS) instead of "osx" - drop the version from .rpm/.deb filenames; the release download URL already carries it. Package metadata still records version + epoch - build arm64 .rpm and .deb in addition to amd64, using each ecosystem's arch convention (x86_64/aarch64 for rpm, amd64/arm64 for deb) - stop publishing the bare "gh-ost" binary; it was just the last build (darwin/arm64) and was ambiguous. Tarballs and packages cover every OS/arch - emit a SHA256SUMS manifest so downloads can be verified with `sha256sum -c SHA256SUMS` Resulting assets: gh-ost-{linux,darwin}-{amd64,arm64}.tar.gz gh-ost.{x86_64,aarch64}.rpm gh-ost.{amd64,arm64}.deb SHA256SUMS Also refactor build() to take GOOS/GOARCH directly, share common fpm flags via an array, and update the README build.sh description. * build: simplify packaging Dockerfile and dock script (#1728) Slim down the release packaging tooling: - Dockerfile.packaging: collapse the per-package RUN layers into one, drop packages already provided by the golang:bookworm base (git, tar, curl, gcc, g++, bash) and the unused rsync, and drop the legacy GOPATH src layout (the build is module-mode). Only ruby/ruby-dev/build-essential (for fpm) and rpm (for rpmbuild) are installed now. - Use a multi-stage build with a `scratch` artifacts stage so `docker build --output type=local` exports the assets straight to the host. This drops the intermediate container, bind mount, -it TTY and find|xargs cp dance from `script/dock pkg`. - build.sh: stage the fpm package tree in the system temp dir instead of under $buildpath, so it never lands among the exported release artifacts. | 3 个月前 | |
test(localtests): multi-flavor docker replica-test harness (#1722) Rework the replica-test harness to run the localtests suite against MySQL 5.7, 8.0, 8.4 and Percona Server 8.0 from one parameterized setup. - Move docker-compose into script/ and split per-version server config into docker/<flavor>-<major.minor>/ dirs (common.cnf, create_user.sql, start_replication.sql), selected from TEST_MYSQL_IMAGE. Optional flavor.env per dir for knobs (Percona reads /etc/my.cnf.d). - Configure replication from the harness once both servers are up rather than at container init, so that replica connects on the first try. - Keep mysql_native_password for 5.7/8.0/percona, caching_sha2_password for 8.4. - Speed-ups: native mysql client with docker-exec fallback, detect server version once, shorter replication polls, configurable copy-throttle window (THROTTLE_SECONDS, default 2s to still exercise binlog DML replay). - toxiproxy split into its own compose file. | 3 个月前 | |
feat: 🦭 MariaDB support 🦭 (#1724) * feat: support MariaDB 11.4 and 11.8 gh-ost mostly works against MariaDB unchanged; this makes the few MariaDB-specific spots correct and runs the replica test suite against MariaDB 11.4 and 11.8 (LTS) alongside the MySQL/Percona flavors. Core: - ReplicaTermFor keeps the legacy slave/master replication terminology on MariaDB. MariaDB reports versions >= 10 (greater than the 8.4 cutoff) but never adopted the new replica/source terms, and its SHOW REPLICA STATUS still emits legacy column names, so the MySQL 8.4 term map doesn't apply. - Accept BINLOG MONITOR as well as REPLICATION CLIENT in the grant check (MariaDB 10.5+ renamed the privilege). Test harness (localtests): - test.sh: pick slave/Master terminology for MariaDB, wait for replica catch-up via MASTER_GTID_WAIT/gtid_binlog_pos, and skip gtid_mode-gated tests where @@gtid_mode is unsupported instead of aborting the run. - Resolve the per-flavor config dir by specificity (docker/<flavor>-<major.minor>, then <flavor>-<major>, then bare <flavor>) so versions that share one config live in a single dir. All MariaDB versions use one docker/mariadb dir (log-bin + ROW + log-slave-updates, native-auth users, MariaDB-GTID replication, MYSQL_CLIENT=mariadb since 11.x dropped the mysql-named client); a version (or major line) needing different config overrides it by adding its own dir. The compose healthcheck honours MYSQL_CLIENT. - Make the zero-date failure tests deterministic across flavors with an explicit sql_mode (NO_ZERO_IN_DATE,NO_ZERO_DATE,...); they relied on MySQL's default sql_mode, which MariaDB doesn't share. - Drop the incidental NOT NULL from the generated-column tests (MariaDB forbids NOT NULL on generated columns) so they run on MariaDB; keep generated-columns-unique skipped (it puts a generated column in the PRIMARY KEY, which MariaDB forbids, Error 1903). - Add mariadb:11.4.12 and mariadb:11.8.8 to the CI matrix. * feat: support MariaDB 10.5, 10.6 and 10.11 Add the MariaDB 10.x LTS lines (10.5, 10.6, 10.11) to the replica test suite. MariaDB versions below 11.1 need one core fix: gh-ost passes transaction_isolation as a DSN param, which the driver applies as `SET transaction_isolation=<v>` on each connection. That system variable does not exist on MariaDB < 11.1 (which has only tx_isolation), so the connection fails with Error 1193 "Unknown system variable 'transaction_isolation'". tx_isolation was in turn removed in MySQL 8.0, so no single variable name is portable across the supported servers. - mysql.OpenDB strips the transaction_isolation param and instead runs the SQL-standard `SET SESSION TRANSACTION ISOLATION LEVEL <level>` on each new connection via a connector wrapper, accepted by every supported MySQL and MariaDB version. GetDBUri still emits the param unchanged, so DSN construction and its tests are unaffected. - Add mariadb:10.5.29, mariadb:10.6.27 and mariadb:10.11.18 to the CI matrix. | 3 个月前 | |
feat: 🦭 MariaDB support 🦭 (#1724) * feat: support MariaDB 11.4 and 11.8 gh-ost mostly works against MariaDB unchanged; this makes the few MariaDB-specific spots correct and runs the replica test suite against MariaDB 11.4 and 11.8 (LTS) alongside the MySQL/Percona flavors. Core: - ReplicaTermFor keeps the legacy slave/master replication terminology on MariaDB. MariaDB reports versions >= 10 (greater than the 8.4 cutoff) but never adopted the new replica/source terms, and its SHOW REPLICA STATUS still emits legacy column names, so the MySQL 8.4 term map doesn't apply. - Accept BINLOG MONITOR as well as REPLICATION CLIENT in the grant check (MariaDB 10.5+ renamed the privilege). Test harness (localtests): - test.sh: pick slave/Master terminology for MariaDB, wait for replica catch-up via MASTER_GTID_WAIT/gtid_binlog_pos, and skip gtid_mode-gated tests where @@gtid_mode is unsupported instead of aborting the run. - Resolve the per-flavor config dir by specificity (docker/<flavor>-<major.minor>, then <flavor>-<major>, then bare <flavor>) so versions that share one config live in a single dir. All MariaDB versions use one docker/mariadb dir (log-bin + ROW + log-slave-updates, native-auth users, MariaDB-GTID replication, MYSQL_CLIENT=mariadb since 11.x dropped the mysql-named client); a version (or major line) needing different config overrides it by adding its own dir. The compose healthcheck honours MYSQL_CLIENT. - Make the zero-date failure tests deterministic across flavors with an explicit sql_mode (NO_ZERO_IN_DATE,NO_ZERO_DATE,...); they relied on MySQL's default sql_mode, which MariaDB doesn't share. - Drop the incidental NOT NULL from the generated-column tests (MariaDB forbids NOT NULL on generated columns) so they run on MariaDB; keep generated-columns-unique skipped (it puts a generated column in the PRIMARY KEY, which MariaDB forbids, Error 1903). - Add mariadb:11.4.12 and mariadb:11.8.8 to the CI matrix. * feat: support MariaDB 10.5, 10.6 and 10.11 Add the MariaDB 10.x LTS lines (10.5, 10.6, 10.11) to the replica test suite. MariaDB versions below 11.1 need one core fix: gh-ost passes transaction_isolation as a DSN param, which the driver applies as `SET transaction_isolation=<v>` on each connection. That system variable does not exist on MariaDB < 11.1 (which has only tx_isolation), so the connection fails with Error 1193 "Unknown system variable 'transaction_isolation'". tx_isolation was in turn removed in MySQL 8.0, so no single variable name is portable across the supported servers. - mysql.OpenDB strips the transaction_isolation param and instead runs the SQL-standard `SET SESSION TRANSACTION ISOLATION LEVEL <level>` on each new connection via a connector wrapper, accepted by every supported MySQL and MariaDB version. GetDBUri still emits the param unchanged, so DSN construction and its tests are unaffected. - Add mariadb:10.5.29, mariadb:10.6.27 and mariadb:10.11.18 to the CI matrix. | 3 个月前 | |
Fix Go 1.25.12 checksum constants in ensure-go-installed | 1 个月前 | |
PanicOnWarnings option to detect SQL warnings and fail the copy process (#1500) * First pass at raise_on_warnings * Update dev.yml * add CLI option, ignore expected warnings, check for row count discrepancy * Count rows in each insert range prepared query - log insert warnings always - terminate if row count doesn't match and the PanicOnWarnings flag is set * Lint comments - ensure errors are handled in show warnings * update TestBuildUniqueKeyRangeEndPreparedQuery * Localtests for PanicOnWarnings with data loss * Unwrap CTE (mysql 5) * Update localtests/panic-on-warnings-duplicate-unique-values-on-column-type-change/extra_args Co-authored-by: Bastian Bartmann <accounts@bastianbartmann.de> * limit BuildUniqueKeyRangeEndPreparedQueryViaOffset subquery properly * Update Applier to support all unique indices with PanicOnWarnings. Add test coverage. * Impl code review feedback for PanicOnWarnings - documentation - remove dev.yml - remove unused variable * bump golangci-lint for local dev * Support altering index names with PanicOnWarnings * Fix string matching for PanicOnWarnings to correctly suppress warnings when renaming unique keys Error message formats are different across mysql distributions and versions --------- Co-authored-by: Bastian Bartmann <accounts@bastianbartmann.de> | 1 年前 | |
test(localtests): multi-flavor docker replica-test harness (#1722) Rework the replica-test harness to run the localtests suite against MySQL 5.7, 8.0, 8.4 and Percona Server 8.0 from one parameterized setup. - Move docker-compose into script/ and split per-version server config into docker/<flavor>-<major.minor>/ dirs (common.cnf, create_user.sql, start_replication.sql), selected from TEST_MYSQL_IMAGE. Optional flavor.env per dir for knobs (Percona reads /etc/my.cnf.d). - Configure replication from the harness once both servers are up rather than at container init, so that replica connects on the first try. - Keep mysql_native_password for 5.7/8.0/percona, caching_sha2_password for 8.4. - Speed-ups: native mysql client with docker-exec fallback, detect server version once, shorter replication polls, configurable copy-throttle window (THROTTLE_SECONDS, default 2s to still exercise binlog DML replay). - toxiproxy split into its own compose file. | 3 个月前 | |
test(localtests): multi-flavor docker replica-test harness (#1722) Rework the replica-test harness to run the localtests suite against MySQL 5.7, 8.0, 8.4 and Percona Server 8.0 from one parameterized setup. - Move docker-compose into script/ and split per-version server config into docker/<flavor>-<major.minor>/ dirs (common.cnf, create_user.sql, start_replication.sql), selected from TEST_MYSQL_IMAGE. Optional flavor.env per dir for knobs (Percona reads /etc/my.cnf.d). - Configure replication from the harness once both servers are up rather than at container init, so that replica connects on the first try. - Keep mysql_native_password for 5.7/8.0/percona, caching_sha2_password for 8.4. - Speed-ups: native mysql client with docker-exec fallback, detect server version once, shorter replication polls, configurable copy-throttle window (THROTTLE_SECONDS, default 2s to still exercise binlog DML replay). - toxiproxy split into its own compose file. | 3 个月前 | |
initial work towards automated builds and CI | 9 年前 | |
Add script and docs for linter (#1151) | 4 年前 | |
test: enable race detection in unit tests | 1 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 2 个月前 | ||
| 3 个月前 | ||
| 6 个月前 | ||
| 5 年前 | ||
| 2 年前 | ||
| 6 年前 | ||
| 6 年前 | ||
| 3 个月前 | ||
| 3 个月前 | ||
| 3 个月前 | ||
| 3 个月前 | ||
| 1 个月前 | ||
| 1 年前 | ||
| 3 个月前 | ||
| 3 个月前 | ||
| 9 年前 | ||
| 4 年前 | ||
| 1 个月前 |