SSergei KlochkovImplement AWS IAM auth for Burrow
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
Implement AWS IAM auth for Burrow In order to implement the IAM auth mechanism for Burrow, the following changes were required. * The iamTokenProvider struct has been implemented to hold the iam parameters provided in the toml configuration file. * A helper function which leverages the aws-msk-iam-sasl-signer-go/signer methods in order to generate the auth token has been provided. * The token is being assigned to the saramaConfig.Net.SASL structure. Document TLS+IAM configuration * tls-iam.md contains a description of the IAM implementation and configuration. * Added a commented section to config/burrow.toml which provides a IAM configuration example. | 1 年前 | |
Fix mail template (#369) | 8 年前 | |
Initial commit | 11 年前 | |
Replace topicMap with topicPartitions in cluster module (#439) * consistent with notifier executeTemplate * replace topicMap with topicPartitions * add partial partitions update test | 7 年前 | |
Fix typo in slack close (#361) | 8 年前 | |
Promote Burrow 1.0 RC branch to master (#283) * Merge burrow-1.0 RC branch * Burrow 1.0 Release Candidate (#258) * Replace burrow with the proposed 1.0 framework Look, it's essentially a complete rewrite. There's almost nothing left of the original code here, and none of the modules have been fleshed out yet. The overall changes: * Make burrow itself a lib wrapped with main, so we can wrap it inside other applications * Move to a modular framework with well-defined interfaces between components * Switch logging to uber/zap and lumberjack * Start with being able to have parallel operation (notifier active eveywhere) so we can share load between instances * Restructure a bit to resolve import cycles * Make sure to gitignore the built binary * Move modules to internal packages * Tweak logging to work on windows * Clean up coordinators a little more * Fix syscalls for unix vs windows * First pass at inmemory storage module * tests for inmemory, and fixes found during testing * Additional tests to make sure channels are closed after replies * Actually start the mainLoop * Assure only 1 storage module is allow, and add coordinator tests * Fix storage code and tests for problems found while testing evaluators * Add a fixture for storage to create a coordinator with storage module for testing code outside storage * Fixes to evaluator code based on testing * Tests for the evaluator coordinator and caching module * Add a fixture for the evaluator that other testing can use * Add start/stop and multiple request tests for the evaluator coordinator * Remove extra parens * Fix config name * Add group whitelists to storage module, along with tests * Fix a potential bug in min-distance where we would never create a new offset * moar logging * Add a group delete request for storage modules * Added expiration of group data via lazy deletion on request * First pass at cluster module for kafka with limited tests * Add a shim interface for sarama.Client and sarama.Broker * Switch kafka cluster module to use the shim interface for sarama * Add tests for the rest of the kafka cluster module * Add a storage request for setting partition owner for a group * Add kafka_client consumer module and tests * Add consumer coordinator tests * Move the storage request send helper to a new file * Refactor names for the sarama shims * Add a shim for go-zookeeper so we'll be able to test * Implement the kafkazk consumer module and tests * Add tests for validation routines * comment fix * Add tests for helpers * Add whitelist support to consumers * Have the PID creator also check if the process exists before exiting * Restructure main ZK as a coordinator to use the common interface * Start notifiers, clean up some testing * Add tests for HTTP notifier module * Refactor notifier coordinator to move common logic out of the modules * Refactor notifier whitelist and threshold accept logic to coordinator * Move template execution up to a coordinator method for consistency * Email notifier * Slack notifier and tests * Use asserts instead of panics for the HTTP tests * Fix a case in the storage fixture where it won't get all the commits * Check http notifier profile configs * Make maxlag template helper use the CurrentLag field * Rename NotifierModule to just Module * Rename StorageModule to just Module * Rename EvaluatorModule to just Module * Add support for ZK locks, as well as tests * Add a ticker that can be stopped and restarted * Make the notifier coordinator use a ZK lock with the restartable ticker * Add HTTP server and tests * Update dependencies * Clean up HTTP tests so we test the router configuration code * Few more HTTP server tests, and flesh out log level set/get * Reorder imports * Fix copyright comments * Formatting cleanup * Set httprouter to master, since it hasn't released in 2 years * touch up logging * Remember to set the config as valid * Use master branch of testify * Updates found in testing * Check for null fields in member metadata * Fixes to metadata handling * Add a worker pool for inmemory to consistently process groups * Remove the kafka_client mainLoop, as it's not useful * Fix formatting and a duplicate logging field * Add support for CORS headers on the HTTP server * Add a template helper for formatting timestamps using normal Time format strings * Add support for basic auth in the HTTP notifier * Refactor config to use viper instead of gcfg * add more logging in Kafka clients, and fix config loading * fix typo in client-id config string * Catch errors when starting coordinators * Log the http listener info * Clean up some of the logging * Fix logging and notifiers from testing (#259) * Fix notifier logic in 1.0 (#261) * Fix how the extras field is pulled into the HTTP response structs * Make sure the module accept group is always called * Pause before testing stop on the storage coordinator * Fix conditions where notifications are sent, and add a much more robust test * 1.0 - Add jitter to notifier evaluations (#263) * Change the loop for evaluations to be started to be timed with jitter for each consumer group * reorder imports * Burrow 1.0 config defaults (#264) * Add owners to consumer group status response * If no storage module configured, use a default * If no evaluator module configured, use a default * Fix default http server * ConfigurationValid gets set by Start, not before * cleanup methods that don't need to be exported * Burrow 1.0 group blacklist (#266) * Add group blacklists * Reduce logging level for storage purging expired groups * Start evaluator and httpserver before clusters/consumers * Remove the requirement that you must have a cluster and consumer module defined * Explicitly update metadata for topics that had errors fetching offsets (#267) * Refresh metadata on leader failures as well (#268) * Make sure that whenever we are reading the cluster map in the notifier, we have a lock (#269) * Burrow 1.0 - No negative lag (#271) * Lag values should always be unsigned ints * unnecessary cast * Update deps * Start notifier before clusters and consumers (#272) * Remove slack notifier (#278) * Remove slack notifier * Add example slack templates * Burrow 1.0 - Godocs for everything (#281) * Godoc docs for everything, and resolve all golint issues * Burrow 1.0 - Doc cleanup (#282) * Update example configuration files * Fix example email template * Update docs | 8 年前 |