| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| Add deferrable mode in RedshiftPauseClusterOperator (#28850) In this PR, I'm adding aiobotocore as an additional dependency until https://github.com/aio-libs/aiobotocore/issues/976 has been resolved. I'm adding AwsBaseAsyncHook a basic async AWS hook. This will at present support the default botocore auth i'e if airflow connection is not provided then auth using ENV and if airflow connection is provided then basic auth with secret-key/access-key-id/profile/token and arn-method. maybe we can support the other auth incrementally depending on the community interest. Because the dependency making things a little bit complicated so I have created a new test module deferrable inside AWS provider tests and I'm keeping the async-related test in this particular module. I have also added a new CI job to test/run the AWS deferable operator tests and I'm ignoring the deferable tests in other CI job test runs. Add a trigger class which will wait until the Redshift pause request reaches a terminal state i.e paused or fail, We also have retry logic like the sync operator. This PR donates the following developed RedshiftPauseClusterOperatorAsync` in [astronomer-providers](https://github.com/astronomer/astronomer-providers) repo to apache airflow. | 3 年前 |
| Prepare docs for 02 2023 midmonth wave of Providers (#29589) * Prepare docs for 02 2023 midmonth wave of Providers * add apache.beam to release * add cncf.kubernetes, docker and update google | 3 年前 |
| Prepare ad hoc provider release for Docker, Cassandra, Papermill (#28999) | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Add note about workaround to potential error with argparse to hdfs (#29614) Instead of removing or reimplementing hdfs provider - in the event of a possibility to break argparse, we simply add a note in the provider documentation to remove argprase if such problem happens. Fixes: #22689 | 3 年前 |
| Prepare docs for 02 2023 midmonth wave of Providers (#29589) * Prepare docs for 02 2023 midmonth wave of Providers * add apache.beam to release * add cncf.kubernetes, docker and update google | 3 年前 |
| Prepare docs for Jan 2023 mid-month wave of Providers (#28929) * Prepare docs for Jan 2023 mid-month wave of Providers | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| Add guide for Apache Pinot operators (#28653) * add guide for Apache Pinot operators | 3 年前 |
| Add spark3-submit to list of allowed spark-binary values (#30068) * Add spark3-submit to list of allowed spark-binary values The list of allowed values for spark-binary was restricted in apache/airflow#27646. Add spark3-submit to this list to allow for distributions of Spark 3 that install the binary this way. See also apache/airflow#30065. * Fix lint errors in spark.rst and test_spark_submit.py | 3 年前 |
| Prepare docs for 02 2023 midmonth wave of Providers (#29589) * Prepare docs for 02 2023 midmonth wave of Providers * add apache.beam to release * add cncf.kubernetes, docker and update google | 3 年前 |
| Prepare docs for Feb 2023 wave of Providers (#29379) | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| Support creating Datadog connection from the UI and add documentation for this conn (#30059) * Add datadog connection to the UI connections list * customize Datadog connection UI * Add datadog connection documentation * Apply suggestions from code review Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Fix static checks --------- Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| Prepare docs for 02 2023 midmonth wave of Providers (#29589) * Prepare docs for 02 2023 midmonth wave of Providers * add apache.beam to release * add cncf.kubernetes, docker and update google | 3 年前 |
| Prepare docs for Feb 2023 wave of Providers (#29379) | 3 年前 |
| Prepare docs for Jan 2023 mid-month wave of Providers (#28929) * Prepare docs for Jan 2023 mid-month wave of Providers | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| Prepare docs for Feb 2023 wave of Providers (#29379) | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Remove unnecessary upper constraints from google provider (#29915) | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| Improve creating Vault connection from the UI and add documentation for this conn (#30057) | 3 年前 |
| Prepare docs for 02 2023 midmonth wave of Providers (#29589) * Prepare docs for 02 2023 midmonth wave of Providers * add apache.beam to release * add cncf.kubernetes, docker and update google | 3 年前 |
| Remove dws-sql | 3 年前 |
| Prepare docs for Jan 2023 wave of Providers (#28651) * Prepare docs for Jan 2023 wave of Providers * update | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| Prepare docs for Jan 2023 mid-month wave of Providers (#28929) * Prepare docs for Jan 2023 mid-month wave of Providers | 3 年前 |
| Add deferrable AzureDataFactoryPipelineRunStatusSensor (#29801) * Add deferrable data factory sensor * Add deferrable data factory sensor * Add tests * Fix tests * Fix tests * Apply review suggestions * Fix tests * Add docs * Add docs * Apply review suggestions * Set async conn variable | 3 年前 |
| Prepare docs for Jan 2023 wave of Providers (#28651) * Prepare docs for Jan 2023 wave of Providers * update | 3 年前 |
| Prepare docs for Jan 2023 mid-month wave of Providers (#28929) * Prepare docs for Jan 2023 mid-month wave of Providers | 3 年前 |
| Prepare docs for Jan 2023 wave of Providers (#28651) * Prepare docs for Jan 2023 wave of Providers * update | 3 年前 |
| Prepare docs for Jan 2023 mid-month wave of Providers (#28929) * Prepare docs for Jan 2023 mid-month wave of Providers | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| Updated docs for RC3 wave of providers (#27937) | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| Prepare ad hoc provider release for Docker, Cassandra, Papermill (#28999) | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| Prepare docs for Feb 2023 wave of Providers (#29379) | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| Prepare docs for Jan 2023 wave of Providers (#28651) * Prepare docs for Jan 2023 wave of Providers * update | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| pRepare docs for November 2022 wave of Providers (#27613) | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| Prepare docs for Jan 2023 mid-month wave of Providers (#28929) * Prepare docs for Jan 2023 mid-month wave of Providers | 3 年前 |
| Creating SMTP provider (#29968) | 3 年前 |
| Prepare docs for 02 2023 midmonth wave of Providers (#29589) * Prepare docs for 02 2023 midmonth wave of Providers * add apache.beam to release * add cncf.kubernetes, docker and update google | 3 年前 |
| Fix providers documentation formatting (#28754) | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Prepare docs for Feb 2023 wave of Providers (#29379) | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Prepare docs for Feb 2023 wave of Providers (#29379) | 3 年前 |
| Updated docs for RC3 wave of providers (#27937) | 3 年前 |
| Prepare docs for 03/2023 wave of Providers (#29878) * Prepare docs for 03/2023 wave of Providers | 3 年前 |
| Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 |
| Fixup some punctuation and grammar (#29342) After noticing one punctuation mistake around "however", I figured there must be more. So I fixed them, and some other stuff adjacent to them. | 3 年前 |
| Add Huawei Cloud provider | 3 年前 |
| Added Snowflake provider to the Docker image (#29171) | 3 年前 |
| Terminate doc build only on third-party sphinx inventories fetching errors (#30070) | 3 年前 |
| Add note about k8s version support (#29884) This will help prevent us accidentally dropping support for k8s versions earlier than we've agreed to it. | 3 年前 |
| Add huawei integration icon | 3 年前 |
| Enable string normalization in python formatting (other) (#27206) | 3 年前 |
| Move the old ./breeze script to scripts/tools/setup_breeze (#25584) We used to use Breeze via ./breeze script in the main airflow folder, but it's already long enough time after new breeze introduction to get-rid of it. However, the Bash script itself is pretty useful to automate Breeze installation on POSIX-compliant OS-es, so turning it into an installation script seems like a good idea. | 3 年前 |
| Fix links to sources for examples (#24386) The links to example sources in exampleinclude have been broken in a number of providers and they were additionally broken by AIP-47. This PR fixes it. Fixes: #23632 Fixes: https://github.com/apache/airflow-site/issues/536 | 4 年前 |
| Switch to ruff for faster static checks (#28893) Gone are: - isort - pyupgrade - pydocstyle - yesqa - autoflake - flake8 All replaced with [ruff](https://github.com/charliermarsh/ruff). A chunk of the performance of ruff comes from the fact that it makes very good use of multiple cores. And since most docker virtual machines are only one or two core I have chosen to run it directly, not inside the breeze docker container so we get the full benefit of speed. * Work around namespace packages issue for providers Ruff is currently detecting "google" as a the name of the current package, so it thinks it goes in the "first" party import section | 3 年前 |
| Revert "Google have re-worked their docs site ane appear not publish objects.inv (#28834)" (#29543) This reverts commit 44d93bea3adbe9429b6fdee14de31f80ac51c3bb. Google re-published the .inv files after deleting them in January. Details in https://github.com/googleapis/google-cloud-python/issues/10814 | 3 年前 |
| Change the name of link to ASF downloads (#19441) The ASF used to use mirrors to distribute their software, however recently they changed to use CDN. The mechanism might change in the future (even if currently CDN is used the ASF 'mirrors' page and closer.lua script provide a fully ASF-controlled mechanism to switch to the right mechanism, however technically speaking the current solution is not 'mirrors' but it is CDN, therefore it makes sense to rename it to generic downloads. | 4 年前 |
| Separate out documentation building per provider (#12444) * POC * fixup! POC | 5 年前 |
| fIx isort problems introduced by recent isort release (#28434) The recent isort changed their mind on sorting the imports. This change follows the change and bumps isort to latest released version (isort has no install_requires on its own so bumping min version has no effect on other dependencies) This change adds a number of isort:skip_file, isort:off, isort:skips in order to handle a very annoying bug in isort, that no matter how much you try, it sometimes treat "known first party" packages differently - depending on how many files it processes at a time. We should be able to restore it after this bug is fixed: https://github.com/PyCQA/isort/issues/2045 This change also updates the common.sql API to skip them from isort for the very same reason (depending on how many files are modified, the isort order might change. | 3 年前 |
| Add Huawei Cloud provider | 3 年前 |
| Update docs/readme.rst (#21346) | 4 年前 |