| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
Refactor installation pages (#18282) This PR splits ad improves the installation-related documentation for Airflow. The "installation" page had become overloaded wiht everything-but-the-kitchen-sink and it became rather difficult to navigate and link to relevant sections. Also there was not a single page where one could have an overview on different installation methods possible, cases when each instalation works best as well as understanding what is involved in following each installation method in terms of maintenance, and expectations that users should have when it comes to what Apache Airflow Community provides. The PR leaves the installation page as basically a summary of all installation methods with all above explained and links to detailed pages explaining prerequisites, dependencies, database setup and supported versions. | 4 年前 | |
Update sftp_to_s3.rst (#29869) This operator does not use AWS transfer family. it just connects to SFTP, copies the file locally and puts it on S3. | 3 年前 | |
Amazon appflow (#24057) * Add Amazon AppFlow hook. * Add Amazon AppFlow operators. * Add Amazon AppFlow examples. * Add Amazon Appflow docs. * Apply comments/docs patterns. * Removing the "private" attribute signal and more. * Fix task_ids for example_appflow. * Move datetime_to_epoch() to utils and more. * Fix the AppflowBaseOperator name. * Ignore AppflowBaseOperator during structure check. * test_short_circuit refactor. * Add get_airflow_version. * Update airflow/providers/amazon/aws/hooks/appflow.py Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Update airflow/providers/amazon/aws/operators/appflow.py Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Update airflow/providers/amazon/aws/operators/appflow.py Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Update airflow/providers/amazon/aws/operators/appflow.py Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Update airflow/providers/amazon/aws/operators/appflow.py Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Update airflow/providers/amazon/aws/operators/appflow.py Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Addressing Josh's requests. * Add cached_property to AppflowHook * Update airflow/providers/amazon/aws/hooks/appflow.py Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Update airflow/providers/amazon/aws/operators/appflow.py Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Update airflow/providers/amazon/aws/operators/appflow.py Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * Update Josh's comment. * Update cached_property import. * Fix mypy. Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> | 4 年前 | |
Add automated version replacement in example dag indexes (#28090) Despite earlier attempts to fix the links there were still many places where the links were pointing to wrong versions of examples. It has been Noticed in: https://github.com/apache/airflow/pull/27774#issuecomment-1321166935 This change addresses it in three ways: * updated tool to fix the linx in historically released documentation (applied already in https://github.com/apache/airflow-site/pull/706) * replaced hard-coded version with |version| everywhere and added sphinx extension to convert those to actual provider versions (though due to sphinx limitation it has to be done with post-processing of the generated .html rather than replacing during generation) * added pre-commit that detects any use of example_dags links to main/master/hard-coded version. | 3 年前 | |
Convert the batch sample dag to system tests (AIP-47) (#24448) | 3 年前 | |
Convert Cloudformation Sample DAG to System Test (#24447) | 4 年前 | |
Migrate datasync sample dag to system tests (AIP-47) (#24354) * Migrate datasync sample dag to system tests (AIP-47) * Use SystemTestContextBuilder to pass along data | 4 年前 | |
Migrate DMS sample dag to system test (#26270) | 3 年前 | |
Add EC2CreateInstanceOperator, EC2TerminateInstanceOperator (#29548) * Add EC2CreateInstanceOperator and EC2TerminteInstanceOperator Change system test to use the new operators Add unit tests for new operators * Add support for multiple ids to EC2TerminateInstanceOperator Change system test to terminate without stopping instances * Fix failing tests for terminate operator * Update doc strings to add that the operators can create/terminate multiple instances Add tests for creating/terminating multiple instances * Fix system test so it passes Fix doc string on EC2TerminateInstanceOperator --------- Co-authored-by: syedahsn <syedahsn@ud74d1a752d7e5b.ant.amazon.com> | 3 年前 | |
ECS System Test (#26808) | 3 年前 | |
Convert example_eks_with_fargate_in_one_step.py and example_eks_with_fargate_profile to AIP-47 (#26537) Co-authored-by: ferruzzi <ferruzzi@amazon.com> | 3 年前 | |
Add EMR Notebook operators (#28312) | 3 年前 | |
Convert emr_eks example dag to system test (#26723) | 3 年前 | |
| 3 年前 | ||
Add GlacierUploadArchiveOperator (#26652) * Add GlacierUploadArchiveOperator | 3 年前 | |
Convert Glue Sample DAG to System Test (#25136) | 3 年前 | |
Move operator guides to provider documentation packages (#12681) | 5 年前 | |
Standardize AWS lambda naming (#29749) closes: https://github.com/apache/airflow/issues/29677 | 3 年前 | |
Convert Quicksight Sample DAG to System Test (#25696) | 3 年前 | |
Add RdsStopDbOperator and RdsStartDbOperator (#27076) * Add operator classes Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com> Co-authored-by: eladkal <45845474+eladkal@users.noreply.github.com> | 3 年前 | |
Add deferrable mode in RedshiftPauseClusterOperator (#28850) In this PR, I'm adding aiobotocore as an additional dependency until https://github.com/aio-libs/aiobotocore/issues/976 has been resolved. I'm adding AwsBaseAsyncHook a basic async AWS hook. This will at present support the default botocore auth i'e if airflow connection is not provided then auth using ENV and if airflow connection is provided then basic auth with secret-key/access-key-id/profile/token and arn-method. maybe we can support the other auth incrementally depending on the community interest. Because the dependency making things a little bit complicated so I have created a new test module deferrable inside AWS provider tests and I'm keeping the async-related test in this particular module. I have also added a new CI job to test/run the AWS deferable operator tests and I'm ignoring the deferable tests in other CI job test runs. Add a trigger class which will wait until the Redshift pause request reaches a terminal state i.e paused or fail, We also have retry logic like the sync operator. This PR donates the following developed RedshiftPauseClusterOperatorAsync` in [astronomer-providers](https://github.com/astronomer/astronomer-providers) repo to apache airflow. | 3 年前 | |
Redshift System Test (AIP-47) (#26187) * Update example_redshift_sql.py to use SystemTestContextBuilder Rebase to main * Create Redshift Cluster to run SQL queries on, as well as a Security Group to allow access - Add template field to RedshiftHook - Add template field to RedshiftCreateClusterOperator - Add template field to RedshiftSQLOperator - Make some changes requested by Dennis regarding formatting * Combine Redshift SQL system test and Redshift Data Execute System test to reduce resource creation overhead. Change RedshiftDeleteClusterOperator to use boto3 API waiter Change RedshiftPauseClusterOperator and RedshiftResumeClusterOperator to raise Exception on failure * Remove example_redshift_cluster.py - this is consolidated in example_redshift.py Remove documentation references to deleted files. Use RedshiftCreateClusterSnapshotOperator and RedshiftDeleteClusterSnapshotOperator instead of custom tasks. * Add missing end-before pattern * Add missing template fields, change function param names to pass checks * fix redshift snapshot test_create_cluster_snapshot_with_wait test * Make fixes to pass static checks * Remove unused imports | 3 年前 | |
Redshift System Test (AIP-47) (#26187) * Update example_redshift_sql.py to use SystemTestContextBuilder Rebase to main * Create Redshift Cluster to run SQL queries on, as well as a Security Group to allow access - Add template field to RedshiftHook - Add template field to RedshiftCreateClusterOperator - Add template field to RedshiftSQLOperator - Make some changes requested by Dennis regarding formatting * Combine Redshift SQL system test and Redshift Data Execute System test to reduce resource creation overhead. Change RedshiftDeleteClusterOperator to use boto3 API waiter Change RedshiftPauseClusterOperator and RedshiftResumeClusterOperator to raise Exception on failure * Remove example_redshift_cluster.py - this is consolidated in example_redshift.py Remove documentation references to deleted files. Use RedshiftCreateClusterSnapshotOperator and RedshiftDeleteClusterSnapshotOperator instead of custom tasks. * Add missing end-before pattern * Add missing template fields, change function param names to pass checks * fix redshift snapshot test_create_cluster_snapshot_with_wait test * Make fixes to pass static checks * Remove unused imports | 3 年前 | |
Convert S3 example DAG to System test (AIP-47) (#26535) * S3 system test (AIP-47) using SystemTestContextBuilder * Move constants to module level Add task to delete second bucket | 3 年前 | |
new operator to create a sagemaker experiment (#28837) * new operator to create a sagemaker experiment * address comments but failing test ? * fix default to right type Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> * make jinja work in test Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com> | 3 年前 | |
Convert SNS Sample DAG to System Test (#24384) | 4 年前 | |
Convert SQS Sample DAG to System Test (#24513) | 4 年前 | |
Convert Step Functions Example DAG to System Test (AIP-47) (#24643) * Add Amazon System Test Context Builder This factory class builds a task that will generate the environment id and fetch any variables at runtime and store them in xcom * Convert step functions example dag to system test Moved example_step_functions example dag AWS system test dir. Convert to AIP-47 standard. | 4 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 4 年前 | ||
| 3 年前 | ||
| 4 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 4 年前 | ||
| 4 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 5 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 4 年前 | ||
| 4 年前 | ||
| 4 年前 |