2.2.6 (Jan 26th, 2021)
Metaflow 2.2.6 Release Notes
The Metaflow 2.2.6 release is a minor patch release.
Features
Support AWS Fargate as compute backend for Metaflow tasks launched on AWS Batch
At AWS re:invent 2020, AWS announced support for AWS Fargate as a compute backend (in addition to EC2) for AWS Batch. With this feature, Metaflow users can now submit their Metaflow jobs to AWS Batch Job Queues which are connected to AWS Fargate Compute Environments as well. By setting the environment variable - METAFLOW_ECS_FARGATE_EXECUTION_ROLE , users can configure the ecsTaskExecutionRole for the AWS Batch container and AWS Fargate agent. PR: #402
Support shared_memory, max_swap, swappiness attributes for Metaflow tasks launched on AWS Batch
The @batch decorator now supports shared_memory, max_swap, swappiness attributes for Metaflow tasks launched on AWS Batch to provide a greater degree of control for memory management. PR: #408
Support wider very-wide workflows on top of AWS Step Functions
The tag metaflow_version: and runtime: is now available for all packaged executions and remote executions as well. This ensures that every run logged by Metaflow will have metaflow_version and runtime system tags available. PR: #403
Bug Fixes
Assign tags to Run objects generated through AWS Step Functions executions
Run objects generated by flows executed on top of AWS Step Functions were missing the tags assigned to the flow; even though the tags were correctly persisted to tasks. This release fixes and brings inline the tagging behavior as observed with local flow executions. PR: #386
Pipe all workflow set-up logs to stderr
Execution set-up logs for @conda and IncludeFile were being piped to stdout which made manipulating the output of commands like python flow.py step-functions create --only-json a bit difficult. This release moves the workflow set-up logs to stderr. PR: #379
Handle null assignment to IncludeFile properly
A workflow executed without a required IncludeFile parameter would fail when the parameter was referenced inside the flow. This release fixes the issue by assigning a null value to the parameter in such cases. PR: #421