Workflows in a nutshell:

The workflow programming model has been highly successful in the last decade in facilitating the composition of medium to large-scale applications from existing basic functions (tasks) defining a sequence of functions to manage a science, business or industrial application. A function can be, for example, a risk management process in a business workflow, a matrix solver or a Fast Fourier Transform in a scientific workflow, or a car crash simulation in an industrial workflow. Functions are typically described through an abstract interface representing their functional signature (input and output data), which is independent of the underlying implementation language and technology. Function implementations can be realised using serial or parallel programming languages. A workflow can therefore be seen as a large-scale application consisting of a collection of such abstract functions, interconnected in a directed graph through control flow and data flow dependencies. Typical control flow constructs include sequences, if conditions, sequential loops, parallel sections, and parallel loops. Data flow constructs represent communication between functions and are usually specified by connecting output ports of predecessor activities to input ports of successor functions. Before being executed by a workflow enactor, a workflow specification needs to be scheduled by mapping each abstract function to an underlying deployed implementation. For each function there can be numerous different implementations.

Apollo overview:

The goal of Apollo is to support an automated life-cycle of scalable, interoperable, and performance-oriented workflows for heterogeneous edge-cloud infrastructures, following 3 main design principles:

  1. Deliver a single unified application programming model for the edge-cloud continuum in the form of portable workflows based on serverless Function-as-a-Service (FaaS) and container technology.
  2. Realise the "everything-as-a-service" (XaaS) policy across all software and hardware layers ranging from applications to the hardware resources of the edge-cloud continuum.
  3. Utilize existing open source technologies, built on widely-used industry standards.

Apollo researches and develops a novel and versatile platform for portable workflows that will harness the edge-cloud continuum over a marketplace of RS services (Resources-as-a-Service and system services). In APOLLO, we set focus on dynamic, data-intensive, event-driven workflows with volatile workloads. APOLLO will shield users and developers from low-level details of workflows, including how they are mapped and executed across well-known FaaS platforms on the edge-cloud continuum. The Apollo has a number of features and services to simplify development, porting and execution of workflows:

  • Abstract Function Choreography Language (AFCL): support the specification of workflows from science, business and industry with a novel YAML-based language that encompasses a rich set of nestable control- and data-flow constructs including parallel functions, selective join and merge operations, and constructs to gather and scatter data collections across workflow functions.
  • FaaSification: semi-automatic FaaSification of legacy codes to serverless functions;
  • Multi-objective scheduling: proactive and reactive scheduling of workflows for multiple, possibly conflicting objectives (makespan, costs, reliability) that can deal with the dynamic and volatile behaviour of applications and RS services;
  • Provider-agnostic workflow execution: provider-agnostic, automatic deployment and execution of workflows with automatic DevOps pipelines and the ability to run different functions of the same workflow across different FaaS providers (e.g. AWS Lambda, Azure Functions, Google Cloud Functions, Alibaba Function Compute and IBM Functions);

Distributed and Parallel Systems  Group   -

University of Innsbruck