AFCL applications

AFCL 1.1

Due to the breaking changes introduced by the step from AFCL 1.0 to AFCL 1.1 certain adaptations are required w.r.t. the workflow description.

Updates:

General overview

AFCL is based on YAML which is a human-readable data serialization language. An AFCL Function Choreography (FC) consists of functions, which can be either base functions or compound functions. The former refers to a single computational task without further splitting it into smaller tasks, while the latter encloses some base functions or even nested compound functions. All base and compound functions can be connected by different control- and data-flow constructs. An FC is also a compound function. In order to create an FC, all its functions (base and compound) as well as control- and data-flow connections among them, must be specified. In order to facilitate optimized execution of FCs, a user optionally can specify properties and constraints for functions and data-flow connections. In order to simplify the reading of AFCL specifications, we use a meta-syntax which extends YAML, such that YAML elements can be contained in "{}" and appended with wildcards "?" (0 or 1), "*" (0 or more), "+" (1 or more), and "|" (logical or).

Base function

A base function represents a computational or data processing task. The name of a function serves as a unique identifier. Functions are described by type - Function Types (FTs) which are abstract descriptions of their corresponding function implementations (FIs). An FI represents an actual implementation of an FT. The FIs of the same function type are semantically equivalent, but may expose different performance or cost behavior or may be implemented with different programming languages, algorithms, etc. FTs shield implementation details from the FC developer. The selection of a specific FI for an FT is done by an underlying runtime system. 
function: {
  name: "name",
  type: "type",
  dataIns: [
    {
      name: "name",
      type: "type",
      source: "source"
    }+
  ]?,
  dataOuts: [
    {
      name: "name",
      type: "type"
    }+
  ]?
}
The input and output data of a function are specified through dataIns and dataOuts ports of the function, respectively. The name and type attributes of the dataIns/Outs ports are uniquely determined by the chosen FT. A dataIns port can be specified by
  • setting its source attributed to the name of a data port of another function within the same FC (dataIns from a parent compound function or dataOuts from a predecessor function) or the name of a dataIns port of the FC; or
  • specifying a constant or an ordered list of constants.
Linking the data ports of different functions through the source attributes defines the data-flow of AFCL FCs. The source of a dataIns/Outs port can be used as well to define constants. dataIns/Outs ports are optional as a function may perform some predefined action in which case there is no need to specify neither input nor output parameters. Every dataIns/Outs port is associated with a data type. The data types supported for AFCL are JSON datatypes: string, number, boolean, null/empty, object and array. The string type could be used to send a reference link (url) of a file located on an external storage.

Info! In the remainder we omit name, type and source for simplicity. Instead, we will use only expressions like dataIns[{}+]? and dataOuts[{}+]? for the list of data inputs and outputs of a function. Additionally, we will use the abbreviation function[{}+] to specify a list of base or compound functions.

Example

The following example represents a base function defined in AFCL. The name of the function is addEmployee, while the type (Function Type) of the function is addPersonToDataBase. The function has six inputs (dataIns):
  • The first input, named fullName, represents an input of type string. The data of this input comes from a function named otherFunction and its corresponding dataOuts named employeeName.
  • The second input represents the budget of an employee, which is of type number. The input does not come from a function, but is a default value of 1000.0
  • The input newcomer is a boolean which is true by default.
  • The address of an employee is of type object, which means that the input is a json object. The input comes from a function named otherFunction and its corresponding output employeeAddress. An example of such a json object could be: { "street_address": "Technikerstraße 21", "city": "Innsbruck", "state": "Austria" }. Additionally, the object type can be empty ({}) or null (null).
  • The payments input is of type object and has the value null.
  • The input personalData is of type array. The corresponding array contains values of different types.
The function has two outputs: 
function: {
  name: "addEmployee",
  type: "addPersonToDataBase",
  dataIns: [
    { name: "fullName" , type: "string" , source: "otherFunction/employeeName" },
    { name: "budget" , type: "number" , source: 1000.0 },
    { name: "newcomer" , type: "boolean" , source: true },
    { name: "address" , type: "object" , source: "otherFunction/employeeAddress" },
    { name: "payments" , type: "object" , source: null },
    { name: "personalData" , type: "array" , source: [ 13 , "2U6D3" , { "gender" : "male" } ] }
  ],
  dataOuts: [
    { name: "outArray" ,  type: "array" },
    { name: "success" ,  type: "boolean" }
  ]
}

Compound function

AFCL introduces a rich set of control-flow constructs (compound functions) to simplify the specification of realistic FCs that are difficult to be composed with any current FC system without support by a skilled software developer. Compound functions contain inner functions, which can be base or compound and they are executed in the order defined by the compound function. The inner functions are called children functions of the compound function. The compound function is called the parent function of the inner functions. An inner function of a compound function can be another compound or base function. The term child function refers to the entire compound function. AFCL introduces the following compound functions: sequence, parallel, if-then-else, while, and parallelFor. The specifications for the name attribute, dataIns and dataOuts ports, along with the corresponding source attribute are similar as for a base function, while the dataOuts port of a compound function is extended with the source attribute.

Info! In the remain der of this text, we will not separately explain the attributes of a compound function and when we use the term function, it can refer to either base function or compound function.

Sequence and Parallel

A sequence or parallel section of base or compount functions is implicitely expressed by the dataflow dependencies between the functions. These dependencies of functions are captured by the relation of their dataIns/dataOuts.

Example

Within this example the functions named f1 and f3 run in parallel. Both functions do not have any dependencies to other functions, and can therefore start immediately. After the termination of f1, the function f2 will start, because f2 depends on the output of f1. 
[
  {
    function: {
      name: "f1",
      type: "f1Type",
      dataOuts: [
        { name: "outVal" ,  type: "number" },
      ]
    }
  },
  {
    function: {
      name: "f2",
      type: "f2Type",
      dataIns: [
        { name: "inVal" , type: "number" , source: "f1/outVal" },
      ]
    }
  },
  {
    function: {
      name: "f3",
      type: "f3Type"
    }
  }
]

Conditional compound

A dataOuts port of a conditional compound has a source value with a comma separated list of dataOut ports of other functions. This entry must contain one element for each possible branch within the compound construct.

If-then-else

The if-then-else compound function is one of two conditional compound functions of AFCL. The condition attribute describes a set of subconditions. Each subcondition is combined to the next one with the combinedWith attribute. A sub-condition contains data1 and data2 of type type which represent the data to be compared according to the value of the operator ( ==, <, >, ≤, ≥ and !=, contains, startsWith and endsWith) and optional negation. The operations are only allowed for specific datatypes.

string number boolean object array
== ✅ ✅ ✅ ✅ ✅
<, >, ≤, ≥ ✅ ✅ ✅ ❌ ✅
!= ✅ ✅ ✅ ✅ ✅
contains ✅ ❌ ❌ ❌ ✅
startsWith ✅ ❌ ❌ ❌ ❌
endsWith ✅ ❌ ❌ ❌ ❌

The values of data1 and data2 can be constants or the output from previous functions. If the condition is satisfied then functions within the then part are executed, otherwise the else branch is executed, if present. 
if: {
  name: "name",
  dataIns: [{}+]?,
  condition:
    [
        {
          data1: "data1",
          data2: "data2",
          type: "type",
          operator: "operator",
          negation: "negation",
          combinedWith: "and/or",
        }+
    ],
  then: [{function: {}}+],
  else: [{function: {}}+]?,
  dataOuts: [{}+]?
}

Example

This example illustrates an if-then-else construct in AFCL. The construct has three inputs: sendNotification which is of type boolean, employeeID which is of type string and defaultObject which is of type object. All inputs come from other functions. The condition within the construct checks whether the input sendNotification is equal to true. In that case the then branch will be executed and the function notifyEmployee will be executed. The function has a sentTo dataOuts of type object. The output of the whole if condition is named notificationSentTo and of type object. The value after the if is either the value returned by the function notifyEmployee or the dataIns port defaultObject of the compound, if the then branch is not executed. 
if: {
  name: "sendNotificationOnSuccess",
  dataIns: [
    { name: "sendNotification" , type: "boolean" , source: "addEmployeeToDataBase/isEmployeeInDataBase" },
    { name: "employeeID" , type: "string" , source: "addEmployeeToDataBase/employeeID" },
    { name: "defaultObject" , type: "object" , source: "otherFunction/employee" }
  ],
  condition:
    [
      {
        data1: "sendNotificationOnSuccess/sendNotification",
        data2: true,
        type: "boolean",
        operator: "==",
        combinedWith: "and",
      }
    ],
  then: [
    {
      function: {
        name: "notifyEmployee",
        type: "notifier",
        dataIns: [
            { name: "employeeID" , type: "string" , source: "sendNotificationOnSuccess/employeeID" }
          ],
        dataOuts: [
            { name: "sentTo" , type: "object" }
        ]
      }
    }
  ],
dataOuts: [
    { name: "notificationSentTo" , type: "object" , source: "sendNotification/sentTo,sendNotificationOnSuccess/defaultObject" }
  ],
}

Sequential Loop

While

The while compound function is used to execute a loopBody zero or more times, depending on the specified condition. For all condition evaluations which are based on numbers, it is possible to enter a reference to a collection as one of the terms. A collection entry referencing the collection collection is then evaluated as a number equal to collection.size() - 1.
Since any data which remains unchanged over the course of the while iterations can be directly references, the dataIns and dataOuts of a while compound are used exclusively to refer to data which changes between subsequent iterations. This means that for each dataIn with a certain name, there must be a dataOut with the same name. The resolution of a reference pointing to the dataIn/out of the while from within a while compound depends on the iteration of the while: Any reference to a dataIn/out of a while compound from outside the compound is resolved as if pointing to the dataOut of the compound. Apart from that the condition has the same structure as in the if-then-else compound.
The loopBody will be executed until the specified condition evaluates to false.
The counter is a special data node which keeps track of the current iteration number. Within the workflow yaml file, the counter of the while compound named name can be referenced via name/counter. 
while: {
  name: "name",
  dataIns: [{}+]?,
  condition: [{}+],
  loopBody: [{function: {}}+],
  dataOuts: [{}+]?
}

Example

The following example shows the while construct in AFCL. The construct will execute the function specified in the loopCounter five times, as specified in condition. The input to the increment function is the output of the function from the previous iteration. The input for the first iteration is specified in the dataIn of the while construct, while the dataOut of the construct specifies the result of the function from the last iteration. 
while: {
  name: "while",
  dataIns:
      [ { name: "sum", type: "number", source: "workflow/input" } ],
  condition:
      [ { data1: "while/counter" , data2: 5 , operator: "<" ,  type: "number" , negation: "false" , combinedWith: "and" } ],
  loopBody: [
    function: {
        name: "increment",
        type: "Addition",
        dataIns: [
            { name: "firstSummand" , type: "number" , source: "while/sum" },
            { name: "secondSummand" , type: "number" , source: 1 },
            { name: "waitTimeIn" , type: "number" , source: 1000 }
        ],
        dataOuts: [
            { name: "sum" , type: "number" }
        ]
      }
  ],
  dataOuts:
        [ { name: "sum", type: "number", source: "increment/sum" } ]
}


ParallelFor

The parallelFor compound function expresses the simultaneous execution of all loop iterations. It is assumed that there are no data dependencies across loop iterations. Instead of the loopCounter, the distribution of given arrays across multiple parallel executions of the same function is specified via the iterators entry. The value of iterators is a list containing either
  • (a) the names of one or multiple dataIns of the parallelFor compound or
  • (b) a single integer (possibly specified via the corresponding dataOut of a function preceding the parallelFor).
In case (a), the dataIns specified in the iterators list must be of type array. In case of multiple entries, the arrays must have the same size. In this case, the content of the parallelFor compound will be executed once for each element of the iterator arrays. In each parallel execution, any function that specified an iterator dataIn as its dataIn will be supplied with exactly one element of the array.
Case (b): In cases where the iterators entry is a single integer (e.g., 5), the contents of the parallelFor compound will be executed for this number of times with the same input. An example usage would be a MonteCarlo simulation which is to be executed for a given number of times.
In both cases (a) and (b), the dataOuts of the parallelFor pointing to a dataOut of any function from within the parallelFor body is created as an array of the outputs of the functions executed in parallel. 
parallelFor: {
  name: "name",
  dataIns: [{}+]?,
  iterators: [iterators],
  loopBody: [{function: {}}+],
  dataOuts: [{}+]?
}

Examples

The following examples show the usage of a parallelFor in AFCL. 
parallelFor: {
  name: "exampleParallelFor",
  dataIns: [
    { name: "dataIn1", type: "number", source: 3 },
    { name: "dataIn2", type: "array", source: [1,2,3,4] },
    { name: "dataIn3", type: "array", source: ["a", "b", "c", "d"] }
  ],
  iterators: [Q], # Q is a variable - see below
  loopBody: [
    {
      function: {
        name: "executedInParallel", type: "exampleType2",
        dataIns: [
          {
            name: "parDataIn1", type: X, # X is a variable - see below
            source: "exampleParallelFor/dataIn1"
          },
          {
            name: "parDataIn2", type: Y, # Y is a variable - see below
            source: "exampleParallelFor/dataIn2"
          },
          {
            name: "parDatIn3", type: Z, # Z is a variable - see below
            source: "exampleParallelFor/dataIn3"
          },
        ],
        dataOuts: [ { name: "otherIntResult", type: "number" } ]
     }
   }
  ],
  dataOuts: [
    {
      name: "parForResult", type: "array",
      source: "executedInParallel/otherIntResult"
    }
  ]
}
  • Iterating over a single array 
    Q = "dataIns2"
X = number
Y = number
Z = array
    Specifying the name of an array dataIn (in this case, "dataIn2") leads to n parallel executions, with n being the number of the elements in the array (4 in this case).

    Inputs: The four parallel iterations of the "executedInParallel" function will get the following inputs:
    • Iteration 0:
      • "parDataIn1" : 3
      • "parDataIn2" : 1
      • "parDataIn3" : ["a","b","c","d"]
    • Iteration 1:
      • "parDataIn1" : 3
      • "parDataIn2" : 2
      • "parDataIn3" : ["a","b","c","d"]
    • Iteration 2:
      • "parDataIn1" : 3
      • "parDataIn2" : 3
      • "parDataIn3" : ["a","b","c","d"]
    • Iteration 3:
      • "parDataIn1" : 3
      • "parDataIn2" : 4
      • "parDataIn3" : ["a","b","c","d"]
    Output: Assuming that Res(n) denotes the result of the n-th iteration (with the inputs as specified above) of the "executedInParallel" function, the output (accessible as the dataOut of the parallelFor compound vie the source "exampleParallelFor/parForResult") will be a array with the content [Res(0), Res(1), Res(2), Res(3)].

  • Iterating over multiple arrays 
    Q = "dataIns2", "dataIns3"
X = number
Y = number
Z = string
    Specifying the name of multiple array sataIns (in this case, "dataIn2" "and dataIn3") leads to n parallel executions, with n being the number of the elements in each array (4 in this case - an exception is thrown in cases where the provided arrays have different size).

    Inputs: The four parallel iterations of the "executedInParallel" function will get the following inputs:
    • Iteration 0:
      • "parDataIn1" : 3
      • "parDataIn2" : 1
      • "parDataIn3" : "a"
    • Iteration 1:
      • "parDataIn1" : 3
      • "parDataIn2" : 2
      • "parDataIn3" : "b"
    • Iteration 2:
      • "parDataIn1" : 3
      • "parDataIn2" : 3
      • "parDataIn3" : "c"
    • Iteration 3:
      • "parDataIn1" : 3
      • "parDataIn2" : 4
      • "parDataIn3" : "d"
    Output: Assuming that Res(n) denotes the result of the n-th iteration (with the inputs as specified above) of the "executedInParallel" function, the output (accessible as the dataOut of the parallelFor compound vie the source "exampleParallelFor/parForResult") will be a array with the content [Res(0), Res(1), Res(2), Res(3)].

  • Given Number of parallel iterations 
    Q = "dataIns1"
X = number
Y = array
Z = array
    Specifying an integer n - possibly by the source of the dataOut producing an integer or an integer dataIn of the parallelFor compound - as the single (!) entry of the "iterators" list results in the parallel execution of n iterations of the parallelFor body, each with the input specified by their dataIns:

    Inputs: The three parallel iterations of the "executedInParallel" function will get the following inputs:
    • Iteration 0:
      • "parDataIn1" : 3
      • "parDataIn2" : [1,2,3,4]
      • "parDataIn3" : ["a","b","c","d"]
    • Iteration 1:
      • "parDataIn1" : 3
      • "parDataIn2" : [1,2,3,4]
      • "parDataIn3" : ["a","b","c","d"]
    • Iteration 2:
      • "parDataIn1" : 3
      • "parDataIn2" : [1,2,3,4]
      • "parDataIn3" : ["a","b","c","d"]
    Output: Assuming that Res(n) denotes the result of the n-th iteration (with the inputs as specified above) of the "executedInParallel" function, the output (accessible as the DataOut of the parallelFor compound vie the source "exampleParallelFor/parForResult") will be a array with the content [Res(0), Res(1), Res(2)].

  • Iterating over a modified array
    By specifying one (or multiple concatenated) operations for the modification of arrays (block, replicate, split, and element-index), in the dataIn of the parallelFor compound, it is possible to modify an array before it is used as an iterator. For example, the application of a block(2,1) operation to the array arrayResult1 is specified by adding the corresponding constraint to the corresponding dataIn of the parallelFor compound: 
    # same as in example above
{
  name: "dataIn2", type: "array",
  source: "exampleFunction/arrayResult1",
  constraints: [ { name: "block", value: "2,1" } ]
}
# same as in example above
    Q = "dataIns2"
X = number
Y = array
Z = array

    Inputs: The three parallel iterations of the "executedInParallel" function will get the following inputs:
    • Iteration 0:
      • "parDataIn1" : 3
      • "parDataIn2" : [1,2]
      • "parDataIn3" : ["a","b","c","d"]
    • Iteration 1:
      • "parDataIn1" : 3
      • "parDataIn2" : [2,3]
      • "parDataIn3" : ["a","b","c","d"]
    • Iteration 2:
      • "parDataIn1" : 3
      • "parDataIn2" : [3,4]
      • "parDataIn3" : ["a","b","c","d"]
    Output: Assuming that Res(n) denotes the result of the n-th iteration (with the inputs as specified above) of the "executedInParallel" function, the output (accessible as the dataOut of the parallelFor compound vie the source "exampleParallelFor/parForResult") will be a array with the content [Res(0), Res(1), Res(2)].

Workflow file structure

The final structure of an AFCL .yaml file looks as follows: 
{
 name: "name",
 dataIns: [{}+]?,
 workflowBody: [{function: {}}+],
 dataOuts: [{}+]?
}

Special characters

AFCL introduces several characters with special meaning:
  • The backslash character ("/") is used to specify the entry of the source field.
  • The comma symbol (",") is used to express a list of possible outputs specify in the source field.
These characters should only be used if their actual purpose is intended. E.g. the function name field should not contain any of these characters.

Properties and Constraints

Properties and constraints are optional attributes, which provide additional information about dataIn ports, dataOut ports, and base and compound functions. Properties can be used to describe hints about the behavior of functions, e.g. expected size of input data or memory required for execution. Constraints (e.g. finish execution time within a time limit, data distributions, fault tolerance settings) should be fulfilled by the runtime system on a best-effort basis. AFCL introduces built-in constraint (dataflow) to specify how data is gathered from or distributed among multiple functions, and built-in constraint element-index to specify a subset of a data array. 
function: {
  name: "name",
  type: "type",
  dataIns: [
    {
      name: "name" , type: "type" , source: "source" , 
      properties: [{ name: "name" , value: "value" }+]?,
      constraints: [{ name: "name" , value: "value" }+]?
    }+
  ]?,
  properties: [{ name: "name" , value: "value" }+]?,
  constraints: [{ name: "name" , value: "value" }+]?,
  dataOuts: [
    {
      name: "name" , type: "type"
      properties: [{ name: "name" , value: "value" }+]?,
      constraints: [{ name: "name" , value: "value" }+]?
    }+
  ]?
}

Dataflow

Most FC systems offer basic support to express data-flow within an FC, primarily by storing outputs of functions to a variable which can be used as input to other FC functions. AFCL supports various constructs to express more complex data-flow scenarios, which can also improve the performance of the resulting FC. The data-flow in AFCL is expressed by connecting source data ports to sink data ports of functions. A source data port can be an input data port for the entire FC, for a compound function, or for an output data port of a base function. A sink data port can be an output data port of the entire FC, an output data port of a compound function, or an input data port of a base function. When a source data port is connected to a sink data port dataflow, the data produced at the source data port will be available at the sink data port at runtime when the data is to be consumed. One source data port may have multiple sink data ports, in which case each sink data port will receive a copy of the data produced at the source data port.

AFCL allows the application developer to describe how data flows from dataOut ports of one or multiple functions into a single dataIn port of subsequent functions. Since AFCL supports nesting of compound functions, we also support data-flow from dataIn ports of a parent compound function to the dataIn ports of inner functions. For every function in AFCL, it must be guaranteed that whenever the control-flow reaches the function, all the dataIn ports of the function have been assigned well defined values. When the control-flow leaves a function that is invoked synchronously, all its dataOut ports must be well-defined, as well. Otherwise, for a function that is invoked asynchronously, the developer is responsible to synchronize data. For functions with basic control-flow, this is straightforward. In the following sections, we describe more complex data-flow scenarios.

Many real world FCs may operate on datasets instead of on single data elements. Furthermore, there are numerous cases where it makes sense to collect data elements from the output of a set of functions and include them in a array (e.g. a consumer of message queues or stream-processing tools) for further processing by subsequent functions. Arrays are also well suited to exploit data parallelism by distributing array data elements to loop iterations or parallel sections. Arrays may contain a static or dynamic number (unknown at the time when an FC is composed but not yet executed) of data elements. In order to support this feature as part of AFCL, we introduce the concept of a data array . The elements of a data collec- tion are of JSON datatypes and they can be distributed onto base and compound functions. The data port with type array represents a list of data elements provided by the user as the ini- tial input of an FC or produced by FC functions as an intermediate result

ELEMENT_INDEX

Subsets of data arrays can be specified by using the build-in constraint element-index. With index, the user can specify certain positions of the data array. 
constraits: [
    { name: "element-index" , value: "<index>" }
]
The value of element-index is a list of comma separated expressions. Note that in the absence of the type element-index, the entire data array is specified. The syntax of the element-index constraint is based on the "list slicing" in Python. The following grammar specifies the syntax of the construct element-index, where e denotes the element index, c a colon expression, s1 the start index, s2 the end index, and s3 a stride.
e ::= c [, c ]∗
c ::= s1 [: [s2] [: [s3] ]]
Such an expression can refer to either a specific index or a range of indexes with an optional stride.

Examples

  • 1,2:6:2: 
    dataIns: [
  {
    name: "elementIndexDataIn" , type: "array" ,
    source: [0,1,2,3,4,5,6,7],
    constraints: [ { name: "element-index", value: "1,2:6:2" } ]
  }
]
    element-index
    Result: [1,2,4,6]

  • 2: 
    dataIns: [
  {
    name: "elementIndexDataIn" , type: "array" ,
    source: [0,1,2,3,4,5,6],
    constraints: [ { name: "element-index", value: "2" } ]
  }
]
    Result: 2 of type number.

Distribution constraints

Most existing FC systems mainly replicate collected data to multiple functions or loop iterations without supporting the concept of data distributions. This inefficient data transfer between functions may initiate a considerable delay in function invocation time. AFCL offers additional, build-in, distribution constraints, which allow the developer to specify how data elements of a array will be distributed across successor functions. AFCL introduces the follwing distribution constraints: block, replicate and split.

BLOCK

Using the built-in constraint block and specifying the block-based data distribution, i.e. size and optional overlap, a data array is partitioned in contiguous blocks of a specific length.
  • Input: an array.
  • Output: an array, where each element is itself an array.
  • Parameters:
    • size: (integer bigger than 1) The number of the elements of the input array which are summarized to one element of the output array. Note that, depending on the chosen size parameter and the size of the processed array, there can be cases where the last element of the output array will contain less than size elements (see examples below).
    • overlap: : (non-negative integer) The number of shared elements between two subsequent elements of the output array.
constraits: [
    { name: "block" , value: "<size>, <overlap>" }
]

Examples

  • 4,2: 
    dataIns: [
  {
    name: "blockDataIn" , type: "array" ,
    source: [0,1,2,3,4,5,6,7],
    constraints: [ { name: "block", value: "4,2" } ]
  }
]
    Italian Trulli
    Result: [[0,1,2,3],[2,3,4,5],[4,5,6,7]]

  • 2,0: 
    dataIns: [
  {
    name: "blockDataIn" , type: "array" ,
    source: [0,1,2,3,4,5,6],
    constraints: [ { name: "block", value: "2,0" } ]
  }
]
    Result: [[0,1],[2,3],[4,5],[6]]. The last entry of the output array contains less than 2 elements, since it is not possible to include it in a entry with 2 elements, while respecting both the size and the overlap.

REPLICATE

Another option is to replicate a certain dataOuts port and then distribute to the different successor functions. AFCL allows data replication by specifying the constraint replicate in combination with the number of replications (times).
  • Input: (a) a single element or (b) an array of elements.
  • Output: an array created by replicating, i.e., creating additional copies of (a) the single given element or (b) each element of the provided array.
  • Parameters:
    • times: (integer bigger than 1) The number of times that each element of the input array (or the single input element) have to be replicated to create the output array.
constraits: [
    { name: "replicate" , value: "<times>" }
]
The elements of a data array will be replicated a specific number of times.

Examples

  • 2: 
    dataIns: [
  {
    name: "replicateDataIn" , type: "array" ,
    source: [0,1,2,3],
    constraints: [ { name: "replicate", value: "2" } ]
  }
]
    Italian Trulli
    Result: [0,0,1,1,2,2,3,3]

  • 5: 
    dataIns: [
  {
    name: "replicateDataIn" , type: "number" ,
    source: 3,
    constraints: [ { name: "replicate", value: "5" } ]
  }
]
    Result: [3,3,3,3,3]

SPLIT

The split operation enables to split an array into a given number of subarrays.
  • Input: an array.
  • Output: an array, where each element is a subarray of the input array (splitting an array in 2 results in an array with 2 elements, each containing one half of the entries of the input array, see examples below).
  • Parameters:
    • split-number: (integer bigger than 1) The number of parts that the input array is to be split into. Each element - potentially with the exception of the last one - of the output array will contain n elements, with n=⌈I/s⌉( I : size of the input array, s : split number). Each element of the output array contains at least one element of the input array. In cases where the provided split number is larger than the size of the input array, the output array will be identical to the input (see examples below).
constraits: [
    { name: "split" , value: "<split-number>" }
]

Examples

  • 2: 
    dataIns: [
  {
    name: "replicateDataIn" , type: "array" ,
    source: [0,1,2,3,4,5,6,7],
    constraints: [ { name: "split", value: "2" } ]
  }
]
    Italian Trullis
    Result: [[0,1,2,3],[4,5,6,7]]

  • 2: 
    dataIns: [
  {
    name: "replicateDataIn" , type: "array" ,
    source: [0,1,2,3,4,5,6],
    constraints: [ { name: "split", value: "2" } ]
  }
]
    Result: [[0,1,2,3],[4,5,6]] (uneven split).

  • 3: 
    dataIns: [
  {
    name: "replicateDataIn" , type: "array" ,
    source: [0,1,2,3,4,5,6],
    constraints: [ { name: "split", value: "3" } ]
  }
]
    Result: [[0,1,2],[3,4,5],[6]] (uneven split).

  • 8: 
    dataIns: [
  {
    name: "replicateDataIn" , type: "array" ,
    source: [0,1,2,3,4,5,6],
    constraints: [ { name: "split", value: "8" } ]
  }
]
    Result: [[0],[1],[2],[3],[4],[5],[6]] (split number too large).

Info! The usage of block, replicate, split and element-index is not limited to the dataIns of the parallelFor compound, but can instead be applied on any dataIn referring to (a) an array in case of block, split and element-index or (b) an array or an element in case of replicate.

Concatenating Multiple Operations

The dataIn operations element-index, block, replicate and split can be concatenated by annotating multiple of these operations in the constraints entry of the dataIn. It is also possible to apply the same operation, e.g., element-index, multiple times. The operations are then applied in the same order as they are specified in the constraints entry of the dataIn.
Note that, during the formulation of a concatenation of these operations, it is, for each pair of subsequently applied operations, necessary to ensure the correct relation of the output of the first operation to the input of the second operation. For instance, in the example below, applying the element-index before the split operation could result in a run-time error in cases where the result of the element index operation is a single element and not an array.
  • Input: same as the input of the dataIn operation which is specified first.
  • Output: same as the output of the dataIn operation which is specified last.
  • Parameters:
    • Each dataIn operation is annotated with the same parameters as in cases without concatenation.

Example

dataIns: [
  {
    name: "concatenationDataIn" , type: "array" ,
    source: [0,1,2,3,4,5,6],
    constraints: [ 
      { name: "split", value: "3" },
      { name: "element-index", value: "0" } 
    ]
  }
]
Result: [0,1,2] (In this case, the concatenation is used to obtain the first third of the array)).

Object access

AFCL introduces a way to access only parts of a json-object. In order to do so, the user accesses sub-objects via the / special character. It is necessary that the user knows the structure of the json-object.

Example

Let us assume that the function getEmployee has the following json object as output, named employee: 
{
    "employee": {
        "employeeID": "AH6FT8Z",
        "address": {
            "street": "Technikerstraße 21",
            "city": "Innsbruck",
            "state": "Austria"
        }
    }
}
The function checkState needs as input only part of the json object from the function getEmployee. The reference to the corresponding part of the object is expressed as getEmployee/employee/address/state in the source attribute of the checkState input. The function checkEmployee needs again the whole object from getEmployee as input. 
sequence: {
  name: "addEmployeeToDataBase",
  sequenceBody: [
    {
      function: {
        name: "getEmployee",
        type: "returnEmployee",
        dataOuts: [ { name: "employee" ,  type: "object" } ]
      }
    },
    {
      function: {
        name: "checkState",
        type: "checkEmployeeState",
        dataIns: [ { name: "state" , type: "string" , source: "getEmployee/employee/address/state" } ],
        dataOuts: [ { name: "status" ,  type: "boolean" } ]
      }
    },
    {
      function: {
        name: "checkEmployee",
        type: "compareWithDatabase",
        dataIns: [
         { name: "employeeID" , type: "boolean" , source: "checkState/status" },
         { name: "employee" , type: "object" , source: "getEmployee/employee" }
        ]
      }
    }
  ]
}

Illegal dataflow

AFCL allows a developer to set the source of dataIns port of a function to the dataOut of a data port of any other function. The following figure shows such data-flow from function f1 to function f4. However, data-flow is challenging for conditional compound functions where not all control-flow branches are executed at runtime. Therefore, we have to prevent that a successor function f4, outside of a conditional compound function, reads data from dataOuts of any inner function f2 or f3 as one of these functions may never execute. In order to prevent this illegal case, any outside successor function can only read data from dataOuts of the whole conditional compound, which will always be defined. A dataOuts port of a conditional compound has a source value with a comma separated list of dataOut ports of other functions. This entry must contain one element for each possible branch within the compound construct.
illegal dataflow
Info! The dataIn from a function inside a compound can access data from a predecessor function outside of that compound. A function after a compund cannot access data from a function inside of that compound. It is necessary to access the data via the dataOuts of the compound. A function can therefore only access data from another function with the same or higher hierarchy.

Distributed and Parallel Systems  Group   -

University of Innsbruck