Data Analytics Platform

This layer exposes the interfaces for the deployment of big-data analytics methods. Analytics workloads in CLASS are intended to be wrapped as a set of OpenWhisk actions (functions), which are pieces of logic that can be explicitly invoked or executed in response to events.

EXtended PREdictability ServerlesS (EXPRESS) is a serverless cloud platform, based on Apache OpenWhisk, that executes functions (called actions) in response to events, at scale, and with enhancements for predictable computation.

Link: https://github.com/class-euproject/express

This layer also exposes a serverless-based map/reduce engine, called PyWren, that executes user’s Python code and its dependencies on serverless computing platforms. Map/Reduce is a generic API, originally from Google and later evolved publically, for processing a large amount of data where each data element or record is processed in a similar way, such as applying a mathematical operation. Naturally, map/reduce can scale out, and efficiently apply parallel processing techniques for performing its computation on a data set. In CLASS, both PyWren and COMPSs support map/reduce, while augmenting it with latency awareness for real-time capabilities.

Link: https://github.com/class-euproject/pywren-ibm-cloud

Moreover, the data analytics tooling includes evaluation capabilities. Owperf is used as a performance evaluation tool for OpenWhisk-based deployments. Combined with an extended monitoring infrastructure, it can provide deep feedback about performance of complex applications.

Link: https://github.com/class-euproject/owperf

 

Order: 
1