Full-Stack, Intelligent DataOps Platform
Composable enterprise is a full-stack intelligent dataops platform with built-in services for data orchestration, automation and analytics. It accelerates data engineering, preparation and analysis.
The main capabilities offered by Composable Enterprise is an environment for discovering data and applications, creating new methods, algorithms and rules, and collaborating through the sharing of data, results, and applications. Users therefore have a single platform for “full stack analytics,” seamlessly executing all levels of data processing: Extraction, Transformation, and Loading (ETL); Querying, Processing, Visualization and Dissemination. Composable Enterprise acts as the underlying “glueware,” providing the necessary integration across the environment, as well as the “presentation layer,” providing an operational analytics workbench on top of the environment. This environment, the enterprise data lake for storage and compute, may include relational databases, NoSQL databases, high-performance data stores, cloud warehouses, analytic frameworks, coding languages (Python, R, SAS, MATLAB, …), etc.
Several operational benefits of Composable Enterprise include:
• Repeatable Ingestion Patterns
Robust and fault tolerant enterprise-grade platform built for bringing together disparate datasets, disseminating results, and producing data feeds/services.
• A true Data Catalog, serves as a “Marketplace” for both Data and Analytics
Data, as well as Analytics, can now be shared, executed and cloned by others within an organization or across organizations.
• Infrastructure agnostic, with minimal deployment costs
No monolithic data model or warehouse required; data lake ready.
• Analytics as operational capabilities
Infinitely configurable, continuous analytics that is pluggable, agile and extensible; centralized management with full audit trails.
Rich, Web-based UI
The platform provides a rich web browser-based user experience for Search & Discovery, Creation and Authoring, Analysis and Visualization, Reporting and Collaboration. Current capabilities of the Composable Enterprise platform include Search & Discovery, Creation and Authoring, Analysis and Visualization, Reporting and Collaboration.
Custom Workflow Interfaces and Web Applications
In addition to a rich front-end UI already built in, Custom Front-End Interfaces and Web Applications can be developed and seamlessly integrated on top of the Composable Enterprise run-time, using well-defined web services and API endpoints. Composable Enterprise leverages the d3.js library for data visualizations.
Custom Interfaces, including visualizations built with d3.js, can be built and seamlessly integrated with Composable Enterprise using well-defined web services and API endpoints.
DataFlows: Event-Driven Analytical Workflows
Business and analytical workflows in Composable Enterprise are represented as event-driven workflows, or DataFlows, using an intuitive, flow-based programming method. Each dataflow consists of modules that are connected together to produce higher-level functionality. A module is an atomic processing element with strongly typed inputs and outputs. All information required for a module to execute is retrieved from its inputs through connections. Modules can be reused easily and interchanged with other modules.
Analysts can compose unique dataflow applications by dragging and dropping modules and connecting them together in an “infinitely configurable” modular design. Composable Enterprise is designed to handle all data types, and is flexible regarding the consumption of all data sources.
In addition to the graphical web user interface available to all users (business, technical, IT, admins, developers, etc.), Composable Enterprise has a FluentAPI suitable for developers, IT admins and other more technical users who prefer a command line interface for creating, executing, and viewing the results of DataFlows in code. Composable Enterprise’s FluentAPI is a set of C# classes that interact with Composable Enterprise services in order to provide a versatile and lightweight method of using DataFlows.
The FluentAPI is divided into three main sections: the ModuleBuilder, the ModuleInputBuilder, and the ApplicationInspector. At a high level, the ModuleBuilder is used to orchestrate ModuleInputs and ModuleOutputs, the ModuleInputBuilder is used to set those inputs and outputs, and the ApplicationInspector is used to run a dataflow application as well as view the outputs that has been run. The FluentAPI works in C# or F# running on .NET 4.5
Rapid Application Development
Composable Enterprise provides not only a complete portfolio of tools for processing, orchestrating, and analyzing complex data models, but also for automatically building a full, responsive, performant, and secure CRUD (create, read, update and delete) user interface for viewing, searching, and changing information.
Composable DataPortals automatically performs the required steps needed to build a full CRUD web UI, including:
1. Building the database itself, with the tables and relationships required to represent the desired data model (specified in a Microsoft Excel worksheet)
2. Implementing an Object-Relational Mapping (ORM) layer to interact with the database
3. Defining & parameterizing queries for the CRUD operations
4. Building out web services to allow a frontend application to manipulate the data model via an API
Rapid Application Development in Composable Enterprise allows users to start with a data model defined in a spreadsheet and automatically generate the web and business layers and custom web user interface in minutes.
In Composable Enterprise, users (regardless of SQL knowledge) can set up tabular views of any data set using the QueryView functionality. QueryView provides the ability to create an interface that allows users to search even huge datasets for records of interest. QueryViews can also be connected into dataflow applications to perform actions based on any number of selected records.
Composable Enterprise’s visual QueryViews allow users to construct custom database queries and interfaces.
Data scientists and analysts are typically given massive amounts of data without much of a data dictionary. Composable Enterprise DataExplorer feature allows users to connect to data repositories to visualize relationships and create a collection of reusable queries and views of the data.
DataExplorer in Composable Enterprise allows users toconnect to a data repository, visualize relationships, explore the data and save a collection of reusable queries.
Private or Shared connection strings can be configured in Composable Enterprise. This allows users to essentially configure ‘environment variables’ that can remain persistent, re-used, and shared, across other DataFlows and users. These shared properties removes the requirements to re-configure connection settings for each individual task.
Composable Enterprise allows users to define and configure connection settings (e.g., to databases) just once, and allow these properties to be used across all resources and shared across users and groups. Using a property entry, a user can configure and save all database configuration and connection strings.
Machine Learning in Composable
Composable Enterprise provides a complete framework for building production-grade machine learning applications.