AI-Powered Business Apps &
Enterprise Performance Management
on the Lakehouse: A Unified Approach

Bala Amavasai

CTO for Industry, Celebal Technologies

Anupam Gupta

Co-Founder, Celebal Technologies

Dael Williamson

CTO for EMEA, Databricks

Celebal Technologies, Sigma and Databricks have developed cutting-edge business applications that streamline Enterprise Performance Management (EPM). These applications work directly with data in a client's existing data Lakehouse, leveraging the power of AI and machine learning for tasks like multi-model forecasting and complex problem-solving. With a focus on user experience, these applications incorporate features like Excel-like interfaces and Teams integration to encourage adoption. Built on scalable and secure cloud-native infrastructure, these applications can easily adapt to a business's changing needs. Flexible business models and GenAI assistance for data understanding make these EPM solutions highly effective.

Modern enterprises require modern platforms and tooling

Organisations face a relentless wave of challenges in the fast-paced world of modern business. From fluctuating market conditions and shifting customer expectations to the emergence of disruptive Industry 4.0 technologies, the need for agility and adaptability has never been more critical. Enter Enterprise Performance Management (EPM), a strategic framework businesses use to navigate this complexity and achieve sustainable success.

Today, many organisations face significant hurdles in achieving effective EPM hindering their ability to adapt and thrive. Key challenges include:

  • Siloed Planning Applications: Departments often rely on separate planning tools and data, resulting in a fragmented view of the organisation. This leads to overly summarised information that obscures critical details and nuances. In addition, individual departments prioritise their own goals, potentially overlooking broader organisational needs.
  • Rigid and Manual Processes: Traditional planning cycles are often time-consuming, and manual data entry and reconciliation are labour-intensive. Annual planning cycles struggle to accommodate unexpected market shifts or changing conditions.

These limitations create brittle organisations that struggle to respond effectively to dynamic environments, ultimately impacting their competitiveness and resilience.

In the parallel AI-driven world, enterprises constantly seek ways to extract maximum value from their ever-growing data assets. Traditional data architectures, with their inherent limitations, often struggle to keep pace. Enter the Databricks Data Intelligence Platform, a modern Lakehouse approach that combines the best of data warehouses and data lakes to deliver a unified, governed, cost-effective, and scalable solution.

In this blog post, we will explore how consolidating EPM applications onto the Data Intelligence Platform and surfacing insights through software technologies like Sigma Computing reduces complexity, decreases costs and drives performance forward.

The Data Intelligence Platform Advantage

The Databricks Data Intelligence Platform offers a robust framework for consolidating diverse data types, from raw unstructured data to structured tables, within a unified platform. This "Lakehouse" architecture transcends traditional divisions between data lakes and data warehouses, empowering organisations to implement consistent governance policies and security measures across their entire data estate. Leveraging open-source components, the platform avoids vendor lock-in, providing flexibility and interoperability.

By combining the flexibility of data lakes with the reliability and performance of data warehouses, Databricks facilitates a collaborative environment where data consumers can leverage AI/ML for advanced analytics. At the same time, business intelligence teams can perform reporting and dashboards, all working together seamlessly on the same data foundation. This unified approach accelerates innovation, improves data-driven decision-making, and fosters a truly data-centric culture.

Figure 1. The modern data estate (right) provides simplification, cost reduction and scalability over legacy estates (left), simplifying the delivery of EPM apps.

Enter Enterprise Performance Management (EPM)

Enterprise Performance Management (EPM) encompasses the processes and technologies that enterprises use to plan, budget, forecast, and report on their financial and operational performance. It provides a holistic view of the business, aligning strategy with execution and enabling informed decision-making.

Think of EPM as a comprehensive approach to aligning business strategy with execution. The key components are:

  • Strategic Planning: Defining long-term goals and objectives, and developing action plans to achieve them.
  • Financial Planning and Budgeting: Forecasting revenue, expenses, and cash flow to ensure financial stability and allocate resources effectively.
  • Operational Planning: Developing detailed plans for various business functions, such as sales, marketing, production, and supply chain.
  • Performance Management: Monitoring key performance indicators (KPIs) to track progress towards goals and identify areas for improvement.
  • Reporting and Analytics: Generating reports and dashboards to provide insights into business performance and support data-driven decision-making.

EPM is essentially about setting clear goals, tracking progress, and making data-driven and AI-driven decisions to optimise business performance. EPM is paramount to modern organisations as it

  • Improves Decision-Making: EPM provides real-time visibility into business performance, enabling leaders to make informed decisions based on accurate data.
  • Increases Agility: By connecting strategy with execution, EPM helps organisations respond quickly to market changes and seize new opportunities.
  • Enhances Collaboration: EPM fosters collaboration across departments, breaking down silos and promoting alignment towards common goals.
  • Reduces Risk: By identifying potential risks and opportunities, EPM enables proactive mitigation strategies and informed risk management.
  • Improves Efficiency: Automating manual processes and streamlining workflows, EPM frees up valuable time and resources.

Modernising and Transforming the Spreadsheet and BI Experience

Sigma Computing enhances data-driven decision-making by seamlessly integrating with Databricks, providing a powerful combination of capabilities without needing to make multiple copies of the data or semantics. Sigma's intuitive spreadsheet-like interface empowers business users of all technical levels to directly interact with massive datasets stored in Databricks, eliminating the need for complex coding. This democratises data access and enables teams to independently perform ad-hoc analyses and generate insights.

Furthermore, this integration strengthens security and compliance by running applications directly on a customer's Databricks instance. This ensures data remains within the organisation's own environment, inheriting Databricks' robust security protocols. By combining Sigma's user-friendly interface with Databricks' powerful data processing, organisations can foster a data-driven culture, accelerate decision-making, and unlock new opportunities for growth and efficiency.

Next-gen EPM with Celebal Technologies

Celebal Technologies is looking to revolutionise the EPM landscape by combining the power of Databricks with the user-friendliness and collaborativeness of Sigma Computing. At the heart of this solution is the Databricks Data Intelligence Platform, which provides a robust and scalable foundation for handling large volumes of business and financial data. Databricks excels at data processing, storage, and analysis, making it an ideal platform for demanding EPM workloads. However, to truly empower business users, an intuitive interface is needed to interact with this data.

This is where Sigma Computing comes in. Sigma offers a spreadsheet-like interface that is familiar and accessible to finance professionals. Users can easily explore data, build reports, and create dashboards without needing specialised technical skills or writing complex code. This democratises access to data and enables finance teams to perform ad-hoc analysis, gain valuable insights, and make informed decisions.

The EPM solutions developed by Celebal Technologies combine the strengths of both Databricks and Sigma Computing. By leveraging Databricks' powerful data processing capabilities and Sigma's user-friendly interface, Celebal has created a suite of solutions that streamlines planning, analysis, and reporting. This empowers organisations to improve accuracy, collaboration, and efficiency in their EPM processes.

Figure 2. The tripartite partnership of Celebal Technologies, Sigma Computing and Databricks delivers an end-to-end experience of business applications.

These innovative EPM business applications that Celebal Technologies has built strongly emphasise advanced analytics and AI integration. They are designed to work directly with data residing in a client's existing Lakehouse or data warehouse, eliminating the need for data movement and boosting efficiency. Leveraging the latest in AI and machine learning, these applications are capable of tasks like multi-model forecasting and tackling complex, high-dimensional problems.

To encourage widespread use, the design is focused on user-friendliness, incorporating features like Excel-like interfaces and Microsoft Teams integration. Built on robust, scalable and secure cloud infrastructure, these EPM applications effortlessly adapt to growing business needs. With evolving business models and data, the use of Databricks’ Mosaic AI GenAI assistance delivers highly customised solutions that empower businesses to optimise operations, make informed decisions, and drive innovation in today’s competitive landscape.

Figure 3. EPM applications developed by Celebal Technologies, powered by Databricks, and delivered through Sigma Computing. The familiar BI and spreadsheet interface empowers users at any level to directly interact with massive datasets stored in Databricks.

Figure 4. The ability to plan and simulate within the spreadsheet UI is paramount to delivering a full enterprise experience.

Conclusion

Integrating AI-powered business applications and EPM on the Databricks Data Intelligence Platform, coupled with advanced BI tools, offers enterprises a modern and efficient approach. This unified strategy streamlines operations, simplifies software architectures, cuts costs, and propels performance by nurturing a data-centric culture and enabling swift, informed decision-making. By embracing this integrated ecosystem, businesses can adeptly navigate today's complex landscape, driving innovation and securing a competitive edge in the data-driven era. The result is a robust foundation for sustainable growth, empowering organisations to thrive amidst the challenges and opportunities of their AI-powered future.